Monday, August 31, 2015


In this Post I'll explain you how to send a GZIPed post body using CURL commands. Its pretty straight forward, noting down if anyone come accross a similar requirement.

First you need to create the body you need to send, In my case I'm sending a XML content. So create a valid xml file, then zip the created file in gzip format, In ubuntu you can just right click on the file and compress it. After compressing the file will have a .gz extension.

e.g : test.xml.gz

Now navigate to the file location in terminal and execute the following command.

curl -v -s --trace-ascii http_trace.log --data-binary @test.xml.gz -H "Content-Type: text/xml" -H "Content-Encoding: gzip" -X POST http://10.100.5.112:8280/api01

After invoking everything will be recorded in the specified log file.

== Info:   Trying 10.100.5.112...
== Info: Connected to 10.100.5.112 (10.100.5.112) port 8280 (#0)
=> Send header, 155 bytes (0x9b)
0000: POST /api01 HTTP/1.1
0016: User-Agent: curl/7.35.0
002f: Host: 10.100.5.112:8280
0048: Accept: */*
0055: Content-Type: text/xml
006d: Content-Encoding: gzip
0085: Content-Length: 42
0099: 
=> Send data, 42 bytes (0x2a)
0000: .......U..test.xml..)I-.....6.`&..-..J....
== Info: upload completely sent off: 42 out of 42 bytes
<= Recv header, 17 bytes (0x11)
0000: HTTP/1.1 200 OK
<= Recv header, 24 bytes (0x18)
0000: Content-Type: text/xml
<= Recv header, 37 bytes (0x25)
0000: Date: Tue, 01 Sep 2015 06:40:03 GMT
<= Recv header, 28 bytes (0x1c)
0000: Transfer-Encoding: chunked
<= Recv header, 2 bytes (0x2)
0000: 
<= Recv data, 34 bytes (0x22)
0000: 17
0004: <abc>Created !!!!</abc>
001d: 0
0020: 
== Info: Connection #0 to host 10.100.5.112 left intact


In this post I'll explain how to monitor WSO2 products remotely using Jconsole. Jconsoles comes by default with Java in linux you can type Jconsole to start Jconsole.

WSO2 Product Configurations

Main JMX ports related configs are located at <Product_Home>/repository/conf/carbon.xml Open <CARBON_HOME>/repository/conf/carbon.xml and specify the ports if you need to change the default ports. Change the following section.

        <!-- The JMX Ports -->
        <JMX>
            <!--The port RMI registry is exposed-->
            <RMIRegistryPort>9999</RMIRegistryPort>
            <!--The port RMI server should be exposed-->
            <RMIServerPort>11111</RMIServerPort>
        </JMX>

Now open <CARBON_HOME>/repository/conf/etc/jmx.xml and make sure the configs are changed as follows, You need to specify the machine IP as the HostName.


<JMX xmlns="http://wso2.org/projects/carbon/jmx.xml">
    <StartRMIServer>true</StartRMIServer>

    <!-- HostName, or Network interface to which this RMI server should be bound -->
    <HostName>192.168.48.49</HostName>

    <!--  ${Ports.JMX.RMIRegistryPort} is defined in the Ports section of the carbon.xml-->
    <RMIRegistryPort>${Ports.JMX.RMIRegistryPort}</RMIRegistryPort>

    <!--  ${Ports.JMX.RMIRegistryPort} is defined in the Ports section of the carbon.xml-->
    <RMIServerPort>${Ports.JMX.RMIServerPort}</RMIServerPort>
</JMX>

Now open <CARBON_HOME>/bin.wso2server.sh and add the following line as a JAVA parameter.

 -Djava.rmi.server.hostname="192.168.48.49"

You should add the above as shown below. Note: I'm using ESB 4.9 to demonstrate this, so your parameters may differ from what I have mentioned.


while [ "$status" = "$START_EXIT_STATUS" ]
do
    $JAVACMD \
    -Xbootclasspath/a:"$CARBON_XBOOTCLASSPATH" \
    -Xms256m -Xmx1024m -XX:MaxPermSize=256m \
    -XX:+HeapDumpOnOutOfMemoryError \
    -XX:HeapDumpPath="$CARBON_HOME/repository/logs/heap-dump.hprof" \
    $JAVA_OPTS \
    -Dcom.sun.management.jmxremote \
    -classpath "$CARBON_CLASSPATH" \
    -Djava.endorsed.dirs="$JAVA_ENDORSED_DIRS" \
    -Djava.io.tmpdir="$CARBON_HOME/tmp" \
    -Dcatalina.base="$CARBON_HOME/lib/tomcat" \
    -Dwso2.server.standalone=true \
    -Dcarbon.registry.root=/ \
    -Djava.command="$JAVACMD" \
    -Dcarbon.home="$CARBON_HOME" \
    -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager \
    -Dcarbon.config.dir.path="$CARBON_HOME/repository/conf" \
    -Djava.util.logging.config.file="$CARBON_HOME/repository/conf/etc/logging-bridge.properties" \
    -Dcomponents.repo="$CARBON_HOME/repository/components/plugins" \
    -Dconf.location="$CARBON_HOME/repository/conf"\
    -Dcom.atomikos.icatch.file="$CARBON_HOME/lib/transactions.properties" \
    -Dcom.atomikos.icatch.hide_init_file_path=true \
    -Dorg.apache.jasper.compiler.Parser.STRICT_QUOTE_ESCAPING=false \
    -Dorg.apache.jasper.runtime.BodyContentImpl.LIMIT_BUFFER=true \
    -Dcom.sun.jndi.ldap.connect.pool.authentication=simple  \
    -Dcom.sun.jndi.ldap.connect.pool.timeout=3000  \
    -Dorg.terracotta.quartz.skipUpdateCheck=true \
    -Djava.security.egd=file:/dev/./urandom \
    -Dfile.encoding=UTF8 \
    -Djava.net.preferIPv4Stack=true \
    -Dcom.ibm.cacheLocalHost=true \
    -DworkerNode=true \
    -Djava.rmi.server.hostname="192.168.48.49" \
    org.wso2.carbon.bootstrap.Bootstrap $*
    status=$?
done

Thats all from the WSO2 side, after the configurations are done you need to restart the server. If the RMI servers are started successfully you will see the following logs on server startup.

TID: [-1234] [] [2015-08-31 14:27:18,854]  INFO {org.wso2.carbon.core.init.JMXServerManager} -  JMX Service URL  : service:jmx:rmi://192.168.48.49:11111/jndi/rmi://192.168.48.49:9999/jmxrmi {org.wso2.carbon.core.init.JMXServerManager}

Connecting Via JConsole.

There are two ways to connect via JConsole remotely, using the IP and port [1] or using JMX service URL [2].

After starting Jconsole Select  Remote process. And add one of the following URLs

[1]    192.168.48.49:9999

[2]    service:jmx:rmi://192.168.48.49:11111/jndi/rmi://192.168.48.49:9999/jmxrmi


For the username and the password, enter the super user name and the password of the WSO2 server. The default username and password is admin/admin. The final configs will look like following.


Now click on connect, When clicked on connect it will give you and error saying secured connection failed, So you need to select insecure connection when this error pops up. If you need to securely connect to JConsole you need add necessary keystores etc.


If everything goes well you should connect to the RMI server and relevant graphs will be generated as shown below.


So that's it please drop a comment if you have any queries.

Friday, August 21, 2015

This post explain how to setup a WSO2 ESB cluster, here I will be using ESB 4.9, MySQL as the underline DB and NginX as the load balancer. Following diagram depicts the overall distribution of the cluster components.


Following are the host names IPs of each node.


NoNode NameHost NameIPLocal Member Port
1Manager mgt.esb.cl2.wso2.com  192.168.48.243  4001
2Worker 01esb.cl2.wso2.com192.168.48.2444001
3Worker 02esb.cl2.wso2.com192.168.48.2494001

Requisites.

So for the cluster I assume you have the following prerequisites fulfilled.

1. MySQL setup
2. NginX installed and configured
3. SVN server to use as the deployment synchronizer.

Note : I will be explaining how to setup the cluster in a distributed setup, meaning one instance will host one ESB instance. So we will not see any port conflicts. If you need setup ESB cluster in a single machine or want to host multiple instnces in a single machine  you need to offset each node to avoid port conflicts. To do that you can open <ESB_HOME>/repository/conf/carbon.xml  and change the following property.

 <Offset>0</Offset>

Also you need to specify unique local member ports for each node. This can be done in  <ESB_HOME>/repository/conf/axis2/axis2.xml under the following proerty

<parameter name="localMemberPort">4100</parameter>

Configuring the Manager Node

1. Open <ESB_HOME>/repository/conf/datasources/master-datasources.xml and add following two datasources. Do not remove the existing carbon datasource.

<datasource>
  <name>WSO2_REG_DB</name>
  <description>The datasource used for shared config and governance registry</description>
  <jndiConfig>
  <name>jdbc/WSO2SharedDB</name>
  </jndiConfig>
  <definition type="RDBMS">
  <configuration>
   <url>jdbc:mysql://192.168.48.6:3306/ESB49_REGDB</url>
   <username>esbuser</username>
   <password>wso2root</password>
   <driverClassName>com.mysql.jdbc.Driver</driverClassName>
   <maxActive>50</maxActive>
   <maxWait>60000</maxWait>
   <testOnBorrow>true</testOnBorrow>
   <validationQuery>SELECT 1</validationQuery>
   <validationInterval>30000</validationInterval>
  </configuration>
  </definition>
 </datasource>

<datasource>
  <name>WSO2_UM_DB</name>
  <description>The datasource used for registry and user manager</description>
  <jndiConfig>
  <name>jdbc/WSO2UmDB</name>
  </jndiConfig>
  <definition type="RDBMS">
  <configuration>
   <url>jdbc:mysql://192.168.48.6:3306/ESB49_USERDB</url>
   <username>esbuser</username>
   <password>wso2root</password>
   <driverClassName>com.mysql.jdbc.Driver</driverClassName>
   <maxActive>50</maxActive>
   <maxWait>60000</maxWait>
   <testOnBorrow>true</testOnBorrow>
   <validationQuery>SELECT 1</validationQuery>
   <validationInterval>30000</validationInterval>
  </configuration>
  </definition>
</datasource>

Note : Make sure you don't remove WSO2_CARBON_DB datasource. WSO2_CARBON_DB will be used as the local registry of each node which will point to the embedded H2 Database, WSO2_UM_DB will be used for user namagement and WSO2_REG_DB will be used as the shared registry among the cluster.

2. Now open <ESB_HOME>/repository/conf/carbon.xml

a. Change the following parameters to add the necessary host names

<HostName>esb.cl2.wso2.com</HostName>
<MgtHostName>mgt.esb.cl2.wso2.com</MgtHostName>

b. Inorder to set the Deploymeny syncronizer do the changes as shown below. Note that AutoCommit option is set to true in the manager node

<DeploymentSynchronizer>
        <Enabled>true</Enabled>
        <AutoCommit>true</AutoCommit>
        <AutoCheckout>true</AutoCheckout>
        <RepositoryType>svn</RepositoryType>
        <SvnUrl>http://192.168.42.273/svn/wso2qarepo/esb49repo</SvnUrl>
        <SvnUser>wso2user1</SvnUser>
        <SvnPassword>wso2pass123</SvnPassword>
        <SvnUrlAppendTenantId>true</SvnUrlAppendTenantId>
    </DeploymentSynchronizer>

3. Now open <ESB_HOME>/repository/conf/user-mgt.xml and change the following property as shown below.

<Property name="dataSource">jdbc/WSO2UmDB</Property>

Notejdbc/WSO2UmDB is the JNDI Name of the datasource we declaired in step 01 for user management DB.

4. Open <ESB_HOME>/repository/conf/registry.xml and add the following entries. Do not delete existing entries.

<dbConfig name="sharedregistry">
<dataSource>jdbc/WSO2SharedDB</dataSource>
</dbConfig>
<remoteInstance url="https://localhost:9444/registry">
<id>instanceid</id>
<dbConfig>sharedregistry</dbConfig>
<readOnly>false</readOnly>
<enableCache>true</enableCache>
<registryRoot>/</registryRoot>
<cacheId>esbuser@jdbc:mysql://192.168.48.6:3306/ESB49_REGDB</cacheId>
</remoteInstance>
<mount path="/_system/config" overwrite="true">
<instanceId>instanceid</instanceId>
<targetPath>/_system/esbnodes</targetPath>
</mount>
<mount path="/_system/governance" overwrite="true">
<instanceId>instanceid</instanceId>
<targetPath>/_system/governance</targetPath>
</mount>
5. Now lets do cluster related configs. Open <ESB_HOME>/repository/conf/axis2/axis2.xml do the following changes.

a. Enable clustering as shown below. make enable true in the following property.

<clustering class="org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent" enable="true">

b.  Make sure that membershipScheme is set to wka (well known address based schema)

<parameter name="membershipScheme">wka</parameter>

c. Set the domain, all cluster nodes will join this domain. This is only used for the members to join the cluster.

<parameter name="domain">wso2.esb49.cl2.domain</parameter>

d. Set the local member host, add the IP of the manager node to this.

<parameter name="localMemberHost">192.168.48.243</parameter>

e. Specify the localmember port to 4100.

Note : if you are hosting all the nodes in the same machine each node should have a unique member port.

<parameter name="localMemberPort">4100</parameter>

f. Now lets add  well known members.

<members>
     <member>
        <hostName>192.168.48.244</hostName>
        <port>4100</port>
     </member>
</members>

Note : Here I'm specifying worker 01 as the well-known member, ideally you can specify all the nodes as well known members. But its optional. When a node goes down the well known address will be used to reconnect to the cluster by the disrupted node. 

g. Uncomment and edit the WSDLEPRPrefix element under org.apache.synapse.transport.passthru.PassThroughHttpListener and
org.apache.synapse.transport.passthru.PassThroughHttpSSLListener in the transportReceiver. Each specifies HTTP and HTTP endpoints for the WSDL url.


<parameter name="WSDLEPRPrefix" locked="false">https://esb.cl2.wso2.com:443</parameter>
<parameter name="WSDLEPRPrefix" locked="false">http://esb.cl2.wso2.com:80</parameter>

6. Now Open <ESB_HOME>repository/conf/tomcat/catalina-server.xml and add proxy ports

Under HTTP connection section add the following parameter.

proxyPort="80"

Full HTTP section will look like following,

<Connector  protocol="org.apache.coyote.http11.Http11NioProtocol"
                port="9763"
                proxyPort="80"
                redirectPort="9443" 
                bindOnInit="false"
                maxHttpHeaderSize="8192"
                acceptorThreadCount="2"
                maxThreads="250"
                minSpareThreads="50"
                disableUploadTimeout="false"
                connectionUploadTimeout="120000"
                maxKeepAliveRequests="200"
                acceptCount="200"
                server="WSO2 Carbon Server"
                compression="on"
                compressionMinSize="2048"
                noCompressionUserAgents="gozilla, traviata"
                compressableMimeType="text/html,text/javascript,application/x-javascript,application/javascript,application/xml,text/css,application/xslt+xml,text/xsl,image/gif,image/jpg,image/jpeg" 
                URIEncoding="UTF-8"/>

Now under HTTPs section add the following proxy port

proxyPort="443"

The Full config section will look like following

<Connector  protocol="org.apache.coyote.http11.Http11NioProtocol"
                port="9443"
                proxyPort="443"
                bindOnInit="false"
                sslProtocol="TLS"
                maxHttpHeaderSize="8192"
                acceptorThreadCount="2" 
                maxThreads="250"
                minSpareThreads="50"
                disableUploadTimeout="false"
                enableLookups="false"
                connectionUploadTimeout="120000"
                maxKeepAliveRequests="200"
                acceptCount="200"
                server="WSO2 Carbon Server"
                clientAuth="false"
                compression="on"
                scheme="https"
                secure="true"
                SSLEnabled="true"
                keystoreFile="${carbon.home}/repository/resources/security/wso2carbon.jks"
                keystorePass="wso2carbon" 
                compressionMinSize="2048"
                noCompressionUserAgents="gozilla, traviata"
                compressableMimeType="text/html,text/javascript,application/x-javascript,application/javascript,application/xml,text/css,application/xslt+xml,text/xsl,image/gif,image/jpg,image/jpeg"
                URIEncoding="UTF-8"/>

That's it, we have completed configuring the manager node. Now lets configure the worker node. Worker node is very similar to manager nodes configurations. But I will walk though worker 01's configurations so you won't miss anything :)

Configuring the Worker Node

1. Open <ESB_HOME>/repository/conf/datasources/master-datasources.xml and add following two datasources. Do not remove the existing carbon datasource.

<datasource>
  <name>WSO2_REG_DB</name>
  <description>The datasource used for shared config and governance registry</description>
  <jndiConfig>
  <name>jdbc/WSO2SharedDB</name>
  </jndiConfig>
  <definition type="RDBMS">
  <configuration>
   <url>jdbc:mysql://192.168.48.6:3306/ESB49_REGDB</url>
   <username>esbuser</username>
   <password>wso2root</password>
   <driverClassName>com.mysql.jdbc.Driver</driverClassName>
   <maxActive>50</maxActive>
   <maxWait>60000</maxWait>
   <testOnBorrow>true</testOnBorrow>
   <validationQuery>SELECT 1</validationQuery>
   <validationInterval>30000</validationInterval>
  </configuration>
  </definition>
 </datasource>

<datasource>
  <name>WSO2_UM_DB</name>
  <description>The datasource used for registry and user manager</description>
  <jndiConfig>
  <name>jdbc/WSO2UmDB</name>
  </jndiConfig>
  <definition type="RDBMS">
  <configuration>
   <url>jdbc:mysql://192.168.48.6:3306/ESB49_USERDB</url>
   <username>esbuser</username>
   <password>wso2root</password>
   <driverClassName>com.mysql.jdbc.Driver</driverClassName>
   <maxActive>50</maxActive>
   <maxWait>60000</maxWait>
   <testOnBorrow>true</testOnBorrow>
   <validationQuery>SELECT 1</validationQuery>
   <validationInterval>30000</validationInterval>
  </configuration>
  </definition>
</datasource>

Note : Make sure you don't remove WSO2_CARBON_DB datasource.

2. Now open <ESB_HOME>/repository/conf/carbon.xml

a. Change the following parameters to add the necessary host names

<HostName>esb.cl2.wso2.com</HostName>
<MgtHostName>mgt.esb.cl2.wso2.com</MgtHostName>

b. Inorder to set the Deploymeny syncronizer do the changes as shown below. Note that AutoCommit option is set to false in the worker node

    <DeploymentSynchronizer>
        <Enabled>true</Enabled>
        <AutoCommit>false</AutoCommit>
        <AutoCheckout>true</AutoCheckout>
        <RepositoryType>svn</RepositoryType>
        <SvnUrl>http://192.168.42.273/svn/wso2qarepo/esb49repo</SvnUrl>
        <SvnUser>wso2user1</SvnUser>
        <SvnPassword>wso2pass123</SvnPassword>
        <SvnUrlAppendTenantId>true</SvnUrlAppendTenantId>
    </DeploymentSynchronizer>

3. Now open <ESB_HOME>/repository/conf/user-mgt.xml and change the following property as shown below.

<Property name="dataSource">jdbc/WSO2UmDB</Property>

Note : jdbc/WSO2UmDB the JNDI Name of the datasource we declaired in step 01 for user management DB.

4. Open <ESB_HOME>/repository/conf/registry.xml and add the following extries. Do not delete existing entries.

<dbConfig name="sharedregistry">
<dataSource>jdbc/WSO2SharedDB</dataSource>
</dbConfig>
<remoteInstance url="https://localhost:9444/registry">
<id>instanceid</id>
<dbConfig>sharedregistry</dbConfig>
<readOnly>true</readOnly>
<enableCache>true</enableCache>
<registryRoot>/</registryRoot>
<cacheId>esbuser@jdbc:mysql://192.168.48.6:3306/ESB49_REGDB</cacheId>
</remoteInstance>
<mount path="/_system/config" overwrite="true">
<instanceId>instanceid</instanceId>
<targetPath>/_system/esbnodes</targetPath>
</mount>
<mount path="/_system/governance" overwrite="true">
<instanceId>instanceid</instanceId>
<targetPath>/_system/governance</targetPath>
</mount>

Note: Note that I have specified readOnly property as false. The worker should not add any artifacts to the registry. 

5. Now lets do cluster related configs. Open <ESB_HOME>/repository/conf/axis2/axis2.xml do the following changes.

a. Enable clustering as shown below. make enable true in the following property.

<clustering class="org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent" enable="true">

b.  Make sure that membershipScheme is set to wka (well known address based schema)

<parameter name="membershipScheme">wka</parameter>

c. Set the domain, all cluster nodes will join this domain. This is only used for the members to join the cluster.

<parameter name="domain">wso2.esb49.cl2.domain</parameter>

d. Set the local member host, add the IP of the worker 01 node to this.

<parameter name="localMemberHost">192.168.48.244</parameter>

e. Specify the localmember port to 4100.

<parameter name="localMemberPort">4100</parameter>

Note : if you are hosting all the nodes in the same machine each node should have a unique member port.

f. Now lets add  well known members.

<members>
     <member>
        <hostName>192.168.48.243</hostName>
        <port>4100</port>
     </member>
     <member>
        <hostName>192.168.48.249</hostName>
        <port>4100</port>
     </member>
</members>

Note : Here I'm specifying manager and worker 02 as the well-known member.

g. Uncomment and edit the WSDLEPRPrefix element under org.apache.synapse.transport.passthru.PassThroughHttpListener and
org.apache.synapse.transport.passthru.PassThroughHttpSSLListener in the transportReceiver. Each specifies HTTP and HTTP endpoints for the WSDL url.

<parameter name="WSDLEPRPrefix" locked="false">https://esb.cl2.wso2.com:443</parameter>
<parameter name="WSDLEPRPrefix" locked="false">http://esb.cl2.wso2.com:80</parameter>

6. Now Open <ESB_HOME>repository/conf/tomcat/catalina-server.xml and add proxy ports

Under HTTP connection section add the following parameter.

proxyPort="80"

Full HTTP section will look like following,

<Connector  protocol="org.apache.coyote.http11.Http11NioProtocol"
                port="9763"
                proxyPort="80"
                redirectPort="9443" 
                bindOnInit="false"
                maxHttpHeaderSize="8192"
                acceptorThreadCount="2"
                maxThreads="250"
                minSpareThreads="50"
                disableUploadTimeout="false"
                connectionUploadTimeout="120000"
                maxKeepAliveRequests="200"
                acceptCount="200"
                server="WSO2 Carbon Server"
                compression="on"
                compressionMinSize="2048"
                noCompressionUserAgents="gozilla, traviata"
                compressableMimeType="text/html,text/javascript,application/x-javascript,application/javascript,application/xml,text/css,application/xslt+xml,text/xsl,image/gif,image/jpg,image/jpeg" 
                URIEncoding="UTF-8"/>

Now under HTTPs section add the following proxy port


proxyPort="443"

The Full config section will look like following

<Connector  protocol="org.apache.coyote.http11.Http11NioProtocol"
                port="9443"
                proxyPort="443"
                bindOnInit="false"
                sslProtocol="TLS"
                maxHttpHeaderSize="8192"
                acceptorThreadCount="2" 
                maxThreads="250"
                minSpareThreads="50"
                disableUploadTimeout="false"
                enableLookups="false"
                connectionUploadTimeout="120000"
                maxKeepAliveRequests="200"
                acceptCount="200"
                server="WSO2 Carbon Server"
                clientAuth="false"
                compression="on"
                scheme="https"
                secure="true"
                SSLEnabled="true"
                keystoreFile="${carbon.home}/repository/resources/security/wso2carbon.jks"
                keystorePass="wso2carbon" 
                compressionMinSize="2048"
                noCompressionUserAgents="gozilla, traviata"
                compressableMimeType="text/html,text/javascript,application/x-javascript,application/javascript,application/xml,text/css,application/xslt+xml,text/xsl,image/gif,image/jpg,image/jpeg"
                URIEncoding="UTF-8"/>

7. Open <ESB_HOME>/bin/wso2server.sh and serch for DworkerNode parameter and change the value to true as shown below.

-DworkerNode=false \

So we have configured one of the worker noded, worker node 02s configurations are identical to the worker 01s change the nesassary ips in relevent places when configuring worker 02.


Copying necessary Drivers and libraries

You need to copy following drivers to both Worker and Manager nodes. 

JDBC Driver

Copy MYSQL JDBC driver to <ESB_HOME>/repository/components/lib directory. You can find the matching driver for your mysql version from here

SVN related JARs

1. Download and copy SVNKit from http://product-dist.wso2.com/tools/svnkit-all-1.8.7.wso2v1.jar into the <ESB_HOME>/repository/components/dropins folder.

2. Download http://maven.wso2.org/nexus/content/groups/wso2-public/com/trilead/trilead-ssh2/1.0.0-build215/trilead-ssh2-1.0.0-build215.jar and copy it to the <ESB_HOME>/repository/components/lib folder.

Configuring NginX


So ESB has two host names, management console host and worker host entries. And ESB also exposes 4 different ports (9443/9763, 8280/8243). NginX will be listening to system default ports 80/443, and it will route all the requests to the respective ESB node. The NginX configs will look like following.

create a new NginX conf file at /etc/nginx/conf.d and add the following content to it.

Note : I'm using my IPs and host names, you should change these according to your setup.


upstream esb49wkhttps {
       
        server 192.168.48.244:8243;
        server 192.168.48.249:8243;
}

upstream esb49wkhttp {
       
        server 192.168.48.244:8280;
        server 192.168.48.249:8280;
}

upstream esb49mgt {
        server 192.168.48.243:9443;        
}


map $http_upgrade $connection_upgrade {
   default upgrade;
    '' close;
}

server {
        listen 80;
        server_name esb.cl2.wso2.com;

        location / {
           proxy_set_header X-Forwarded-Host $host;
           proxy_set_header X-Forwarded-Server $host;
           proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

           proxy_set_header Host $http_host;
           proxy_read_timeout 5m;
           proxy_send_timeout 5m;
   
   
           proxy_pass http://esb49wkhttp/;
           proxy_redirect http://esb49wkhttp/ http://gw.am.wso2.com/;
 
    proxy_set_header Upgrade $http_upgrade;
           proxy_set_header Connection "upgrade";
           proxy_http_version 1.1;
      }
}

server {

       listen 443;
       server_name esb.cl2.wso2.com;

       ssl on;
       ssl_certificate /etc/nginx/ssl/am_wso2_com.crt;
       ssl_certificate_key /etc/nginx/ssl/am_wso2_com.key;

       location / {
           proxy_set_header X-Forwarded-Host $host;
           proxy_set_header X-Forwarded-Server $host;
           proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

           proxy_set_header Host $http_host;
           proxy_read_timeout 5m;
           proxy_send_timeout 5m;

           proxy_pass https://esb49wkhttps/;
           proxy_redirect https://esb49wkhttps/ https://esb.cl2.wso2.com/;

    #proxy_set_header Upgrade $http_upgrade;
           #proxy_set_header Connection "upgrade";
           #proxy_http_version 1.1;
       }
}

server {

       listen 443;
       server_name mgt.esb.cl2.wso2.com;

       ssl on;
       ssl_certificate /etc/nginx/ssl/am_wso2_com.crt;
       ssl_certificate_key /etc/nginx/ssl/am_wso2_com.key;

       location / {
           proxy_set_header X-Forwarded-Host $host;
           proxy_set_header X-Forwarded-Server $host;
           proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

           proxy_set_header Host $http_host;
           proxy_read_timeout 5m;
           proxy_send_timeout 5m;

           proxy_pass https://esb49mgt/;
           proxy_redirect https://esb49mgt/ https://mgt.esb.cl2.wso2.com/;

           #proxy_set_header Upgrade $http_upgrade;
           #proxy_set_header Connection "upgrade";
           #proxy_http_version 1.1;
       }
}


So that's it after adding the configs to NginX you need to restart or reload NginX.

Starting the cluster and accessing the cluster

If you haven't already created the necessary tables in the DBs you can start the servers in the following way to create necessary tables automatically.

./wso2server.sh -Dsetup

After starting the manager node you can start workers one by one, if the members successfully join the cluster a log will be printed like following.


TID: [-1] [] [2015-08-21 12:07:50,815]  INFO {org.wso2.carbon.core.clustering.hazelcast.wka.WKABasedMembershipScheme} -  Member joined [6974ca1c-8403-4711-9408-9de0cfaadda2]: /192.168.48.244:4100 {org.wso2.carbon.core.clustering.hazelcast.wka.WKABasedMembershipScheme}
TID: [-1] [] [2015-08-21 12:08:05,043]  INFO {org.wso2.carbon.core.clustering.hazelcast.wka.WKABasedMembershipScheme} -  Member joined [3ebb327b-91db-4d98-8c8a-95e2604e3a9c]: /192.168.48.249:4100 {org.wso2.carbon.core.clustering.hazelcast.wka.WKABasedMembershipScheme}
TID: [-1] [] [2015-08-21 12:08:07,159]  INFO {org.wso2.carbon.core.clustering.hazelcast.util.MemberUtils} -  Added member: Host:192.168.48.249, Remote Host:null, Port: 4100, HTTP:8280, HTTPS:8243, Domain: wso2.esb49.cl2.domain, Sub-domain:__$default, Active:true {org.wso2.carbon.core.clustering.hazelcast.util.MemberUtils}


After successfully starting the cluster you should be able to access the cluster in the following manner.
https://mgt.esb.cl2.wso2.com/carbon

 Note : Make sure you add necessary host entries to your /etc/hosts file. So you can access the URLs with the host names. Your host names should resolve to the NginX node.

So that's it we have sucessfully created a ESB cluster, Please drop a comment if you have any queries. :)


Tuesday, August 11, 2015



In this post I'll explain you how to take a DB dump in Oracle with deprecated imp/exp and using oracle data pump.

First lets create a user schema.

create user yasaas identified by yasas;
grant create session, grant any privilege to yasas;
grant all privileges to yasas;
Above commands will give most of the privileges to the user that are required for DB operations. But when exporting and importing the user needs database export and import privileges. So execute the following to give necessary privileges to the user.
grant exp_full_database to yasas;
grant imp_full_database to yasas;
Note: If you do not set the above previledges it will cause the following error
ORA-31631: privileges are required
ORA-39161: Full database jobs require privileges
Now add some tables and data to the schema. And we are ready to export the schema. You should note that there are two ways to do this
  • Using deprecated  imp/exp command which came with 10G
  • Using Oracle Datapump
If you use the data pump you will not be able to import the same dump to a diferent oracle server.

First lets see how to export and import the dump via exp command.

Exporting via exp command
Execute the following commands to export the dp to a dump file. Folllowing are OS commands.

exp yasas/yasas@192.168.23.112:1521/rac.localdomain owner=yasas file=mydbdump.dmp log=dump_log.log

If you are executing the command from the server it self you can do it like following

exp yasas/yasas owner=yasas file=mydbdump.dmp log=export_dump.log

Note : If you want to export the entire DB you can parse the full=y parameter instead of owner=yasas and talso not that the export log will be written to the specified file.

Importing the exported dump via imp command
You can import the dump file with following command

imp yasas/yasas file=mydbdump.dmp log=import_dump.log full=y
Note : if there are multiple sachems in the dump you can use 'fromuser' and 'touser' parameters to import this, also ignore=y is also use full to ignore any errors when importing. 

Exporting via expdp command

Before exporting the dump you should set the data pump directory, all the dumps and logs will be created here. First lets create a physicaldirectory.

mkdir /home/oracle/datapump

Now login to the DB via SQLPLUS or other client and execute the following.

create or replace directory sql_data_pump as '/home/oracle/datapump';
grant read,write on DIRECTORY sql_data_pump to yasas;
grant exp_full_database to yasas;

Now lets Export the dumps. These are OS commands.

expdp yasas/yasas schemas=yasas dumpfile=mydbdump.dmp logfile=explog.log;

Importing via impdp command

As shown above for the import you need to set data pump directory.

create or replace directory sql_data_pump as '/home/oracle/datapump';
grant read,write on DIRECTORY sql_data_pump to yasas;
grant imp_full_database to yasas;

Lets import the dump now,


impdp yasas/yasas schemas=yasas dumpfile=mydbdump.dmp logfile=implog.log;

Note : If you are executing this in a windows environment you might need to add directory parameter to the above command and give the data_pump directory name that was set.

e.g : impdp yasas/yasas schemas=yasas directory=sql_data_pump dumpfile=mydbdump.dmp logfile=implog.log;

So that's it, please drop a comment if you have any queries.



When importing a DB dump of a different schema you may come across the followiing two errors.

IMP-00003: ORACLE error 959 encountered
ORA-00959: tablespace 'DEV_GOV' does not exist

To get this issue fixed you can create the missing table space and point to the related table space when creating the schema.

Execute following to create new table space


CREATE TABLESPACE "DEV_GOV" DATAFILE '/home/oracle/dumps/file/DEV_GOV.dbf' SIZE 20M

The DEV_GOV.dbf file will be created automatically when you execute the above command.

Now create the user by setting the default table space of the user to above created table space.

create user DEV_AM identified by rootpass default tablespace DEV_GOV;
grant create session, grant any privilege to DEV_AM;
grant all privileges to DEV_AM;
grant imp_full_database to DEV_AM;


Now you should be able to import the dumps :)
Subscribe to RSS Feed Follow me on Twitter!