Saving Request Body to a file with WSO2 ESB - With File Connector and VFS

In this post I'll explain you how to save a request body to a file using WSO2 ESB. Here the approach I'm using is File connector. I'm using ESB 4.9 and File connector 2.0.

Note the File connector version I'm using because the syntax/key words differ from connector version to version.

The message flow is depicted as shown below.

To achieve the above use case we can use two approaces.

1. Using VFS transport.

a. Fist you need to enable VFS transport in ESB. To do this follow this.

b. Now create the following API.

<api xmlns="http://ws.apache.org/ns/synapse" name="Api01" context="/api01">
   <resource methods="POST" uri-template="/write-file-system">
      <inSequence>
         <log level="full">
            <property name="====== WRITING THE " value=" FILE ========"/>
         </log>
         <property name="FORCE_SC_ACCEPTED" value="true" scope="axis2" type="STRING"/>
         <property name="REST_URL_POSTFIX" value="" scope="axis2"/>
         <property name="OUT_ONLY" value="true" scope="default" type="STRING"/>
         <property name="transport.vfs.ReplyFileName" value="Hello55.txt" scope="transport" type="STRING"/>
         <log level="custom" separator=",">
            <property name="Output file" expression="$trp:transport.vfs.ReplyFileName"/>
         </log>
         <send>
            <endpoint>
               <address uri="vfs:ftp://ftpuser:ftpuser@localhost/files/in"/>
            </endpoint>
         </send>
      </inSequence>
      <outSequence/>
      <faultSequence/>
   </resource>
</api>

Note : In the above API I'm using a uri-template, If you use a uri-template the context will be appended to the end of the endpoint. So the ESB will try to access a location with the attached context which might cause issues. So to avoid this I'm using the following property to nullify the appending context.

<property name="REST_URL_POSTFIX" value="" scope="axis2"/>

2. You can achieve the same with file connector as well.  Inorder to use file connector you need to install the file connector first. You can refer [1] for more details.

a. After installing the connector and enabling it you can create the following API to achieve the requirement.

<api xmlns="http://ws.apache.org/ns/synapse" name="Api01" context="/api01">
   <resource methods="POST" uri-template="/write-file-system">
      <inSequence>
         <log level="full">
            <property name="In The API ==============" value="==================="/>
         </log>
         <property name="FORCE_SC_ACCEPTED" value="true" scope="axis2" type="STRING"/>
         <property name="OUT_ONLY" value="true" scope="default" type="STRING"/>
         <property name="transport.vfs.ReplyFileName" value="HelloConnector.txt" scope="transport" type="STRING"/>
         <property name="uri" value="ftp://ftpuser:ftpuser@192.168.48.119/files/in/" type="STRING"/>
         <log level="custom" separator=",">
            <property name="Output file" expression="$trp:transport.vfs.ReplyFileName"/>
         </log>
         <fileconnector.create>
            <source>{fn:concat($ctx:uri,$trp:transport.vfs.ReplyFileName)}</source>
            <inputContent>{$body}</inputContent>
         </fileconnector.create>
      </inSequence>
      <outSequence/>
      <faultSequence/>
   </resource>
</api>

Please drop a comment if you have any queries

[1] - https://store.wso2.com/store/assets/esbconnector/aec1554a-29ea-4dbb-b8c5-5d529a853aa2

Read More

How to switch between two Java version easily in Linux

In this post I will reveal how you can easily setup multiple java environments and switch between them easily.

This is a simple method I use, sharing in case anyone is interested. Please follow the steps below.

1. Download and extract the java version you require in your PC. I will be using Java 1.7 and Java 1.8 for the demonstration purpose. I have extracted java 1.7 and 1.8 to following locations.

e.g : /home/yasassri/soft/java/jdk1.8.0_05
        /home/yasassri/soft/java/jdk1.7.0_65

2. Now Open the bachrc file using your favorite text editor. You can use the following commands to do this.

vim ~/.bashrc

3. Now Add the following two alias at the end of the file.

# Setting Java 8
alias setjava18='export JAVA_HOME='/home/yasassri/soft/java/jdk1.8.0_05' && export PATH=$JAVA_HOME/bin:$PATH && source ~/.bashrc'

#Setting Java 7
alias setjava17='export JAVA_HOME='/home/yasassri/soft/java/jdk1.7.0_65' && export PATH=$JAVA_HOME/bin:$PATH && source ~/.bashrc'

Note : You need to replace '/home/yasassri/soft/java/jdk1.8.0_05' and '/home/yasassri/soft/java/jdk1.7.0_65' with the file location of extracted Java folders in your PC.

4. Save and exit the file.

5. Execute the following command.

source ~/.bashrc

6. Everything is set now. Now execute the following command in your terminal.

setjava18

Now check the Java version that is set with java -version, Java 8 will be set.

7. Now execute following 

setjava17

Now Java 1.7 will be set.

Thanks for reading and please drop a comment if you have any issues. 

Read More

Sending a custom response after storing a JMS message with WSO2 ESB.

In this Post I'll explain you how to Send a message to a ESB proxy service, Store the message in a messaging queue and respond the client with a custom respone.

The message flow is depicted as below.

As per the above diagram, A request is sent to the ESB and ESB will send out the message to ActiveMQ as a OUT_ONLY message, where ESB will not expect a response from ActiveMQ, at the same time the ESB will send a response to the client.

To achieve this first you need to enable JMS transport senders and listeners in the ESB in-order to receive and send JMS messages. You can do this by following this.

After configuring the ESB create the following Proxy service. Now when you send a request to the following proxy it will send store a message in the JMS queue and send a response to the client.

<?xml version="1.0" encoding="UTF-8"?>
<proxy xmlns="http://ws.apache.org/ns/synapse"
       name="TestJMS"
       transports="http,jms"
       statistics="disable"
       trace="disable"
       startOnLoad="true">
   <target>
      <inSequence>
         <property name="OUT_ONLY" value="true" scope="default" type="STRING"/>
         <log>
            <property name="==" value="============== In the In Sequence ================"/>
         </log>
         <call>
            <endpoint>
               <address uri="jms:/TESTQUEUE?transport.jms.ConnectionFactoryJNDIName=QueueConnectionFactory&amp;java.naming.factory.initial=org.apache.activemq.jndi.ActiveMQInitialContextFactory&amp;java.naming.provider.url=tcp://localhost:61616&amp;transport.jms.DestinationType=queue"/>
            </endpoint>
         </call>
         <payloadFactory media-type="xml">
            <format>
               <Response xmlns="">CUSTOM RESPONSE!!!</Response>
            </format>
            <args/>
         </payloadFactory>
         <respond/>
      </inSequence>
      <outSequence>
         <send/>
      </outSequence>
   </target>
   <description/>
</proxy>

In the above proxy please note the OUT_ONLY property which tells the ESB to send the message out and ignore any response.

Hope this post is useful to someone, Feel free to drop a comment if you have any queries. :)

Read More

Sending a GZIP compressed request using CURL

In this Post I'll explain you how to send a GZIPed post body using CURL commands. Its pretty straight forward, noting down if anyone come accross a similar requirement.

First you need to create the body you need to send, In my case I'm sending a XML content. So create a valid xml file, then zip the created file in gzip format, In ubuntu you can just right click on the file and compress it. After compressing the file will have a .gz extension.

e.g : test.xml.gz

Now navigate to the file location in terminal and execute the following command.

curl -v -s --trace-ascii http_trace.log --data-binary @test.xml.gz -H "Content-Type: text/xml" -H "Content-Encoding: gzip" -X POST http://10.100.5.112:8280/api01

After invoking everything will be recorded in the specified log file.

== Info:   Trying 10.100.5.112...
== Info: Connected to 10.100.5.112 (10.100.5.112) port 8280 (#0)
=> Send header, 155 bytes (0x9b)
0000: POST /api01 HTTP/1.1
0016: User-Agent: curl/7.35.0
002f: Host: 10.100.5.112:8280
0048: Accept: */*
0055: Content-Type: text/xml
006d: Content-Encoding: gzip
0085: Content-Length: 42
0099: 
=> Send data, 42 bytes (0x2a)
0000: .......U..test.xml..)I-.....6.`&..-..J....
== Info: upload completely sent off: 42 out of 42 bytes
<= Recv header, 17 bytes (0x11)
0000: HTTP/1.1 200 OK
<= Recv header, 24 bytes (0x18)
0000: Content-Type: text/xml
<= Recv header, 37 bytes (0x25)
0000: Date: Tue, 01 Sep 2015 06:40:03 GMT
<= Recv header, 28 bytes (0x1c)
0000: Transfer-Encoding: chunked
<= Recv header, 2 bytes (0x2)
0000: 
<= Recv data, 34 bytes (0x22)
0000: 17
0004: <abc>Created !!!!</abc>
001d: 0
0020: 
== Info: Connection #0 to host 10.100.5.112 left intact

Read More

How to Remotely monitor WSO2 Servers with JConsole

In this post I'll explain how to monitor WSO2 products remotely using Jconsole. Jconsoles comes by default with Java in linux you can type Jconsole to start Jconsole.

WSO2 Product Configurations

Main JMX ports related configs are located at <Product_Home>/repository/conf/carbon.xml Open <CARBON_HOME>/repository/conf/carbon.xml and specify the ports if you need to change the default ports. Change the following section.

        <!-- The JMX Ports -->
        <JMX>
            <!--The port RMI registry is exposed-->
            <RMIRegistryPort>9999</RMIRegistryPort>
            <!--The port RMI server should be exposed-->
            <RMIServerPort>11111</RMIServerPort>
        </JMX>

Now open <CARBON_HOME>/repository/conf/etc/jmx.xml and make sure the configs are changed as follows, You need to specify the machine IP as the HostName.

<JMX xmlns="http://wso2.org/projects/carbon/jmx.xml">
    <StartRMIServer>true</StartRMIServer>

    <!-- HostName, or Network interface to which this RMI server should be bound -->
    <HostName>192.168.48.49</HostName>

    <!--  ${Ports.JMX.RMIRegistryPort} is defined in the Ports section of the carbon.xml-->
    <RMIRegistryPort>${Ports.JMX.RMIRegistryPort}</RMIRegistryPort>

    <!--  ${Ports.JMX.RMIRegistryPort} is defined in the Ports section of the carbon.xml-->
    <RMIServerPort>${Ports.JMX.RMIServerPort}</RMIServerPort>
</JMX>

Now open <CARBON_HOME>/bin.wso2server.sh and add the following line as a JAVA parameter.

 -Djava.rmi.server.hostname="192.168.48.49"

You should add the above as shown below. Note: I'm using ESB 4.9 to demonstrate this, so your parameters may differ from what I have mentioned.

while [ "$status" = "$START_EXIT_STATUS" ]
do
    $JAVACMD \
    -Xbootclasspath/a:"$CARBON_XBOOTCLASSPATH" \
    -Xms256m -Xmx1024m -XX:MaxPermSize=256m \
    -XX:+HeapDumpOnOutOfMemoryError \
    -XX:HeapDumpPath="$CARBON_HOME/repository/logs/heap-dump.hprof" \
    $JAVA_OPTS \
    -Dcom.sun.management.jmxremote \
    -classpath "$CARBON_CLASSPATH" \
    -Djava.endorsed.dirs="$JAVA_ENDORSED_DIRS" \
    -Djava.io.tmpdir="$CARBON_HOME/tmp" \
    -Dcatalina.base="$CARBON_HOME/lib/tomcat" \
    -Dwso2.server.standalone=true \
    -Dcarbon.registry.root=/ \
    -Djava.command="$JAVACMD" \
    -Dcarbon.home="$CARBON_HOME" \
    -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager \
    -Dcarbon.config.dir.path="$CARBON_HOME/repository/conf" \
    -Djava.util.logging.config.file="$CARBON_HOME/repository/conf/etc/logging-bridge.properties" \
    -Dcomponents.repo="$CARBON_HOME/repository/components/plugins" \
    -Dconf.location="$CARBON_HOME/repository/conf"\
    -Dcom.atomikos.icatch.file="$CARBON_HOME/lib/transactions.properties" \
    -Dcom.atomikos.icatch.hide_init_file_path=true \
    -Dorg.apache.jasper.compiler.Parser.STRICT_QUOTE_ESCAPING=false \
    -Dorg.apache.jasper.runtime.BodyContentImpl.LIMIT_BUFFER=true \
    -Dcom.sun.jndi.ldap.connect.pool.authentication=simple  \
    -Dcom.sun.jndi.ldap.connect.pool.timeout=3000  \
    -Dorg.terracotta.quartz.skipUpdateCheck=true \
    -Djava.security.egd=file:/dev/./urandom \
    -Dfile.encoding=UTF8 \
    -Djava.net.preferIPv4Stack=true \
    -Dcom.ibm.cacheLocalHost=true \
    -DworkerNode=true \
    -Djava.rmi.server.hostname="192.168.48.49" \
    org.wso2.carbon.bootstrap.Bootstrap $*
    status=$?
done

Thats all from the WSO2 side, after the configurations are done you need to restart the server. If the RMI servers are started successfully you will see the following logs on server startup.

TID: [-1234] [] [2015-08-31 14:27:18,854]  INFO {org.wso2.carbon.core.init.JMXServerManager} -  JMX Service URL  : service:jmx:rmi://192.168.48.49:11111/jndi/rmi://192.168.48.49:9999/jmxrmi {org.wso2.carbon.core.init.JMXServerManager}

Connecting Via JConsole.

There are two ways to connect via JConsole remotely, using the IP and port [1] or using JMX service URL [2].

After starting Jconsole Select  Remote process. And add one of the following URLs

[1]    192.168.48.49:9999

[2]    service:jmx:rmi://192.168.48.49:11111/jndi/rmi://192.168.48.49:9999/jmxrmi

For the username and the password, enter the super user name and the password of the WSO2 server. The default username and password is admin/admin. The final configs will look like following.

Now click on connect, When clicked on connect it will give you and error saying secured connection failed, So you need to select insecure connection when this error pops up. If you need to securely connect to JConsole you need add necessary keystores etc.

If everything goes well you should connect to the RMI server and relevant graphs will be generated as shown below.

So that's it please drop a comment if you have any queries.

Read More

Clustering WSO2 ESB 4.9 with NginX

This post explain how to setup a WSO2 ESB cluster, here I will be using ESB 4.9, MySQL as the underline DB and NginX as the load balancer. Following diagram depicts the overall distribution of the cluster components.

Following are the host names IPs of each node.

No	Node Name	Host Name	IP	Local Member Port
1	Manager	mgt.esb.cl2.wso2.com	192.168.48.243	4001
2	Worker 01	esb.cl2.wso2.com	192.168.48.244	4001
3	Worker 02	esb.cl2.wso2.com	192.168.48.249	4001

Requisites.

So for the cluster I assume you have the following prerequisites fulfilled.

1. MySQL setup
2. NginX installed and configured
3. SVN server to use as the deployment synchronizer.

Note : I will be explaining how to setup the cluster in a distributed setup, meaning one instance will host one ESB instance. So we will not see any port conflicts. If you need setup ESB cluster in a single machine or want to host multiple instnces in a single machine  you need to offset each node to avoid port conflicts. To do that you can open <ESB_HOME>/repository/conf/carbon.xml  and change the following property.

 <Offset>0</Offset>

Also you need to specify unique local member ports for each node. This can be done in  <ESB_HOME>/repository/conf/axis2/axis2.xml under the following proerty

<parameter name="localMemberPort">4100</parameter>

Configuring the Manager Node

1. Open <ESB_HOME>/repository/conf/datasources/master-datasources.xml and add following two datasources. Do not remove the existing carbon datasource.

<datasource>

  <name>WSO2_REG_DB</name>
  <description>The datasource used for shared config and governance registry</description>
  <jndiConfig>
  <name>jdbc/WSO2SharedDB</name>
  </jndiConfig>
  <definition type="RDBMS">
  <configuration>
   <url>jdbc:mysql://192.168.48.6:3306/ESB49_REGDB</url>
   <username>esbuser</username>
   <password>wso2root</password>
   <driverClassName>com.mysql.jdbc.Driver</driverClassName>
   <maxActive>50</maxActive>
   <maxWait>60000</maxWait>
   <testOnBorrow>true</testOnBorrow>
   <validationQuery>SELECT 1</validationQuery>
   <validationInterval>30000</validationInterval>
  </configuration>
  </definition>
 </datasource>

<datasource>
  <name>WSO2_UM_DB</name>
  <description>The datasource used for registry and user manager</description>
  <jndiConfig>
  <name>jdbc/WSO2UmDB</name>
  </jndiConfig>
  <definition type="RDBMS">
  <configuration>
   <url>jdbc:mysql://192.168.48.6:3306/ESB49_USERDB</url>
   <username>esbuser</username>
   <password>wso2root</password>
   <driverClassName>com.mysql.jdbc.Driver</driverClassName>
   <maxActive>50</maxActive>
   <maxWait>60000</maxWait>
   <testOnBorrow>true</testOnBorrow>
   <validationQuery>SELECT 1</validationQuery>
   <validationInterval>30000</validationInterval>
  </configuration>
  </definition>
</datasource>

Note : Make sure you don't remove WSO2_CARBON_DB datasource. WSO2_CARBON_DB will be used as the local registry of each node which will point to the embedded H2 Database, WSO2_UM_DB will be used for user namagement and WSO2_REG_DB will be used as the shared registry among the cluster.

2. Now open <ESB_HOME>/repository/conf/carbon.xml

a. Change the following parameters to add the necessary host names

<HostName>esb.cl2.wso2.com</HostName>
<MgtHostName>mgt.esb.cl2.wso2.com</MgtHostName>

b. Inorder to set the Deploymeny syncronizer do the changes as shown below. Note that AutoCommit option is set to true in the manager node

<DeploymentSynchronizer>
        <Enabled>true</Enabled>
        <AutoCommit>true</AutoCommit>
        <AutoCheckout>true</AutoCheckout>
        <RepositoryType>svn</RepositoryType>
        <SvnUrl>http://192.168.42.273/svn/wso2qarepo/esb49repo</SvnUrl>
        <SvnUser>wso2user1</SvnUser>
        <SvnPassword>wso2pass123</SvnPassword>
        <SvnUrlAppendTenantId>true</SvnUrlAppendTenantId>
    </DeploymentSynchronizer>

3. Now open <ESB_HOME>/repository/conf/user-mgt.xml and change the following property as shown below.

<Property name="dataSource">jdbc/WSO2UmDB</Property>

Note : jdbc/WSO2UmDB is the JNDI Name of the datasource we declaired in step 01 for user management DB.

4. Open <ESB_HOME>/repository/conf/registry.xml and add the following entries. Do not delete existing entries.

<dbConfig name="sharedregistry">
<dataSource>jdbc/WSO2SharedDB</dataSource>
</dbConfig>
<remoteInstance url="https://localhost:9444/registry">
<id>instanceid</id>
<dbConfig>sharedregistry</dbConfig>
<readOnly>false</readOnly>
<enableCache>true</enableCache>
<registryRoot>/</registryRoot>
<cacheId>esbuser@jdbc:mysql://192.168.48.6:3306/ESB49_REGDB</cacheId>
</remoteInstance>
<mount path="/_system/config" overwrite="true">
<instanceId>instanceid</instanceId>
<targetPath>/_system/esbnodes</targetPath>
</mount>
<mount path="/_system/governance" overwrite="true">
<instanceId>instanceid</instanceId>
<targetPath>/_system/governance</targetPath>
</mount>

5. Now lets do cluster related configs. Open <ESB_HOME>/repository/conf/axis2/axis2.xml do the following changes.

a. Enable clustering as shown below. make enable true in the following property.

<clustering class="org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent" enable="true">

b.  Make sure that membershipScheme is set to wka (well known address based schema)

<parameter name="membershipScheme">wka</parameter>

c. Set the domain, all cluster nodes will join this domain. This is only used for the members to join the cluster.

<parameter name="domain">wso2.esb49.cl2.domain</parameter>

d. Set the local member host, add the IP of the manager node to this.

<parameter name="localMemberHost">192.168.48.243</parameter>

e. Specify the localmember port to 4100.

Note : if you are hosting all the nodes in the same machine each node should have a unique member port.

<parameter name="localMemberPort">4100</parameter>

f. Now lets add  well known members.

<members>
     <member>
        <hostName>192.168.48.244</hostName>
        <port>4100</port>
     </member>
</members>

Note : Here I'm specifying worker 01 as the well-known member, ideally you can specify all the nodes as well known members. But its optional. When a node goes down the well known address will be used to reconnect to the cluster by the disrupted node. 

g. Uncomment and edit the WSDLEPRPrefix element under org.apache.synapse.transport.passthru.PassThroughHttpListener and
org.apache.synapse.transport.passthru.PassThroughHttpSSLListener in the transportReceiver. Each specifies HTTP and HTTP endpoints for the WSDL url.

<parameter name="WSDLEPRPrefix" locked="false">https://esb.cl2.wso2.com:443</parameter>
<parameter name="WSDLEPRPrefix" locked="false">http://esb.cl2.wso2.com:80</parameter>

6. Now Open <ESB_HOME>repository/conf/tomcat/catalina-server.xml and add proxy ports

Under HTTP connection section add the following parameter.

proxyPort="80"

Full HTTP section will look like following,

<Connector  protocol="org.apache.coyote.http11.Http11NioProtocol"
                port="9763"
                proxyPort="80"
                redirectPort="9443" 
                bindOnInit="false"
                maxHttpHeaderSize="8192"
                acceptorThreadCount="2"
                maxThreads="250"
                minSpareThreads="50"
                disableUploadTimeout="false"
                connectionUploadTimeout="120000"
                maxKeepAliveRequests="200"
                acceptCount="200"
                server="WSO2 Carbon Server"
                compression="on"
                compressionMinSize="2048"
                noCompressionUserAgents="gozilla, traviata"
                compressableMimeType="text/html,text/javascript,application/x-javascript,application/javascript,application/xml,text/css,application/xslt+xml,text/xsl,image/gif,image/jpg,image/jpeg" 
                URIEncoding="UTF-8"/>

Now under HTTPs section add the following proxy port

proxyPort="443"

The Full config section will look like following

<Connector  protocol="org.apache.coyote.http11.Http11NioProtocol"
                port="9443"
                proxyPort="443"
                bindOnInit="false"
                sslProtocol="TLS"
                maxHttpHeaderSize="8192"
                acceptorThreadCount="2" 
                maxThreads="250"
                minSpareThreads="50"
                disableUploadTimeout="false"
                enableLookups="false"
                connectionUploadTimeout="120000"
                maxKeepAliveRequests="200"
                acceptCount="200"
                server="WSO2 Carbon Server"
                clientAuth="false"
                compression="on"
                scheme="https"
                secure="true"
                SSLEnabled="true"
                keystoreFile="${carbon.home}/repository/resources/security/wso2carbon.jks"
                keystorePass="wso2carbon" 
                compressionMinSize="2048"
                noCompressionUserAgents="gozilla, traviata"
                compressableMimeType="text/html,text/javascript,application/x-javascript,application/javascript,application/xml,text/css,application/xslt+xml,text/xsl,image/gif,image/jpg,image/jpeg"
                URIEncoding="UTF-8"/>

That's it, we have completed configuring the manager node. Now lets configure the worker node. Worker node is very similar to manager nodes configurations. But I will walk though worker 01's configurations so you won't miss anything :)

Configuring the Worker Node

1. Open <ESB_HOME>/repository/conf/datasources/master-datasources.xml and add following two datasources. Do not remove the existing carbon datasource.

<datasource>
  <name>WSO2_REG_DB</name>
  <description>The datasource used for shared config and governance registry</description>
  <jndiConfig>
  <name>jdbc/WSO2SharedDB</name>
  </jndiConfig>
  <definition type="RDBMS">
  <configuration>
   <url>jdbc:mysql://192.168.48.6:3306/ESB49_REGDB</url>
   <username>esbuser</username>
   <password>wso2root</password>
   <driverClassName>com.mysql.jdbc.Driver</driverClassName>
   <maxActive>50</maxActive>
   <maxWait>60000</maxWait>
   <testOnBorrow>true</testOnBorrow>
   <validationQuery>SELECT 1</validationQuery>
   <validationInterval>30000</validationInterval>
  </configuration>
  </definition>
 </datasource>

<datasource>
  <name>WSO2_UM_DB</name>
  <description>The datasource used for registry and user manager</description>
  <jndiConfig>
  <name>jdbc/WSO2UmDB</name>
  </jndiConfig>
  <definition type="RDBMS">
  <configuration>
   <url>jdbc:mysql://192.168.48.6:3306/ESB49_USERDB</url>
   <username>esbuser</username>
   <password>wso2root</password>
   <driverClassName>com.mysql.jdbc.Driver</driverClassName>
   <maxActive>50</maxActive>
   <maxWait>60000</maxWait>
   <testOnBorrow>true</testOnBorrow>
   <validationQuery>SELECT 1</validationQuery>
   <validationInterval>30000</validationInterval>
  </configuration>
  </definition>
</datasource>

Note : Make sure you don't remove WSO2_CARBON_DB datasource.

2. Now open <ESB_HOME>/repository/conf/carbon.xml

a. Change the following parameters to add the necessary host names

<HostName>esb.cl2.wso2.com</HostName>
<MgtHostName>mgt.esb.cl2.wso2.com</MgtHostName>

b. Inorder to set the Deploymeny syncronizer do the changes as shown below. Note that AutoCommit option is set to false in the worker node

    <DeploymentSynchronizer>
        <Enabled>true</Enabled>
        <AutoCommit>false</AutoCommit>
        <AutoCheckout>true</AutoCheckout>
        <RepositoryType>svn</RepositoryType>
        <SvnUrl>http://192.168.42.273/svn/wso2qarepo/esb49repo</SvnUrl>
        <SvnUser>wso2user1</SvnUser>
        <SvnPassword>wso2pass123</SvnPassword>
        <SvnUrlAppendTenantId>true</SvnUrlAppendTenantId>
    </DeploymentSynchronizer>

3. Now open <ESB_HOME>/repository/conf/user-mgt.xml and change the following property as shown below.

<Property name="dataSource">jdbc/WSO2UmDB</Property>

Note : jdbc/WSO2UmDB the JNDI Name of the datasource we declaired in step 01 for user management DB.

4. Open <ESB_HOME>/repository/conf/registry.xml and add the following extries. Do not delete existing entries.

<dbConfig name="sharedregistry">
<dataSource>jdbc/WSO2SharedDB</dataSource>
</dbConfig>
<remoteInstance url="https://localhost:9444/registry">
<id>instanceid</id>
<dbConfig>sharedregistry</dbConfig>
<readOnly>true</readOnly>
<enableCache>true</enableCache>
<registryRoot>/</registryRoot>
<cacheId>esbuser@jdbc:mysql://192.168.48.6:3306/ESB49_REGDB</cacheId>
</remoteInstance>
<mount path="/_system/config" overwrite="true">
<instanceId>instanceid</instanceId>
<targetPath>/_system/esbnodes</targetPath>
</mount>
<mount path="/_system/governance" overwrite="true">
<instanceId>instanceid</instanceId>
<targetPath>/_system/governance</targetPath>
</mount>

Note: Note that I have specified readOnly property as false. The worker should not add any artifacts to the registry. 

5. Now lets do cluster related configs. Open <ESB_HOME>/repository/conf/axis2/axis2.xml do the following changes.

a. Enable clustering as shown below. make enable true in the following property.

<clustering class="org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent" enable="true">

b.  Make sure that membershipScheme is set to wka (well known address based schema)

<parameter name="membershipScheme">wka</parameter>

c. Set the domain, all cluster nodes will join this domain. This is only used for the members to join the cluster.

<parameter name="domain">wso2.esb49.cl2.domain</parameter>

d. Set the local member host, add the IP of the worker 01 node to this.

<parameter name="localMemberHost">192.168.48.244</parameter>

e. Specify the localmember port to 4100.

<parameter name="localMemberPort">4100</parameter>

Note : if you are hosting all the nodes in the same machine each node should have a unique member port.

f. Now lets add  well known members.

<members>
     <member>
        <hostName>192.168.48.243</hostName>
        <port>4100</port>
     </member>

     <member>
        <hostName>192.168.48.249</hostName>
        <port>4100</port>
     </member>
</members>

Note : Here I'm specifying manager and worker 02 as the well-known member.

g. Uncomment and edit the WSDLEPRPrefix element under org.apache.synapse.transport.passthru.PassThroughHttpListener and

org.apache.synapse.transport.passthru.PassThroughHttpSSLListener in the transportReceiver. Each specifies HTTP and HTTP endpoints for the WSDL url.

<parameter name="WSDLEPRPrefix" locked="false">https://esb.cl2.wso2.com:443</parameter>
<parameter name="WSDLEPRPrefix" locked="false">http://esb.cl2.wso2.com:80</parameter>

6. Now Open <ESB_HOME>repository/conf/tomcat/catalina-server.xml and add proxy ports

Under HTTP connection section add the following parameter.

proxyPort="80"

Full HTTP section will look like following,

<Connector  protocol="org.apache.coyote.http11.Http11NioProtocol"
                port="9763"
                proxyPort="80"
                redirectPort="9443" 
                bindOnInit="false"
                maxHttpHeaderSize="8192"
                acceptorThreadCount="2"
                maxThreads="250"
                minSpareThreads="50"
                disableUploadTimeout="false"
                connectionUploadTimeout="120000"
                maxKeepAliveRequests="200"
                acceptCount="200"
                server="WSO2 Carbon Server"
                compression="on"
                compressionMinSize="2048"
                noCompressionUserAgents="gozilla, traviata"
                compressableMimeType="text/html,text/javascript,application/x-javascript,application/javascript,application/xml,text/css,application/xslt+xml,text/xsl,image/gif,image/jpg,image/jpeg" 
                URIEncoding="UTF-8"/>

Now under HTTPs section add the following proxy port

proxyPort="443"

The Full config section will look like following

<Connector  protocol="org.apache.coyote.http11.Http11NioProtocol"
                port="9443"
                proxyPort="443"
                bindOnInit="false"
                sslProtocol="TLS"
                maxHttpHeaderSize="8192"
                acceptorThreadCount="2" 
                maxThreads="250"
                minSpareThreads="50"
                disableUploadTimeout="false"
                enableLookups="false"
                connectionUploadTimeout="120000"
                maxKeepAliveRequests="200"
                acceptCount="200"
                server="WSO2 Carbon Server"
                clientAuth="false"
                compression="on"
                scheme="https"
                secure="true"
                SSLEnabled="true"
                keystoreFile="${carbon.home}/repository/resources/security/wso2carbon.jks"
                keystorePass="wso2carbon" 
                compressionMinSize="2048"
                noCompressionUserAgents="gozilla, traviata"
                compressableMimeType="text/html,text/javascript,application/x-javascript,application/javascript,application/xml,text/css,application/xslt+xml,text/xsl,image/gif,image/jpg,image/jpeg"
                URIEncoding="UTF-8"/>

7. Open <ESB_HOME>/bin/wso2server.sh and serch for DworkerNode parameter and change the value to true as shown below.

-DworkerNode=false \

So we have configured one of the worker noded, worker node 02s configurations are identical to the worker 01s change the nesassary ips in relevent places when configuring worker 02.

Copying necessary Drivers and libraries

You need to copy following drivers to both Worker and Manager nodes. 

JDBC Driver

Copy MYSQL JDBC driver to <ESB_HOME>/repository/components/lib directory. You can find the matching driver for your mysql version from here. 

SVN related JARs

1. Download and copy SVNKit from http://product-dist.wso2.com/tools/svnkit-all-1.8.7.wso2v1.jar into the <ESB_HOME>/repository/components/dropins folder.

2. Download http://maven.wso2.org/nexus/content/groups/wso2-public/com/trilead/trilead-ssh2/1.0.0-build215/trilead-ssh2-1.0.0-build215.jar and copy it to the <ESB_HOME>/repository/components/lib folder.

Configuring NginX

So ESB has two host names, management console host and worker host entries. And ESB also exposes 4 different ports (9443/9763, 8280/8243). NginX will be listening to system default ports 80/443, and it will route all the requests to the respective ESB node. The NginX configs will look like following.

create a new NginX conf file at /etc/nginx/conf.d and add the following content to it.

Note : I'm using my IPs and host names, you should change these according to your setup.

upstream esb49wkhttps {
       
        server 192.168.48.244:8243;
        server 192.168.48.249:8243;
}

upstream esb49wkhttp {
       
        server 192.168.48.244:8280;
        server 192.168.48.249:8280;
}

upstream esb49mgt {
        server 192.168.48.243:9443;        
}


map $http_upgrade $connection_upgrade {
   default upgrade;
    '' close;
}

server {
        listen 80;
        server_name esb.cl2.wso2.com;

        location / {
           proxy_set_header X-Forwarded-Host $host;
           proxy_set_header X-Forwarded-Server $host;
           proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

           proxy_set_header Host $http_host;
           proxy_read_timeout 5m;
           proxy_send_timeout 5m;
   
   
           proxy_pass http://esb49wkhttp/;
           proxy_redirect http://esb49wkhttp/ http://gw.am.wso2.com/;
 
    proxy_set_header Upgrade $http_upgrade;
           proxy_set_header Connection "upgrade";
           proxy_http_version 1.1;
      }
}

server {

       listen 443;
       server_name esb.cl2.wso2.com;

       ssl on;
       ssl_certificate /etc/nginx/ssl/am_wso2_com.crt;
       ssl_certificate_key /etc/nginx/ssl/am_wso2_com.key;

       location / {
           proxy_set_header X-Forwarded-Host $host;
           proxy_set_header X-Forwarded-Server $host;
           proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

           proxy_set_header Host $http_host;
           proxy_read_timeout 5m;
           proxy_send_timeout 5m;

           proxy_pass https://esb49wkhttps/;
           proxy_redirect https://esb49wkhttps/ https://esb.cl2.wso2.com/;

    #proxy_set_header Upgrade $http_upgrade;
           #proxy_set_header Connection "upgrade";
           #proxy_http_version 1.1;
       }
}

server {

       listen 443;
       server_name mgt.esb.cl2.wso2.com;

       ssl on;
       ssl_certificate /etc/nginx/ssl/am_wso2_com.crt;
       ssl_certificate_key /etc/nginx/ssl/am_wso2_com.key;

       location / {
           proxy_set_header X-Forwarded-Host $host;
           proxy_set_header X-Forwarded-Server $host;
           proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

           proxy_set_header Host $http_host;
           proxy_read_timeout 5m;
           proxy_send_timeout 5m;

           proxy_pass https://esb49mgt/;
           proxy_redirect https://esb49mgt/ https://mgt.esb.cl2.wso2.com/;

           #proxy_set_header Upgrade $http_upgrade;
           #proxy_set_header Connection "upgrade";
           #proxy_http_version 1.1;
       }
}

So that's it after adding the configs to NginX you need to restart or reload NginX.

Starting the cluster and accessing the cluster

If you haven't already created the necessary tables in the DBs you can start the servers in the following way to create necessary tables automatically.

./wso2server.sh -Dsetup

After starting the manager node you can start workers one by one, if the members successfully join the cluster a log will be printed like following.

TID: [-1] [] [2015-08-21 12:07:50,815]  INFO {org.wso2.carbon.core.clustering.hazelcast.wka.WKABasedMembershipScheme} -  Member joined [6974ca1c-8403-4711-9408-9de0cfaadda2]: /192.168.48.244:4100 {org.wso2.carbon.core.clustering.hazelcast.wka.WKABasedMembershipScheme}
TID: [-1] [] [2015-08-21 12:08:05,043]  INFO {org.wso2.carbon.core.clustering.hazelcast.wka.WKABasedMembershipScheme} -  Member joined [3ebb327b-91db-4d98-8c8a-95e2604e3a9c]: /192.168.48.249:4100 {org.wso2.carbon.core.clustering.hazelcast.wka.WKABasedMembershipScheme}
TID: [-1] [] [2015-08-21 12:08:07,159]  INFO {org.wso2.carbon.core.clustering.hazelcast.util.MemberUtils} -  Added member: Host:192.168.48.249, Remote Host:null, Port: 4100, HTTP:8280, HTTPS:8243, Domain: wso2.esb49.cl2.domain, Sub-domain:__$default, Active:true {org.wso2.carbon.core.clustering.hazelcast.util.MemberUtils}

After successfully starting the cluster you should be able to access the cluster in the following manner.
https://mgt.esb.cl2.wso2.com/carbon

 Note : Make sure you add necessary host entries to your /etc/hosts file. So you can access the URLs with the host names. Your host names should resolve to the NginX node.

So that's it we have sucessfully created a ESB cluster, Please drop a comment if you have any queries. :)

Read More