Customizing the service URI of a Proxy Service in Enterprise Integrator

In WSO2 Enterprise integrator we have proxy services. In a proxy service by default if you create a proxy service the context of the service URL will be auto generated with the name of the proxy service name, For example if you create a proxy service name "myproxy". The service URL will be like the below.

http://localhost:8290/services/myproxy

But what if you want to have a custom URI? For ecample.

http://localhost:8290/services/myproxy/idservice/1.0

Following is how you can do this.

Step 01 - Open your axis2.xml, this can be found <EI_HOME>/conf/axis2/axis2.xml

Step 02 : In the axis2xml find the phaseOrder type="InFlow" and add the following section.

Add following dispatcher to the Phaseorder,

<handler name="RequestURIBasedDispatcher"
class="org.apache.axis2.dispatchers.RequestURIBasedDispatcher">

The dispatcher should be inserted in to the In-Flow at the Dispatch phase. It should be the first handler in the Dispatch phase.

Note : In axis2.xml we have different message phases. Namely (Inflow, Outflow and FaultFlow) So make sure you have added the dispatcher to the correct phase. You need to add the dispatcher to InFlow.

<phaseOrder type="InFlow">
<!-- System pre defined phases -->
<!--
The MsgInObservation phase is used to observe messages as soon as they are
received. In this phase, we could do some things such as SOAP message tracing & keeping
track of the time at which a particular message was received


NOTE: This should be the very first phase in this flow
-->
<phase name="MsgInObservation">
<handler name="TraceMessageBuilderDispatchHandler"
class="org.apache.synapse.transport.passthru.util.TraceMessageBuilderDispatchHandler"/>
</phase>
<phase name="Validation"/>
<phase name="Transport">
<handler name="RequestURIBasedDispatcher"
class="org.apache.axis2.dispatchers.RequestURIBasedDispatcher">
<order phase="Transport"/>
</handler>
<handler name="CarbonContextConfigurator"
class="org.wso2.carbon.mediation.initializer.handler.CarbonContextConfigurator"/>
<handler name="RelaySecuirtyMessageBuilderDispatchandler"
class="org.apache.synapse.transport.passthru.util.RelaySecuirtyMessageBuilderDispatchandler"/>
<handler name="SOAPActionBasedDispatcher"
class="org.apache.axis2.dispatchers.SOAPActionBasedDispatcher">
<order phase="Transport"/>
</handler>
<!--handler name="SMTPFaultHandler"
class="org.wso2.carbon.core.transports.smtp.SMTPFaultHandler">
<order phase="Transport"/>
</handler-->
</phase>
<phase name="Addressing">
<handler name="AddressingBasedDispatcher"
class="org.apache.axis2.dispatchers.AddressingBasedDispatcher">
<order phase="Addressing"/>
</handler>
</phase>
<phase name="Security"/>
<phase name="PreDispatch">
<!--Uncomment following handler to enable logging in ESB log UI-->
<!--<handler name="TenantDomainSetter"-->
<!--class="org.wso2.carbon.utils.logging.handler.TenantDomainSetter"/>-->
</phase>
<phase name="Dispatch" class="org.apache.axis2.engine.DispatchPhase">
<handler name="CustomURIBasedDispatcher"
class="org.apache.synapse.core.axis2.CustomURIBasedDispatcher"/>
<handler name="RequestURIBasedDispatcher"
class="org.apache.axis2.dispatchers.RequestURIBasedDispatcher"/>
<handler name="SOAPActionBasedDispatcher"
class="org.apache.axis2.dispatchers.SOAPActionBasedDispatcher"/>
<handler name="RequestURIOperationDispatcher"
class="org.apache.axis2.dispatchers.RequestURIOperationDispatcher"/>
<handler name="SOAPMessageBodyBasedDispatcher"
class="org.apache.axis2.dispatchers.SOAPMessageBodyBasedDispatcher"/>


<handler name="HTTPLocationBasedDispatcher"
class="org.apache.axis2.dispatchers.HTTPLocationBasedDispatcher"/>
<handler name="MultitenantDispatcher"
class="org.wso2.carbon.tenant.dispatcher.MultitenantDispatcher"/>
<handler name="SynapseDispatcher"
class="org.apache.synapse.core.axis2.SynapseDispatcher"/>
<handler name="SynapseMustUnderstandHandler"
class="org.apache.synapse.core.axis2.SynapseMustUnderstandHandler"/>
</phase>
<!-- System pre defined phases -->
<phase name="RMPhase"/>
<phase name="OpPhase"/>
<phase name="AuthPhase"/>
<phase name="MUPhase"/>
<!-- After Postdispatch phase module author or or service author can add any phase he want -->
<phase name="OperationInPhase"/>
</phaseOrder>

Step 03 : Now restart your server and you can create the following proxy service. Note the ServiceURI parameter.

<?xml version="1.0" encoding="UTF-8"?>
<proxy xmlns="http://ws.apache.org/ns/synapse"
       name="SparkJobServerProxy"
       transports="https,http"
       statistics="disable"
       trace="disable"
       startOnLoad="true">
   <target>
      <inSequence>
         <log>
            <property name="=============TReached" value="=============="/>
         </log>
      </inSequence>
   </target>
   <parameter name="ServiceURI">/services/myproxy/idservice/1.0</parameter>
   <description/>
</proxy>

                                

So that is it, please drop a comment if you have any queries or need assistance.

Read More

Reading Query Parameters in Enterprise Integrator

Following is how you can read query params from your rest service when using Enterprise Integrator.

There are two ways to read your query params.

1. Assigning value to a property within a URI template. (get-property('uri.var.qtest')).

If you look at the following synapse config I'm assigning the required query param to

<api xmlns="http://ws.apache.org/ns/synapse" name="test" context="/qtest">
   <resource methods="GET" uri-template="/search?q={qtest}" outSequence="" faultSequence="">
      <inSequence>
         <log>
            <property name="QUERY PARAM IS =" expression="get-property('uri.var.qtest')"/>
         </log>
      </inSequence>
   </resource>
</api>

2. Reading as a query params. (get-property('query.param.q'))

By default query params are loaded by the EI it self, so you can directly access them without any extra configurations. Following is how you can do this.

<api xmlns="http://ws.apache.org/ns/synapse" name="test" context="/qtest">
   <resource methods="GET" uri-template="/search?q=*" outSequence="" faultSequence="">
      <inSequence>
         <log>
            <property name="QUERY PARAM IS =" expression="get-property('query.param.q')"/>
         </log>
      </inSequence>
   </resource>
</api>

Please drop a comment if you have any further queries.

Read More

Securing MySQL and Connecting WSO2 Servers

Settingup MYSQL

Generating the Keys and Signing them

Execute following commands to generate necessary keys and to sign them.

openssl genrsa 2048 > ca-key.pem
openssl req -new -x509 -nodes -days 3600 -key ca-key.pem -out ca.pem
openssl req -newkey rsa:2048 -days 3600 -nodes -keyout server-key.pem -out server-req.pem
openssl rsa -in server-key.pem -out server-key.pem
openssl x509 -req -in server-req.pem -days 3600 -CA ca.pem -CAkey ca-key.pem -set_serial 01 -out server-cert.pem
openssl req -newkey rsa:2048 -days 3600 -nodes -keyout client-key.pem -out client-req.pem
openssl rsa -in client-key.pem -out client-key.pem
openssl x509 -req -in client-req.pem -days 3600 -CA ca.pem -CAkey ca-key.pem -set_serial 01 -out client-cert.pem

Now open my.cnf and add the following configurations. Its located at /etc/mysql/my.cnf in Ubuntu.

[mysqld]
ssl-ca=/etc/mysql/ca.pem
ssl-cert=/etc/mysql/server-cert.pem
ssl-key=/etc/mysql/server-key.pem

An sample my.cnf would look like following.

Now restart mysql server.  You can use the following command to do this.

sudo service mysql restart

Now to check whether SSL certificates are properly set. Login to MySQL and execute the following query.

SHOW VARIABLES LIKE '%ssl%';

Above will give the below output.

+---------------+----------------------------+

| Variable_name | Value                      |

+---------------+----------------------------+

| have_openssl     | YES                                 |

| have_ssl             | YES                                  |

| ssl_ca                 | /etc/mysql/ca.pem         |

| ssl_capath         |                            |

| ssl_cert             | /etc/mysql/server-cert.pem |

| ssl_cipher         |                            |

| ssl_crl               |                                |

| ssl_crlpath        |                            |

| ssl_key              | /etc/mysql/server-key.pem  |

+---------------+----------------------------+

Now MYSQL configurations are done. Now lets configure WSO2 products to connect to MYSQL via SSL.

Connecting WSO2 Products to secured MySQL Server

1. First, we need to import client and server certificates to the client-truststore of WSO2 server. You can do this with following commands. (The certificates we created when configuring MySQL)

keytool -import -alias wso2qamysqlclient -file  /etc/mysql-ssl/server-cert.pem -keystore repository/resources/security/client-truststore.jks

keytool -import -alias wso2qamysqlserver -file  /etc/mysql-ssl/client-cert.pem -keystore repository/resources/security/client-truststore.jks

2. Now specify the SSL parameters in the connection URL. Make sure you specify both options useSSL and requireSSL.

jdbc:mysql://192.168.48.98:3306/ds21_carbon?autoReconnect=true&useSSL=true&requireSSL=true

The Full datasource will look like following.

<configuration>
    <url>jdbc:mysql://192.168.48.98:3306/ds21_carbon?autoReconnect=true&amp;useSSL=true&amp;requireSSL=true</url>
    <username>root</username>
    <defaultAutoCommit>false</defaultAutoCommit>
    <password>root</password>
    <driverClassName>com.mysql.jdbc.Driver</driverClassName>
    <maxActive>80</maxActive>
    <maxWait>60000</maxWait>
    <minIdle>5</minIdle>
    <testOnBorrow>true</testOnBorrow>
    <validationQuery>SELECT 1</validationQuery>
    <validationInterval>30000</validationInterval>
</configuration>

3. Now you can start the server. If everything is set properly, the server should start without errors.

Read More

Enable HTTP Access to WSO2 Management Console

By default the management console of WSO2 products are not accessible through HTTP, this is restricted due to security concerns. But if anyone desires to access the management console via the HTTP port Following is how you can do this.

1. Open <WSO2 HOME>/repository/conf/carbon.xml.
2. Search for the following configuration and uncommnet it if its commented or enable it.

    <EnableHTTPAdminConsole>true</EnableHTTPAdminConsole>

3. Restart the server.

And Now you should be able to access the management console with the HTTP URL as shown below.

http://localhost:9763/carbon/admin/login.jsp

Read More

Recieve GMail through Mailto transport -WSO2 ESB

In this Post I will explain How we can receive Emails through WSO2 ESB Mailto transport and perform some operations on it.

1. First lets Enable the transport. Open "repository/conf/axis2/axis2.xml" and uncomment the following section.

<transportReceiver name="mailto" class="org.apache.axis2.transport.mail.MailTransportListener">
        <!-- configure any optional POP3/IMAP properties
        check com.sun.mail.pop3 and com.sun.mail.imap package documentation for more details-->
    </transportReceiver>

2. Now lets create a proxy service to consumes the Emails. You can Copy and paste the following proxy service to source view to create the proxy. Make sure you change access parameters accordingly.

<?xml version="1.0" encoding="UTF-8"?>
<proxy xmlns="http://ws.apache.org/ns/synapse"
       name="JasperEmailPOP3"
       transports="mailto"
       statistics="disable"
       trace="disable"
       startOnLoad="true">
   <target>
      <inSequence>
         <property name="senderAddress"
                   expression="get-property('transport', 'From')"
                   scope="default"
                   type="STRING"/>
         <log level="custom">
            <property name="Date" expression="$trp:Date"/>
            <property name="Subject" expression="$trp:Subject"/>
            <property name="Content-Type" expression="$trp:Content-Type"/>
            <property name="From" expression="$trp:From"/>
         </log>
         <drop/>
      </inSequence>
      <outSequence>
         <send/>
      </outSequence>
   </target>
   <parameter name="mail.pop3.socketFactory.class">javax.net.ssl.SSLSocketFactory</parameter>
   <parameter name="transport.PollInterval">5</parameter>
   <parameter name="mail.pop3.host">pop.gmail.com</parameter>
   <parameter name="mail.pop3.user">ycwso2test</parameter>
   <parameter name="transport.mail.Protocol">pop3</parameter>
   <parameter name="mail.pop3.socketFactory.port">995</parameter>
   <parameter name="transport.mail.Address">ycwso2test@gmail.com</parameter>
   <parameter name="mail.pop3.password">wso2qa123</parameter>
   <parameter name="mail.pop3.port">995</parameter>
   <parameter name="mail.pop3.socketFactory.fallback">false</parameter>
   <description/>
</proxy>

3. Now Send a Email to your Email account and you will see the following log in backlogs.

[2016-08-03 12:10:05,232]  INFO - LogMediator Date = Wed, 3 Aug 2016 12:09:25 +0530, Subject = This is a Test, Content-Type = multipart/alternative; boundary=001a11402208fef0bb0539251978, From = Yasassri Ratnayake <ycrnet@gmail.com>

So thats it. If you cannot get this to work you can enable Debug logs and check whats wrong. To do this add the following line to repository/conf/log4j.properties and restart the ESB server.

log4j.logger.org.apache.axis2.transport.mail.MailTransportListener=DEBUG

Read More

Saving Request Body to a file with WSO2 ESB - With File Connector and VFS

In this post I'll explain you how to save a request body to a file using WSO2 ESB. Here the approach I'm using is File connector. I'm using ESB 4.9 and File connector 2.0.

Note the File connector version I'm using because the syntax/key words differ from connector version to version.

The message flow is depicted as shown below.

To achieve the above use case we can use two approaces.

1. Using VFS transport.

a. Fist you need to enable VFS transport in ESB. To do this follow this.

b. Now create the following API.

<api xmlns="http://ws.apache.org/ns/synapse" name="Api01" context="/api01">
   <resource methods="POST" uri-template="/write-file-system">
      <inSequence>
         <log level="full">
            <property name="====== WRITING THE " value=" FILE ========"/>
         </log>
         <property name="FORCE_SC_ACCEPTED" value="true" scope="axis2" type="STRING"/>
         <property name="REST_URL_POSTFIX" value="" scope="axis2"/>
         <property name="OUT_ONLY" value="true" scope="default" type="STRING"/>
         <property name="transport.vfs.ReplyFileName" value="Hello55.txt" scope="transport" type="STRING"/>
         <log level="custom" separator=",">
            <property name="Output file" expression="$trp:transport.vfs.ReplyFileName"/>
         </log>
         <send>
            <endpoint>
               <address uri="vfs:ftp://ftpuser:ftpuser@localhost/files/in"/>
            </endpoint>
         </send>
      </inSequence>
      <outSequence/>
      <faultSequence/>
   </resource>
</api>

Note : In the above API I'm using a uri-template, If you use a uri-template the context will be appended to the end of the endpoint. So the ESB will try to access a location with the attached context which might cause issues. So to avoid this I'm using the following property to nullify the appending context.

<property name="REST_URL_POSTFIX" value="" scope="axis2"/>

2. You can achieve the same with file connector as well.  Inorder to use file connector you need to install the file connector first. You can refer [1] for more details.

a. After installing the connector and enabling it you can create the following API to achieve the requirement.

<api xmlns="http://ws.apache.org/ns/synapse" name="Api01" context="/api01">
   <resource methods="POST" uri-template="/write-file-system">
      <inSequence>
         <log level="full">
            <property name="In The API ==============" value="==================="/>
         </log>
         <property name="FORCE_SC_ACCEPTED" value="true" scope="axis2" type="STRING"/>
         <property name="OUT_ONLY" value="true" scope="default" type="STRING"/>
         <property name="transport.vfs.ReplyFileName" value="HelloConnector.txt" scope="transport" type="STRING"/>
         <property name="uri" value="ftp://ftpuser:ftpuser@192.168.48.119/files/in/" type="STRING"/>
         <log level="custom" separator=",">
            <property name="Output file" expression="$trp:transport.vfs.ReplyFileName"/>
         </log>
         <fileconnector.create>
            <source>{fn:concat($ctx:uri,$trp:transport.vfs.ReplyFileName)}</source>
            <inputContent>{$body}</inputContent>
         </fileconnector.create>
      </inSequence>
      <outSequence/>
      <faultSequence/>
   </resource>
</api>

Please drop a comment if you have any queries

[1] - https://store.wso2.com/store/assets/esbconnector/aec1554a-29ea-4dbb-b8c5-5d529a853aa2

Read More

How to Remotely monitor WSO2 Servers with JConsole

In this post I'll explain how to monitor WSO2 products remotely using Jconsole. Jconsoles comes by default with Java in linux you can type Jconsole to start Jconsole.

WSO2 Product Configurations

Main JMX ports related configs are located at <Product_Home>/repository/conf/carbon.xml Open <CARBON_HOME>/repository/conf/carbon.xml and specify the ports if you need to change the default ports. Change the following section.

        <!-- The JMX Ports -->
        <JMX>
            <!--The port RMI registry is exposed-->
            <RMIRegistryPort>9999</RMIRegistryPort>
            <!--The port RMI server should be exposed-->
            <RMIServerPort>11111</RMIServerPort>
        </JMX>

Now open <CARBON_HOME>/repository/conf/etc/jmx.xml and make sure the configs are changed as follows, You need to specify the machine IP as the HostName.

<JMX xmlns="http://wso2.org/projects/carbon/jmx.xml">
    <StartRMIServer>true</StartRMIServer>

    <!-- HostName, or Network interface to which this RMI server should be bound -->
    <HostName>192.168.48.49</HostName>

    <!--  ${Ports.JMX.RMIRegistryPort} is defined in the Ports section of the carbon.xml-->
    <RMIRegistryPort>${Ports.JMX.RMIRegistryPort}</RMIRegistryPort>

    <!--  ${Ports.JMX.RMIRegistryPort} is defined in the Ports section of the carbon.xml-->
    <RMIServerPort>${Ports.JMX.RMIServerPort}</RMIServerPort>
</JMX>

Now open <CARBON_HOME>/bin.wso2server.sh and add the following line as a JAVA parameter.

 -Djava.rmi.server.hostname="192.168.48.49"

You should add the above as shown below. Note: I'm using ESB 4.9 to demonstrate this, so your parameters may differ from what I have mentioned.

while [ "$status" = "$START_EXIT_STATUS" ]
do
    $JAVACMD \
    -Xbootclasspath/a:"$CARBON_XBOOTCLASSPATH" \
    -Xms256m -Xmx1024m -XX:MaxPermSize=256m \
    -XX:+HeapDumpOnOutOfMemoryError \
    -XX:HeapDumpPath="$CARBON_HOME/repository/logs/heap-dump.hprof" \
    $JAVA_OPTS \
    -Dcom.sun.management.jmxremote \
    -classpath "$CARBON_CLASSPATH" \
    -Djava.endorsed.dirs="$JAVA_ENDORSED_DIRS" \
    -Djava.io.tmpdir="$CARBON_HOME/tmp" \
    -Dcatalina.base="$CARBON_HOME/lib/tomcat" \
    -Dwso2.server.standalone=true \
    -Dcarbon.registry.root=/ \
    -Djava.command="$JAVACMD" \
    -Dcarbon.home="$CARBON_HOME" \
    -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager \
    -Dcarbon.config.dir.path="$CARBON_HOME/repository/conf" \
    -Djava.util.logging.config.file="$CARBON_HOME/repository/conf/etc/logging-bridge.properties" \
    -Dcomponents.repo="$CARBON_HOME/repository/components/plugins" \
    -Dconf.location="$CARBON_HOME/repository/conf"\
    -Dcom.atomikos.icatch.file="$CARBON_HOME/lib/transactions.properties" \
    -Dcom.atomikos.icatch.hide_init_file_path=true \
    -Dorg.apache.jasper.compiler.Parser.STRICT_QUOTE_ESCAPING=false \
    -Dorg.apache.jasper.runtime.BodyContentImpl.LIMIT_BUFFER=true \
    -Dcom.sun.jndi.ldap.connect.pool.authentication=simple  \
    -Dcom.sun.jndi.ldap.connect.pool.timeout=3000  \
    -Dorg.terracotta.quartz.skipUpdateCheck=true \
    -Djava.security.egd=file:/dev/./urandom \
    -Dfile.encoding=UTF8 \
    -Djava.net.preferIPv4Stack=true \
    -Dcom.ibm.cacheLocalHost=true \
    -DworkerNode=true \
    -Djava.rmi.server.hostname="192.168.48.49" \
    org.wso2.carbon.bootstrap.Bootstrap $*
    status=$?
done

Thats all from the WSO2 side, after the configurations are done you need to restart the server. If the RMI servers are started successfully you will see the following logs on server startup.

TID: [-1234] [] [2015-08-31 14:27:18,854]  INFO {org.wso2.carbon.core.init.JMXServerManager} -  JMX Service URL  : service:jmx:rmi://192.168.48.49:11111/jndi/rmi://192.168.48.49:9999/jmxrmi {org.wso2.carbon.core.init.JMXServerManager}

Connecting Via JConsole.

There are two ways to connect via JConsole remotely, using the IP and port [1] or using JMX service URL [2].

After starting Jconsole Select  Remote process. And add one of the following URLs

[1]    192.168.48.49:9999

[2]    service:jmx:rmi://192.168.48.49:11111/jndi/rmi://192.168.48.49:9999/jmxrmi

For the username and the password, enter the super user name and the password of the WSO2 server. The default username and password is admin/admin. The final configs will look like following.

Now click on connect, When clicked on connect it will give you and error saying secured connection failed, So you need to select insecure connection when this error pops up. If you need to securely connect to JConsole you need add necessary keystores etc.

If everything goes well you should connect to the RMI server and relevant graphs will be generated as shown below.

So that's it please drop a comment if you have any queries.

Read More

Clustering WSO2 ESB 4.9 with NginX

This post explain how to setup a WSO2 ESB cluster, here I will be using ESB 4.9, MySQL as the underline DB and NginX as the load balancer. Following diagram depicts the overall distribution of the cluster components.

Following are the host names IPs of each node.

No	Node Name	Host Name	IP	Local Member Port
1	Manager	mgt.esb.cl2.wso2.com	192.168.48.243	4001
2	Worker 01	esb.cl2.wso2.com	192.168.48.244	4001
3	Worker 02	esb.cl2.wso2.com	192.168.48.249	4001

Requisites.

So for the cluster I assume you have the following prerequisites fulfilled.

1. MySQL setup
2. NginX installed and configured
3. SVN server to use as the deployment synchronizer.

Note : I will be explaining how to setup the cluster in a distributed setup, meaning one instance will host one ESB instance. So we will not see any port conflicts. If you need setup ESB cluster in a single machine or want to host multiple instnces in a single machine  you need to offset each node to avoid port conflicts. To do that you can open <ESB_HOME>/repository/conf/carbon.xml  and change the following property.

 <Offset>0</Offset>

Also you need to specify unique local member ports for each node. This can be done in  <ESB_HOME>/repository/conf/axis2/axis2.xml under the following proerty

<parameter name="localMemberPort">4100</parameter>

Configuring the Manager Node

1. Open <ESB_HOME>/repository/conf/datasources/master-datasources.xml and add following two datasources. Do not remove the existing carbon datasource.

<datasource>

  <name>WSO2_REG_DB</name>
  <description>The datasource used for shared config and governance registry</description>
  <jndiConfig>
  <name>jdbc/WSO2SharedDB</name>
  </jndiConfig>
  <definition type="RDBMS">
  <configuration>
   <url>jdbc:mysql://192.168.48.6:3306/ESB49_REGDB</url>
   <username>esbuser</username>
   <password>wso2root</password>
   <driverClassName>com.mysql.jdbc.Driver</driverClassName>
   <maxActive>50</maxActive>
   <maxWait>60000</maxWait>
   <testOnBorrow>true</testOnBorrow>
   <validationQuery>SELECT 1</validationQuery>
   <validationInterval>30000</validationInterval>
  </configuration>
  </definition>
 </datasource>

<datasource>
  <name>WSO2_UM_DB</name>
  <description>The datasource used for registry and user manager</description>
  <jndiConfig>
  <name>jdbc/WSO2UmDB</name>
  </jndiConfig>
  <definition type="RDBMS">
  <configuration>
   <url>jdbc:mysql://192.168.48.6:3306/ESB49_USERDB</url>
   <username>esbuser</username>
   <password>wso2root</password>
   <driverClassName>com.mysql.jdbc.Driver</driverClassName>
   <maxActive>50</maxActive>
   <maxWait>60000</maxWait>
   <testOnBorrow>true</testOnBorrow>
   <validationQuery>SELECT 1</validationQuery>
   <validationInterval>30000</validationInterval>
  </configuration>
  </definition>
</datasource>

Note : Make sure you don't remove WSO2_CARBON_DB datasource. WSO2_CARBON_DB will be used as the local registry of each node which will point to the embedded H2 Database, WSO2_UM_DB will be used for user namagement and WSO2_REG_DB will be used as the shared registry among the cluster.

2. Now open <ESB_HOME>/repository/conf/carbon.xml

a. Change the following parameters to add the necessary host names

<HostName>esb.cl2.wso2.com</HostName>
<MgtHostName>mgt.esb.cl2.wso2.com</MgtHostName>

b. Inorder to set the Deploymeny syncronizer do the changes as shown below. Note that AutoCommit option is set to true in the manager node

<DeploymentSynchronizer>
        <Enabled>true</Enabled>
        <AutoCommit>true</AutoCommit>
        <AutoCheckout>true</AutoCheckout>
        <RepositoryType>svn</RepositoryType>
        <SvnUrl>http://192.168.42.273/svn/wso2qarepo/esb49repo</SvnUrl>
        <SvnUser>wso2user1</SvnUser>
        <SvnPassword>wso2pass123</SvnPassword>
        <SvnUrlAppendTenantId>true</SvnUrlAppendTenantId>
    </DeploymentSynchronizer>

3. Now open <ESB_HOME>/repository/conf/user-mgt.xml and change the following property as shown below.

<Property name="dataSource">jdbc/WSO2UmDB</Property>

Note : jdbc/WSO2UmDB is the JNDI Name of the datasource we declaired in step 01 for user management DB.

4. Open <ESB_HOME>/repository/conf/registry.xml and add the following entries. Do not delete existing entries.

<dbConfig name="sharedregistry">
<dataSource>jdbc/WSO2SharedDB</dataSource>
</dbConfig>
<remoteInstance url="https://localhost:9444/registry">
<id>instanceid</id>
<dbConfig>sharedregistry</dbConfig>
<readOnly>false</readOnly>
<enableCache>true</enableCache>
<registryRoot>/</registryRoot>
<cacheId>esbuser@jdbc:mysql://192.168.48.6:3306/ESB49_REGDB</cacheId>
</remoteInstance>
<mount path="/_system/config" overwrite="true">
<instanceId>instanceid</instanceId>
<targetPath>/_system/esbnodes</targetPath>
</mount>
<mount path="/_system/governance" overwrite="true">
<instanceId>instanceid</instanceId>
<targetPath>/_system/governance</targetPath>
</mount>

5. Now lets do cluster related configs. Open <ESB_HOME>/repository/conf/axis2/axis2.xml do the following changes.

a. Enable clustering as shown below. make enable true in the following property.

<clustering class="org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent" enable="true">

b.  Make sure that membershipScheme is set to wka (well known address based schema)

<parameter name="membershipScheme">wka</parameter>

c. Set the domain, all cluster nodes will join this domain. This is only used for the members to join the cluster.

<parameter name="domain">wso2.esb49.cl2.domain</parameter>

d. Set the local member host, add the IP of the manager node to this.

<parameter name="localMemberHost">192.168.48.243</parameter>

e. Specify the localmember port to 4100.

Note : if you are hosting all the nodes in the same machine each node should have a unique member port.

<parameter name="localMemberPort">4100</parameter>

f. Now lets add  well known members.

<members>
     <member>
        <hostName>192.168.48.244</hostName>
        <port>4100</port>
     </member>
</members>

Note : Here I'm specifying worker 01 as the well-known member, ideally you can specify all the nodes as well known members. But its optional. When a node goes down the well known address will be used to reconnect to the cluster by the disrupted node. 

g. Uncomment and edit the WSDLEPRPrefix element under org.apache.synapse.transport.passthru.PassThroughHttpListener and
org.apache.synapse.transport.passthru.PassThroughHttpSSLListener in the transportReceiver. Each specifies HTTP and HTTP endpoints for the WSDL url.

<parameter name="WSDLEPRPrefix" locked="false">https://esb.cl2.wso2.com:443</parameter>
<parameter name="WSDLEPRPrefix" locked="false">http://esb.cl2.wso2.com:80</parameter>

6. Now Open <ESB_HOME>repository/conf/tomcat/catalina-server.xml and add proxy ports

Under HTTP connection section add the following parameter.

proxyPort="80"

Full HTTP section will look like following,

<Connector  protocol="org.apache.coyote.http11.Http11NioProtocol"
                port="9763"
                proxyPort="80"
                redirectPort="9443" 
                bindOnInit="false"
                maxHttpHeaderSize="8192"
                acceptorThreadCount="2"
                maxThreads="250"
                minSpareThreads="50"
                disableUploadTimeout="false"
                connectionUploadTimeout="120000"
                maxKeepAliveRequests="200"
                acceptCount="200"
                server="WSO2 Carbon Server"
                compression="on"
                compressionMinSize="2048"
                noCompressionUserAgents="gozilla, traviata"
                compressableMimeType="text/html,text/javascript,application/x-javascript,application/javascript,application/xml,text/css,application/xslt+xml,text/xsl,image/gif,image/jpg,image/jpeg" 
                URIEncoding="UTF-8"/>

Now under HTTPs section add the following proxy port

proxyPort="443"

The Full config section will look like following

<Connector  protocol="org.apache.coyote.http11.Http11NioProtocol"
                port="9443"
                proxyPort="443"
                bindOnInit="false"
                sslProtocol="TLS"
                maxHttpHeaderSize="8192"
                acceptorThreadCount="2" 
                maxThreads="250"
                minSpareThreads="50"
                disableUploadTimeout="false"
                enableLookups="false"
                connectionUploadTimeout="120000"
                maxKeepAliveRequests="200"
                acceptCount="200"
                server="WSO2 Carbon Server"
                clientAuth="false"
                compression="on"
                scheme="https"
                secure="true"
                SSLEnabled="true"
                keystoreFile="${carbon.home}/repository/resources/security/wso2carbon.jks"
                keystorePass="wso2carbon" 
                compressionMinSize="2048"
                noCompressionUserAgents="gozilla, traviata"
                compressableMimeType="text/html,text/javascript,application/x-javascript,application/javascript,application/xml,text/css,application/xslt+xml,text/xsl,image/gif,image/jpg,image/jpeg"
                URIEncoding="UTF-8"/>

That's it, we have completed configuring the manager node. Now lets configure the worker node. Worker node is very similar to manager nodes configurations. But I will walk though worker 01's configurations so you won't miss anything :)

Configuring the Worker Node

1. Open <ESB_HOME>/repository/conf/datasources/master-datasources.xml and add following two datasources. Do not remove the existing carbon datasource.

<datasource>
  <name>WSO2_REG_DB</name>
  <description>The datasource used for shared config and governance registry</description>
  <jndiConfig>
  <name>jdbc/WSO2SharedDB</name>
  </jndiConfig>
  <definition type="RDBMS">
  <configuration>
   <url>jdbc:mysql://192.168.48.6:3306/ESB49_REGDB</url>
   <username>esbuser</username>
   <password>wso2root</password>
   <driverClassName>com.mysql.jdbc.Driver</driverClassName>
   <maxActive>50</maxActive>
   <maxWait>60000</maxWait>
   <testOnBorrow>true</testOnBorrow>
   <validationQuery>SELECT 1</validationQuery>
   <validationInterval>30000</validationInterval>
  </configuration>
  </definition>
 </datasource>

<datasource>
  <name>WSO2_UM_DB</name>
  <description>The datasource used for registry and user manager</description>
  <jndiConfig>
  <name>jdbc/WSO2UmDB</name>
  </jndiConfig>
  <definition type="RDBMS">
  <configuration>
   <url>jdbc:mysql://192.168.48.6:3306/ESB49_USERDB</url>
   <username>esbuser</username>
   <password>wso2root</password>
   <driverClassName>com.mysql.jdbc.Driver</driverClassName>
   <maxActive>50</maxActive>
   <maxWait>60000</maxWait>
   <testOnBorrow>true</testOnBorrow>
   <validationQuery>SELECT 1</validationQuery>
   <validationInterval>30000</validationInterval>
  </configuration>
  </definition>
</datasource>

Note : Make sure you don't remove WSO2_CARBON_DB datasource.

2. Now open <ESB_HOME>/repository/conf/carbon.xml

a. Change the following parameters to add the necessary host names

<HostName>esb.cl2.wso2.com</HostName>
<MgtHostName>mgt.esb.cl2.wso2.com</MgtHostName>

b. Inorder to set the Deploymeny syncronizer do the changes as shown below. Note that AutoCommit option is set to false in the worker node

    <DeploymentSynchronizer>
        <Enabled>true</Enabled>
        <AutoCommit>false</AutoCommit>
        <AutoCheckout>true</AutoCheckout>
        <RepositoryType>svn</RepositoryType>
        <SvnUrl>http://192.168.42.273/svn/wso2qarepo/esb49repo</SvnUrl>
        <SvnUser>wso2user1</SvnUser>
        <SvnPassword>wso2pass123</SvnPassword>
        <SvnUrlAppendTenantId>true</SvnUrlAppendTenantId>
    </DeploymentSynchronizer>

3. Now open <ESB_HOME>/repository/conf/user-mgt.xml and change the following property as shown below.

<Property name="dataSource">jdbc/WSO2UmDB</Property>

Note : jdbc/WSO2UmDB the JNDI Name of the datasource we declaired in step 01 for user management DB.

4. Open <ESB_HOME>/repository/conf/registry.xml and add the following extries. Do not delete existing entries.

<dbConfig name="sharedregistry">
<dataSource>jdbc/WSO2SharedDB</dataSource>
</dbConfig>
<remoteInstance url="https://localhost:9444/registry">
<id>instanceid</id>
<dbConfig>sharedregistry</dbConfig>
<readOnly>true</readOnly>
<enableCache>true</enableCache>
<registryRoot>/</registryRoot>
<cacheId>esbuser@jdbc:mysql://192.168.48.6:3306/ESB49_REGDB</cacheId>
</remoteInstance>
<mount path="/_system/config" overwrite="true">
<instanceId>instanceid</instanceId>
<targetPath>/_system/esbnodes</targetPath>
</mount>
<mount path="/_system/governance" overwrite="true">
<instanceId>instanceid</instanceId>
<targetPath>/_system/governance</targetPath>
</mount>

Note: Note that I have specified readOnly property as false. The worker should not add any artifacts to the registry. 

5. Now lets do cluster related configs. Open <ESB_HOME>/repository/conf/axis2/axis2.xml do the following changes.

a. Enable clustering as shown below. make enable true in the following property.

<clustering class="org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent" enable="true">

b.  Make sure that membershipScheme is set to wka (well known address based schema)

<parameter name="membershipScheme">wka</parameter>

c. Set the domain, all cluster nodes will join this domain. This is only used for the members to join the cluster.

<parameter name="domain">wso2.esb49.cl2.domain</parameter>

d. Set the local member host, add the IP of the worker 01 node to this.

<parameter name="localMemberHost">192.168.48.244</parameter>

e. Specify the localmember port to 4100.

<parameter name="localMemberPort">4100</parameter>

Note : if you are hosting all the nodes in the same machine each node should have a unique member port.

f. Now lets add  well known members.

<members>
     <member>
        <hostName>192.168.48.243</hostName>
        <port>4100</port>
     </member>

     <member>
        <hostName>192.168.48.249</hostName>
        <port>4100</port>
     </member>
</members>

Note : Here I'm specifying manager and worker 02 as the well-known member.

g. Uncomment and edit the WSDLEPRPrefix element under org.apache.synapse.transport.passthru.PassThroughHttpListener and

org.apache.synapse.transport.passthru.PassThroughHttpSSLListener in the transportReceiver. Each specifies HTTP and HTTP endpoints for the WSDL url.

<parameter name="WSDLEPRPrefix" locked="false">https://esb.cl2.wso2.com:443</parameter>
<parameter name="WSDLEPRPrefix" locked="false">http://esb.cl2.wso2.com:80</parameter>

6. Now Open <ESB_HOME>repository/conf/tomcat/catalina-server.xml and add proxy ports

Under HTTP connection section add the following parameter.

proxyPort="80"

Full HTTP section will look like following,

<Connector  protocol="org.apache.coyote.http11.Http11NioProtocol"
                port="9763"
                proxyPort="80"
                redirectPort="9443" 
                bindOnInit="false"
                maxHttpHeaderSize="8192"
                acceptorThreadCount="2"
                maxThreads="250"
                minSpareThreads="50"
                disableUploadTimeout="false"
                connectionUploadTimeout="120000"
                maxKeepAliveRequests="200"
                acceptCount="200"
                server="WSO2 Carbon Server"
                compression="on"
                compressionMinSize="2048"
                noCompressionUserAgents="gozilla, traviata"
                compressableMimeType="text/html,text/javascript,application/x-javascript,application/javascript,application/xml,text/css,application/xslt+xml,text/xsl,image/gif,image/jpg,image/jpeg" 
                URIEncoding="UTF-8"/>

Now under HTTPs section add the following proxy port

proxyPort="443"

The Full config section will look like following

<Connector  protocol="org.apache.coyote.http11.Http11NioProtocol"
                port="9443"
                proxyPort="443"
                bindOnInit="false"
                sslProtocol="TLS"
                maxHttpHeaderSize="8192"
                acceptorThreadCount="2" 
                maxThreads="250"
                minSpareThreads="50"
                disableUploadTimeout="false"
                enableLookups="false"
                connectionUploadTimeout="120000"
                maxKeepAliveRequests="200"
                acceptCount="200"
                server="WSO2 Carbon Server"
                clientAuth="false"
                compression="on"
                scheme="https"
                secure="true"
                SSLEnabled="true"
                keystoreFile="${carbon.home}/repository/resources/security/wso2carbon.jks"
                keystorePass="wso2carbon" 
                compressionMinSize="2048"
                noCompressionUserAgents="gozilla, traviata"
                compressableMimeType="text/html,text/javascript,application/x-javascript,application/javascript,application/xml,text/css,application/xslt+xml,text/xsl,image/gif,image/jpg,image/jpeg"
                URIEncoding="UTF-8"/>

7. Open <ESB_HOME>/bin/wso2server.sh and serch for DworkerNode parameter and change the value to true as shown below.

-DworkerNode=false \

So we have configured one of the worker noded, worker node 02s configurations are identical to the worker 01s change the nesassary ips in relevent places when configuring worker 02.

Copying necessary Drivers and libraries

You need to copy following drivers to both Worker and Manager nodes. 

JDBC Driver

Copy MYSQL JDBC driver to <ESB_HOME>/repository/components/lib directory. You can find the matching driver for your mysql version from here. 

SVN related JARs

1. Download and copy SVNKit from http://product-dist.wso2.com/tools/svnkit-all-1.8.7.wso2v1.jar into the <ESB_HOME>/repository/components/dropins folder.

2. Download http://maven.wso2.org/nexus/content/groups/wso2-public/com/trilead/trilead-ssh2/1.0.0-build215/trilead-ssh2-1.0.0-build215.jar and copy it to the <ESB_HOME>/repository/components/lib folder.

Configuring NginX

So ESB has two host names, management console host and worker host entries. And ESB also exposes 4 different ports (9443/9763, 8280/8243). NginX will be listening to system default ports 80/443, and it will route all the requests to the respective ESB node. The NginX configs will look like following.

create a new NginX conf file at /etc/nginx/conf.d and add the following content to it.

Note : I'm using my IPs and host names, you should change these according to your setup.

upstream esb49wkhttps {
       
        server 192.168.48.244:8243;
        server 192.168.48.249:8243;
}

upstream esb49wkhttp {
       
        server 192.168.48.244:8280;
        server 192.168.48.249:8280;
}

upstream esb49mgt {
        server 192.168.48.243:9443;        
}


map $http_upgrade $connection_upgrade {
   default upgrade;
    '' close;
}

server {
        listen 80;
        server_name esb.cl2.wso2.com;

        location / {
           proxy_set_header X-Forwarded-Host $host;
           proxy_set_header X-Forwarded-Server $host;
           proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

           proxy_set_header Host $http_host;
           proxy_read_timeout 5m;
           proxy_send_timeout 5m;
   
   
           proxy_pass http://esb49wkhttp/;
           proxy_redirect http://esb49wkhttp/ http://gw.am.wso2.com/;
 
    proxy_set_header Upgrade $http_upgrade;
           proxy_set_header Connection "upgrade";
           proxy_http_version 1.1;
      }
}

server {

       listen 443;
       server_name esb.cl2.wso2.com;

       ssl on;
       ssl_certificate /etc/nginx/ssl/am_wso2_com.crt;
       ssl_certificate_key /etc/nginx/ssl/am_wso2_com.key;

       location / {
           proxy_set_header X-Forwarded-Host $host;
           proxy_set_header X-Forwarded-Server $host;
           proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

           proxy_set_header Host $http_host;
           proxy_read_timeout 5m;
           proxy_send_timeout 5m;

           proxy_pass https://esb49wkhttps/;
           proxy_redirect https://esb49wkhttps/ https://esb.cl2.wso2.com/;

    #proxy_set_header Upgrade $http_upgrade;
           #proxy_set_header Connection "upgrade";
           #proxy_http_version 1.1;
       }
}

server {

       listen 443;
       server_name mgt.esb.cl2.wso2.com;

       ssl on;
       ssl_certificate /etc/nginx/ssl/am_wso2_com.crt;
       ssl_certificate_key /etc/nginx/ssl/am_wso2_com.key;

       location / {
           proxy_set_header X-Forwarded-Host $host;
           proxy_set_header X-Forwarded-Server $host;
           proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

           proxy_set_header Host $http_host;
           proxy_read_timeout 5m;
           proxy_send_timeout 5m;

           proxy_pass https://esb49mgt/;
           proxy_redirect https://esb49mgt/ https://mgt.esb.cl2.wso2.com/;

           #proxy_set_header Upgrade $http_upgrade;
           #proxy_set_header Connection "upgrade";
           #proxy_http_version 1.1;
       }
}

So that's it after adding the configs to NginX you need to restart or reload NginX.

Starting the cluster and accessing the cluster

If you haven't already created the necessary tables in the DBs you can start the servers in the following way to create necessary tables automatically.

./wso2server.sh -Dsetup

After starting the manager node you can start workers one by one, if the members successfully join the cluster a log will be printed like following.

TID: [-1] [] [2015-08-21 12:07:50,815]  INFO {org.wso2.carbon.core.clustering.hazelcast.wka.WKABasedMembershipScheme} -  Member joined [6974ca1c-8403-4711-9408-9de0cfaadda2]: /192.168.48.244:4100 {org.wso2.carbon.core.clustering.hazelcast.wka.WKABasedMembershipScheme}
TID: [-1] [] [2015-08-21 12:08:05,043]  INFO {org.wso2.carbon.core.clustering.hazelcast.wka.WKABasedMembershipScheme} -  Member joined [3ebb327b-91db-4d98-8c8a-95e2604e3a9c]: /192.168.48.249:4100 {org.wso2.carbon.core.clustering.hazelcast.wka.WKABasedMembershipScheme}
TID: [-1] [] [2015-08-21 12:08:07,159]  INFO {org.wso2.carbon.core.clustering.hazelcast.util.MemberUtils} -  Added member: Host:192.168.48.249, Remote Host:null, Port: 4100, HTTP:8280, HTTPS:8243, Domain: wso2.esb49.cl2.domain, Sub-domain:__$default, Active:true {org.wso2.carbon.core.clustering.hazelcast.util.MemberUtils}

After successfully starting the cluster you should be able to access the cluster in the following manner.
https://mgt.esb.cl2.wso2.com/carbon

 Note : Make sure you add necessary host entries to your /etc/hosts file. So you can access the URLs with the host names. Your host names should resolve to the NginX node.

So that's it we have sucessfully created a ESB cluster, Please drop a comment if you have any queries. :)

Read More