Clustering WSO2 ESB 4.9 with NginX

This post explain how to setup a WSO2 ESB cluster, here I will be using ESB 4.9, MySQL as the underline DB and NginX as the load balancer. Following diagram depicts the overall distribution of the cluster components.

Following are the host names IPs of each node.

No	Node Name	Host Name	IP	Local Member Port
1	Manager	mgt.esb.cl2.wso2.com	192.168.48.243	4001
2	Worker 01	esb.cl2.wso2.com	192.168.48.244	4001
3	Worker 02	esb.cl2.wso2.com	192.168.48.249	4001

Requisites.

So for the cluster I assume you have the following prerequisites fulfilled.

1. MySQL setup
2. NginX installed and configured
3. SVN server to use as the deployment synchronizer.

Note : I will be explaining how to setup the cluster in a distributed setup, meaning one instance will host one ESB instance. So we will not see any port conflicts. If you need setup ESB cluster in a single machine or want to host multiple instnces in a single machine  you need to offset each node to avoid port conflicts. To do that you can open <ESB_HOME>/repository/conf/carbon.xml  and change the following property.

 <Offset>0</Offset>

Also you need to specify unique local member ports for each node. This can be done in  <ESB_HOME>/repository/conf/axis2/axis2.xml under the following proerty

<parameter name="localMemberPort">4100</parameter>

Configuring the Manager Node

1. Open <ESB_HOME>/repository/conf/datasources/master-datasources.xml and add following two datasources. Do not remove the existing carbon datasource.

<datasource>

  <name>WSO2_REG_DB</name>
  <description>The datasource used for shared config and governance registry</description>
  <jndiConfig>
  <name>jdbc/WSO2SharedDB</name>
  </jndiConfig>
  <definition type="RDBMS">
  <configuration>
   <url>jdbc:mysql://192.168.48.6:3306/ESB49_REGDB</url>
   <username>esbuser</username>
   <password>wso2root</password>
   <driverClassName>com.mysql.jdbc.Driver</driverClassName>
   <maxActive>50</maxActive>
   <maxWait>60000</maxWait>
   <testOnBorrow>true</testOnBorrow>
   <validationQuery>SELECT 1</validationQuery>
   <validationInterval>30000</validationInterval>
  </configuration>
  </definition>
 </datasource>

<datasource>
  <name>WSO2_UM_DB</name>
  <description>The datasource used for registry and user manager</description>
  <jndiConfig>
  <name>jdbc/WSO2UmDB</name>
  </jndiConfig>
  <definition type="RDBMS">
  <configuration>
   <url>jdbc:mysql://192.168.48.6:3306/ESB49_USERDB</url>
   <username>esbuser</username>
   <password>wso2root</password>
   <driverClassName>com.mysql.jdbc.Driver</driverClassName>
   <maxActive>50</maxActive>
   <maxWait>60000</maxWait>
   <testOnBorrow>true</testOnBorrow>
   <validationQuery>SELECT 1</validationQuery>
   <validationInterval>30000</validationInterval>
  </configuration>
  </definition>
</datasource>

Note : Make sure you don't remove WSO2_CARBON_DB datasource. WSO2_CARBON_DB will be used as the local registry of each node which will point to the embedded H2 Database, WSO2_UM_DB will be used for user namagement and WSO2_REG_DB will be used as the shared registry among the cluster.

2. Now open <ESB_HOME>/repository/conf/carbon.xml

a. Change the following parameters to add the necessary host names

<HostName>esb.cl2.wso2.com</HostName>
<MgtHostName>mgt.esb.cl2.wso2.com</MgtHostName>

b. Inorder to set the Deploymeny syncronizer do the changes as shown below. Note that AutoCommit option is set to true in the manager node

<DeploymentSynchronizer>
        <Enabled>true</Enabled>
        <AutoCommit>true</AutoCommit>
        <AutoCheckout>true</AutoCheckout>
        <RepositoryType>svn</RepositoryType>
        <SvnUrl>http://192.168.42.273/svn/wso2qarepo/esb49repo</SvnUrl>
        <SvnUser>wso2user1</SvnUser>
        <SvnPassword>wso2pass123</SvnPassword>
        <SvnUrlAppendTenantId>true</SvnUrlAppendTenantId>
    </DeploymentSynchronizer>

3. Now open <ESB_HOME>/repository/conf/user-mgt.xml and change the following property as shown below.

<Property name="dataSource">jdbc/WSO2UmDB</Property>

Note : jdbc/WSO2UmDB is the JNDI Name of the datasource we declaired in step 01 for user management DB.

4. Open <ESB_HOME>/repository/conf/registry.xml and add the following entries. Do not delete existing entries.

<dbConfig name="sharedregistry">
<dataSource>jdbc/WSO2SharedDB</dataSource>
</dbConfig>
<remoteInstance url="https://localhost:9444/registry">
<id>instanceid</id>
<dbConfig>sharedregistry</dbConfig>
<readOnly>false</readOnly>
<enableCache>true</enableCache>
<registryRoot>/</registryRoot>
<cacheId>esbuser@jdbc:mysql://192.168.48.6:3306/ESB49_REGDB</cacheId>
</remoteInstance>
<mount path="/_system/config" overwrite="true">
<instanceId>instanceid</instanceId>
<targetPath>/_system/esbnodes</targetPath>
</mount>
<mount path="/_system/governance" overwrite="true">
<instanceId>instanceid</instanceId>
<targetPath>/_system/governance</targetPath>
</mount>

5. Now lets do cluster related configs. Open <ESB_HOME>/repository/conf/axis2/axis2.xml do the following changes.

a. Enable clustering as shown below. make enable true in the following property.

<clustering class="org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent" enable="true">

b.  Make sure that membershipScheme is set to wka (well known address based schema)

<parameter name="membershipScheme">wka</parameter>

c. Set the domain, all cluster nodes will join this domain. This is only used for the members to join the cluster.

<parameter name="domain">wso2.esb49.cl2.domain</parameter>

d. Set the local member host, add the IP of the manager node to this.

<parameter name="localMemberHost">192.168.48.243</parameter>

e. Specify the localmember port to 4100.

Note : if you are hosting all the nodes in the same machine each node should have a unique member port.

<parameter name="localMemberPort">4100</parameter>

f. Now lets add  well known members.

<members>
     <member>
        <hostName>192.168.48.244</hostName>
        <port>4100</port>
     </member>
</members>

Note : Here I'm specifying worker 01 as the well-known member, ideally you can specify all the nodes as well known members. But its optional. When a node goes down the well known address will be used to reconnect to the cluster by the disrupted node. 

g. Uncomment and edit the WSDLEPRPrefix element under org.apache.synapse.transport.passthru.PassThroughHttpListener and
org.apache.synapse.transport.passthru.PassThroughHttpSSLListener in the transportReceiver. Each specifies HTTP and HTTP endpoints for the WSDL url.

<parameter name="WSDLEPRPrefix" locked="false">https://esb.cl2.wso2.com:443</parameter>
<parameter name="WSDLEPRPrefix" locked="false">http://esb.cl2.wso2.com:80</parameter>

6. Now Open <ESB_HOME>repository/conf/tomcat/catalina-server.xml and add proxy ports

Under HTTP connection section add the following parameter.

proxyPort="80"

Full HTTP section will look like following,

<Connector  protocol="org.apache.coyote.http11.Http11NioProtocol"
                port="9763"
                proxyPort="80"
                redirectPort="9443" 
                bindOnInit="false"
                maxHttpHeaderSize="8192"
                acceptorThreadCount="2"
                maxThreads="250"
                minSpareThreads="50"
                disableUploadTimeout="false"
                connectionUploadTimeout="120000"
                maxKeepAliveRequests="200"
                acceptCount="200"
                server="WSO2 Carbon Server"
                compression="on"
                compressionMinSize="2048"
                noCompressionUserAgents="gozilla, traviata"
                compressableMimeType="text/html,text/javascript,application/x-javascript,application/javascript,application/xml,text/css,application/xslt+xml,text/xsl,image/gif,image/jpg,image/jpeg" 
                URIEncoding="UTF-8"/>

Now under HTTPs section add the following proxy port

proxyPort="443"

The Full config section will look like following

<Connector  protocol="org.apache.coyote.http11.Http11NioProtocol"
                port="9443"
                proxyPort="443"
                bindOnInit="false"
                sslProtocol="TLS"
                maxHttpHeaderSize="8192"
                acceptorThreadCount="2" 
                maxThreads="250"
                minSpareThreads="50"
                disableUploadTimeout="false"
                enableLookups="false"
                connectionUploadTimeout="120000"
                maxKeepAliveRequests="200"
                acceptCount="200"
                server="WSO2 Carbon Server"
                clientAuth="false"
                compression="on"
                scheme="https"
                secure="true"
                SSLEnabled="true"
                keystoreFile="${carbon.home}/repository/resources/security/wso2carbon.jks"
                keystorePass="wso2carbon" 
                compressionMinSize="2048"
                noCompressionUserAgents="gozilla, traviata"
                compressableMimeType="text/html,text/javascript,application/x-javascript,application/javascript,application/xml,text/css,application/xslt+xml,text/xsl,image/gif,image/jpg,image/jpeg"
                URIEncoding="UTF-8"/>

That's it, we have completed configuring the manager node. Now lets configure the worker node. Worker node is very similar to manager nodes configurations. But I will walk though worker 01's configurations so you won't miss anything :)

Configuring the Worker Node

1. Open <ESB_HOME>/repository/conf/datasources/master-datasources.xml and add following two datasources. Do not remove the existing carbon datasource.

<datasource>
  <name>WSO2_REG_DB</name>
  <description>The datasource used for shared config and governance registry</description>
  <jndiConfig>
  <name>jdbc/WSO2SharedDB</name>
  </jndiConfig>
  <definition type="RDBMS">
  <configuration>
   <url>jdbc:mysql://192.168.48.6:3306/ESB49_REGDB</url>
   <username>esbuser</username>
   <password>wso2root</password>
   <driverClassName>com.mysql.jdbc.Driver</driverClassName>
   <maxActive>50</maxActive>
   <maxWait>60000</maxWait>
   <testOnBorrow>true</testOnBorrow>
   <validationQuery>SELECT 1</validationQuery>
   <validationInterval>30000</validationInterval>
  </configuration>
  </definition>
 </datasource>

<datasource>
  <name>WSO2_UM_DB</name>
  <description>The datasource used for registry and user manager</description>
  <jndiConfig>
  <name>jdbc/WSO2UmDB</name>
  </jndiConfig>
  <definition type="RDBMS">
  <configuration>
   <url>jdbc:mysql://192.168.48.6:3306/ESB49_USERDB</url>
   <username>esbuser</username>
   <password>wso2root</password>
   <driverClassName>com.mysql.jdbc.Driver</driverClassName>
   <maxActive>50</maxActive>
   <maxWait>60000</maxWait>
   <testOnBorrow>true</testOnBorrow>
   <validationQuery>SELECT 1</validationQuery>
   <validationInterval>30000</validationInterval>
  </configuration>
  </definition>
</datasource>

Note : Make sure you don't remove WSO2_CARBON_DB datasource.

2. Now open <ESB_HOME>/repository/conf/carbon.xml

a. Change the following parameters to add the necessary host names

<HostName>esb.cl2.wso2.com</HostName>
<MgtHostName>mgt.esb.cl2.wso2.com</MgtHostName>

b. Inorder to set the Deploymeny syncronizer do the changes as shown below. Note that AutoCommit option is set to false in the worker node

    <DeploymentSynchronizer>
        <Enabled>true</Enabled>
        <AutoCommit>false</AutoCommit>
        <AutoCheckout>true</AutoCheckout>
        <RepositoryType>svn</RepositoryType>
        <SvnUrl>http://192.168.42.273/svn/wso2qarepo/esb49repo</SvnUrl>
        <SvnUser>wso2user1</SvnUser>
        <SvnPassword>wso2pass123</SvnPassword>
        <SvnUrlAppendTenantId>true</SvnUrlAppendTenantId>
    </DeploymentSynchronizer>

3. Now open <ESB_HOME>/repository/conf/user-mgt.xml and change the following property as shown below.

<Property name="dataSource">jdbc/WSO2UmDB</Property>

Note : jdbc/WSO2UmDB the JNDI Name of the datasource we declaired in step 01 for user management DB.

4. Open <ESB_HOME>/repository/conf/registry.xml and add the following extries. Do not delete existing entries.

<dbConfig name="sharedregistry">
<dataSource>jdbc/WSO2SharedDB</dataSource>
</dbConfig>
<remoteInstance url="https://localhost:9444/registry">
<id>instanceid</id>
<dbConfig>sharedregistry</dbConfig>
<readOnly>true</readOnly>
<enableCache>true</enableCache>
<registryRoot>/</registryRoot>
<cacheId>esbuser@jdbc:mysql://192.168.48.6:3306/ESB49_REGDB</cacheId>
</remoteInstance>
<mount path="/_system/config" overwrite="true">
<instanceId>instanceid</instanceId>
<targetPath>/_system/esbnodes</targetPath>
</mount>
<mount path="/_system/governance" overwrite="true">
<instanceId>instanceid</instanceId>
<targetPath>/_system/governance</targetPath>
</mount>

Note: Note that I have specified readOnly property as false. The worker should not add any artifacts to the registry. 

5. Now lets do cluster related configs. Open <ESB_HOME>/repository/conf/axis2/axis2.xml do the following changes.

a. Enable clustering as shown below. make enable true in the following property.

<clustering class="org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent" enable="true">

b.  Make sure that membershipScheme is set to wka (well known address based schema)

<parameter name="membershipScheme">wka</parameter>

c. Set the domain, all cluster nodes will join this domain. This is only used for the members to join the cluster.

<parameter name="domain">wso2.esb49.cl2.domain</parameter>

d. Set the local member host, add the IP of the worker 01 node to this.

<parameter name="localMemberHost">192.168.48.244</parameter>

e. Specify the localmember port to 4100.

<parameter name="localMemberPort">4100</parameter>

Note : if you are hosting all the nodes in the same machine each node should have a unique member port.

f. Now lets add  well known members.

<members>
     <member>
        <hostName>192.168.48.243</hostName>
        <port>4100</port>
     </member>

     <member>
        <hostName>192.168.48.249</hostName>
        <port>4100</port>
     </member>
</members>

Note : Here I'm specifying manager and worker 02 as the well-known member.

g. Uncomment and edit the WSDLEPRPrefix element under org.apache.synapse.transport.passthru.PassThroughHttpListener and

org.apache.synapse.transport.passthru.PassThroughHttpSSLListener in the transportReceiver. Each specifies HTTP and HTTP endpoints for the WSDL url.

<parameter name="WSDLEPRPrefix" locked="false">https://esb.cl2.wso2.com:443</parameter>
<parameter name="WSDLEPRPrefix" locked="false">http://esb.cl2.wso2.com:80</parameter>

6. Now Open <ESB_HOME>repository/conf/tomcat/catalina-server.xml and add proxy ports

Under HTTP connection section add the following parameter.

proxyPort="80"

Full HTTP section will look like following,

<Connector  protocol="org.apache.coyote.http11.Http11NioProtocol"
                port="9763"
                proxyPort="80"
                redirectPort="9443" 
                bindOnInit="false"
                maxHttpHeaderSize="8192"
                acceptorThreadCount="2"
                maxThreads="250"
                minSpareThreads="50"
                disableUploadTimeout="false"
                connectionUploadTimeout="120000"
                maxKeepAliveRequests="200"
                acceptCount="200"
                server="WSO2 Carbon Server"
                compression="on"
                compressionMinSize="2048"
                noCompressionUserAgents="gozilla, traviata"
                compressableMimeType="text/html,text/javascript,application/x-javascript,application/javascript,application/xml,text/css,application/xslt+xml,text/xsl,image/gif,image/jpg,image/jpeg" 
                URIEncoding="UTF-8"/>

Now under HTTPs section add the following proxy port

proxyPort="443"

The Full config section will look like following

<Connector  protocol="org.apache.coyote.http11.Http11NioProtocol"
                port="9443"
                proxyPort="443"
                bindOnInit="false"
                sslProtocol="TLS"
                maxHttpHeaderSize="8192"
                acceptorThreadCount="2" 
                maxThreads="250"
                minSpareThreads="50"
                disableUploadTimeout="false"
                enableLookups="false"
                connectionUploadTimeout="120000"
                maxKeepAliveRequests="200"
                acceptCount="200"
                server="WSO2 Carbon Server"
                clientAuth="false"
                compression="on"
                scheme="https"
                secure="true"
                SSLEnabled="true"
                keystoreFile="${carbon.home}/repository/resources/security/wso2carbon.jks"
                keystorePass="wso2carbon" 
                compressionMinSize="2048"
                noCompressionUserAgents="gozilla, traviata"
                compressableMimeType="text/html,text/javascript,application/x-javascript,application/javascript,application/xml,text/css,application/xslt+xml,text/xsl,image/gif,image/jpg,image/jpeg"
                URIEncoding="UTF-8"/>

7. Open <ESB_HOME>/bin/wso2server.sh and serch for DworkerNode parameter and change the value to true as shown below.

-DworkerNode=false \

So we have configured one of the worker noded, worker node 02s configurations are identical to the worker 01s change the nesassary ips in relevent places when configuring worker 02.

Copying necessary Drivers and libraries

You need to copy following drivers to both Worker and Manager nodes. 

JDBC Driver

Copy MYSQL JDBC driver to <ESB_HOME>/repository/components/lib directory. You can find the matching driver for your mysql version from here. 

SVN related JARs

1. Download and copy SVNKit from http://product-dist.wso2.com/tools/svnkit-all-1.8.7.wso2v1.jar into the <ESB_HOME>/repository/components/dropins folder.

2. Download http://maven.wso2.org/nexus/content/groups/wso2-public/com/trilead/trilead-ssh2/1.0.0-build215/trilead-ssh2-1.0.0-build215.jar and copy it to the <ESB_HOME>/repository/components/lib folder.

Configuring NginX

So ESB has two host names, management console host and worker host entries. And ESB also exposes 4 different ports (9443/9763, 8280/8243). NginX will be listening to system default ports 80/443, and it will route all the requests to the respective ESB node. The NginX configs will look like following.

create a new NginX conf file at /etc/nginx/conf.d and add the following content to it.

Note : I'm using my IPs and host names, you should change these according to your setup.

upstream esb49wkhttps {
       
        server 192.168.48.244:8243;
        server 192.168.48.249:8243;
}

upstream esb49wkhttp {
       
        server 192.168.48.244:8280;
        server 192.168.48.249:8280;
}

upstream esb49mgt {
        server 192.168.48.243:9443;        
}


map $http_upgrade $connection_upgrade {
   default upgrade;
    '' close;
}

server {
        listen 80;
        server_name esb.cl2.wso2.com;

        location / {
           proxy_set_header X-Forwarded-Host $host;
           proxy_set_header X-Forwarded-Server $host;
           proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

           proxy_set_header Host $http_host;
           proxy_read_timeout 5m;
           proxy_send_timeout 5m;
   
   
           proxy_pass http://esb49wkhttp/;
           proxy_redirect http://esb49wkhttp/ http://gw.am.wso2.com/;
 
    proxy_set_header Upgrade $http_upgrade;
           proxy_set_header Connection "upgrade";
           proxy_http_version 1.1;
      }
}

server {

       listen 443;
       server_name esb.cl2.wso2.com;

       ssl on;
       ssl_certificate /etc/nginx/ssl/am_wso2_com.crt;
       ssl_certificate_key /etc/nginx/ssl/am_wso2_com.key;

       location / {
           proxy_set_header X-Forwarded-Host $host;
           proxy_set_header X-Forwarded-Server $host;
           proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

           proxy_set_header Host $http_host;
           proxy_read_timeout 5m;
           proxy_send_timeout 5m;

           proxy_pass https://esb49wkhttps/;
           proxy_redirect https://esb49wkhttps/ https://esb.cl2.wso2.com/;

    #proxy_set_header Upgrade $http_upgrade;
           #proxy_set_header Connection "upgrade";
           #proxy_http_version 1.1;
       }
}

server {

       listen 443;
       server_name mgt.esb.cl2.wso2.com;

       ssl on;
       ssl_certificate /etc/nginx/ssl/am_wso2_com.crt;
       ssl_certificate_key /etc/nginx/ssl/am_wso2_com.key;

       location / {
           proxy_set_header X-Forwarded-Host $host;
           proxy_set_header X-Forwarded-Server $host;
           proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

           proxy_set_header Host $http_host;
           proxy_read_timeout 5m;
           proxy_send_timeout 5m;

           proxy_pass https://esb49mgt/;
           proxy_redirect https://esb49mgt/ https://mgt.esb.cl2.wso2.com/;

           #proxy_set_header Upgrade $http_upgrade;
           #proxy_set_header Connection "upgrade";
           #proxy_http_version 1.1;
       }
}

So that's it after adding the configs to NginX you need to restart or reload NginX.

Starting the cluster and accessing the cluster

If you haven't already created the necessary tables in the DBs you can start the servers in the following way to create necessary tables automatically.

./wso2server.sh -Dsetup

After starting the manager node you can start workers one by one, if the members successfully join the cluster a log will be printed like following.

TID: [-1] [] [2015-08-21 12:07:50,815]  INFO {org.wso2.carbon.core.clustering.hazelcast.wka.WKABasedMembershipScheme} -  Member joined [6974ca1c-8403-4711-9408-9de0cfaadda2]: /192.168.48.244:4100 {org.wso2.carbon.core.clustering.hazelcast.wka.WKABasedMembershipScheme}
TID: [-1] [] [2015-08-21 12:08:05,043]  INFO {org.wso2.carbon.core.clustering.hazelcast.wka.WKABasedMembershipScheme} -  Member joined [3ebb327b-91db-4d98-8c8a-95e2604e3a9c]: /192.168.48.249:4100 {org.wso2.carbon.core.clustering.hazelcast.wka.WKABasedMembershipScheme}
TID: [-1] [] [2015-08-21 12:08:07,159]  INFO {org.wso2.carbon.core.clustering.hazelcast.util.MemberUtils} -  Added member: Host:192.168.48.249, Remote Host:null, Port: 4100, HTTP:8280, HTTPS:8243, Domain: wso2.esb49.cl2.domain, Sub-domain:__$default, Active:true {org.wso2.carbon.core.clustering.hazelcast.util.MemberUtils}

After successfully starting the cluster you should be able to access the cluster in the following manner.
https://mgt.esb.cl2.wso2.com/carbon

 Note : Make sure you add necessary host entries to your /etc/hosts file. So you can access the URLs with the host names. Your host names should resolve to the NginX node.

So that's it we have sucessfully created a ESB cluster, Please drop a comment if you have any queries. :)

Read More

How to take a DB Dump and Restore it in Oracle RAC 11gR2

In this post I'll explain you how to take a DB dump in Oracle with deprecated imp/exp and using oracle data pump.

First lets create a user schema.

create user yasaas identified by yasas;
grant create session, grant any privilege to yasas;
grant all privileges to yasas;

Above commands will give most of the privileges to the user that are required for DB operations. But when exporting and importing the user needs database export and import privileges. So execute the following to give necessary privileges to the user.

grant exp_full_database to yasas;
grant imp_full_database to yasas;

Note: If you do not set the above previledges it will cause the following error

ORA-31631: privileges are required

ORA-39161: Full database jobs require privileges

Now add some tables and data to the schema. And we are ready to export the schema. You should note that there are two ways to do this

• Using deprecated  imp/exp command which came with 10G
• Using Oracle Datapump

If you use the data pump you will not be able to import the same dump to a diferent oracle server.

First lets see how to export and import the dump via exp command.

Exporting via exp command

Execute the following commands to export the dp to a dump file. Folllowing are OS commands.

exp yasas/yasas@192.168.23.112:1521/rac.localdomain owner=yasas file=mydbdump.dmp log=dump_log.log

If you are executing the command from the server it self you can do it like following

exp yasas/yasas owner=yasas file=mydbdump.dmp log=export_dump.log

Note : If you want to export the entire DB you can parse the full=y parameter instead of owner=yasas and talso not that the export log will be written to the specified file.

Importing the exported dump via imp command

You can import the dump file with following command

imp yasas/yasas file=mydbdump.dmp log=import_dump.log full=y

Note : if there are multiple sachems in the dump you can use 'fromuser' and 'touser' parameters to import this, also ignore=y is also use full to ignore any errors when importing. 

Exporting via expdp command

Before exporting the dump you should set the data pump directory, all the dumps and logs will be created here. First lets create a physicaldirectory.

mkdir /home/oracle/datapump

Now login to the DB via SQLPLUS or other client and execute the following.

create or replace directory sql_data_pump as '/home/oracle/datapump';
grant read,write on DIRECTORY sql_data_pump to yasas;
grant exp_full_database to yasas;

Now lets Export the dumps. These are OS commands.

expdp yasas/yasas schemas=yasas dumpfile=mydbdump.dmp logfile=explog.log;

Importing via impdp command

As shown above for the import you need to set data pump directory.

create or replace directory sql_data_pump as '/home/oracle/datapump';
grant read,write on DIRECTORY sql_data_pump to yasas;
grant imp_full_database to yasas;

Lets import the dump now,

impdp yasas/yasas schemas=yasas dumpfile=mydbdump.dmp logfile=implog.log;

Note : If you are executing this in a windows environment you might need to add directory parameter to the above command and give the data_pump directory name that was set.

e.g : impdp yasas/yasas schemas=yasas directory=sql_data_pump dumpfile=mydbdump.dmp logfile=implog.log;

So that's it, please drop a comment if you have any queries.

Read More

How to Get IMP-00003 and ORA-00959 fixed

When importing a DB dump of a different schema you may come across the followiing two errors.

IMP-00003: ORACLE error 959 encountered
ORA-00959: tablespace 'DEV_GOV' does not exist

To get this issue fixed you can create the missing table space and point to the related table space when creating the schema.

Execute following to create new table space

CREATE TABLESPACE "DEV_GOV" DATAFILE '/home/oracle/dumps/file/DEV_GOV.dbf' SIZE 20M

The DEV_GOV.dbf file will be created automatically when you execute the above command.

Now create the user by setting the default table space of the user to above created table space.

create user DEV_AM identified by rootpass default tablespace DEV_GOV;
grant create session, grant any privilege to DEV_AM;
grant all privileges to DEV_AM;
grant imp_full_database to DEV_AM;

Now you should be able to import the dumps :)

Read More

How to Setup Simple SVN server, Expose it via DAV with Apache2 Over SSL in Ubuntu

In this Post I will explain you how to install subversion and expose it via Apache2 server while enabling SSL. So I have blogged on how to install Apache2 and enable SSL here. So As a prerequisite follow the above blog and install Apache2 server and configure SSL.

So lets install and configure subversion with Apache2.

Installing Subversion

First update Ubuntu repositories

sudo apt-get update
sudo apt-get upgrade

Now install Subversion

sudo apt-get install subversion

After installing you can verify the installed version by executing the following command

svn --version

Now create a directory and navigate to that directory

mkdir ~/svn
cd ~/svn

By executing following create a new SVN repository

svnadmin create svnqarepo

Now changer ownership of the directory

1	sudo chown -R www-data:www-data *

change permission of the directory

chmod 755 -R *

Configure Apache2 with SVN

First lets install SVN related packages to Apache2

sudo apt-get install libapache2-svn

Install extra Packages that are required

sudo apt-get install apache2-utils

Note: apache2-utills are required to htpasswd when adding users

Now Open /etc/apache2/mods-available/dav_svn.conf and add the following

1 2 3 4 5 6 7 8

<Location /svn> DAV svn SVNPath /home/ubuntu/svn/svnqarepo AuthType Basic AuthName "myproject subversion repository" AuthUserFile /etc/subversion/passwd Require valid-user </Location>

Now lets add users, to add initial users execute the following 

sudo htpasswd -c /etc/subversion/passwd wso2qauser

Execute following command and restart Apache server

sudo a2enmod dav_svn

sudo service apache2 restart

Type the password when password is asked. Now you should be able to access the SVN repository with the following URL

http://<HOST_NAME>/svn
https://<HOST_NAME>/svn

When you access the URL it will be like following, and username/password should be entered when accessing the repository.

Now you are done configuring the server, Please drop a comment if you have queries. :)

Read More

How to Install Apache2 server and enable SSL in Ubuntu

In this post I'll explain you how to install Apache2 Server and enable SSL for Apache2. Here I'm working on ubuntu 14.04 LTS. So lets get started.

Installing Apache2

First Lets update the Ubuntu repositories.

sudo apt-get install udgrade
sudo apt-get install update

Now Lets install apache2

sudo apt-get install apache2

Enable SSL in Apache Server

Create a directory as shown below,

sudo mkdir /etc/apache2/ssl

Now go to the created directory

cd /etc/apache2/ssl

Creating the certificates. 

Execute the following commands to create the key

sudo openssl genrsa -out "apache.key" 2048

Execute the the following to create the certificate sign request

sudo openssl req -new -key "apache.key" -out "apache.csr"

Fill information similar to following whencreating the CSR

Lets Sign the Key

sudo openssl x509 -req -days 3650 -in "apache.csr" -signkey "apache.key" -out "apache.crt"

Now Enable SSL module, if this is module is not installed you will have to install it :). Execute following to enable SSL in apache2

sudo a2enmod ssl

Now add the following to 000-default.conf in sites-enabled which is located at /etc/apache2/sites-enabled/000-default.conf

<VirtualHost *:443>
   
    SSLEngine on
    SSLCertificateFile  /etc/apache2/ssl/apache.crt
    SSLCertificateKeyFile /etc/apache2/ssl/apache.key

</VirtualHost>

Note : You can define server names and other parameters when configurin this further, above is the minimum content that is required.

Now restart the server.

sudo service apache2 restart

That is it, Now you are good to go :).

Please drop a comment if you have any Queries.

Read More

Installing RabitMQ Server and Management Console in Ubuntu

Hi All, In this post I'll describe how you can install RabbitMQ sever and Management console in the easy way using apt-get :). You can follow the following instructions to get this done.

Installing RabbitMQ Server

1. Add the following line to your /etc/apt/sources.list file

   deb http://www.rabbitmq.com/debian/ testing main

2. To avoid warnings about unsigned packages, add repo public key to your trusted key list using apt-key: execute following commads,

   wget https://www.rabbitmq.com/rabbitmq-signing-key-public.asc
   sudo apt-key add rabbitmq-signing-key-public.asc

   
   
3. Run apt-get update.

1. Install packages for RabitMQ server, you can execute the following command for this.

   sudo apt-get install rabbitmq-server

Note : After Installing RabbitMQ some default configuration files may not exist by default so you need to create these files manually in relevant locations if you need to change default settings of RabbitMQ.
For e,g.: If rabbitmq-env.conf is not available create the file in /etc/rabbitmq/rabbitmq-env.conf and you can add the parameters you need. In the same way you also can add /etc/rabbitmq/rabbitmq.config file as well, in this you can define system parameters which will be used by RabbitMQ. For now we will continue with the default settings. 

Install the Management interface

The management plugin is included in the RabbitMQ distribution. To enable it, use rabbitmq-plugins execute the following command

sudo rabbitmq-plugins enable rabbitmq_management

Now lets create a new user and Grant Permission to the suer to enter the management console and perform queue operations. For this execute the followig commands It will create a user named admin with password admin.

sudo rabbitmqctl add_user admin admin

sudo rabbitmqctl set_user_tags admin administrator

sudo rabbitmqctl set_permissions -p / admin ".*" ".*" ".*"

Now you should be able to access the management console in the following URL.

http://192.168.48.240:15672/#/

It is also recommend to setup OS level parameters to tuneup RabbitMQ. e.g: ulimits etc. But for testing purposes above configurations will do.


Now You can navigate to the Queue section of the management console and create a new Queue.

Read More

Adding a delay to the response when invoking a SoapUI mock service

In this post I will explain you how to add a delay to a response when working with mock services in SOAPUI. This is quit simple, in-order to create a SOAP UI mock service you follow this guide. After creating a mock service do the following to add a delay to the response.

1. Open The mock service and click on the specific action  and open the response tab as shown below.

2. Now from dispatch option select Script option.

3. Now add the following code to the content. Please note that I'm using groovy as the scripting language and you can change this from settings.

Thread.sleep(5000)

You can change the delay by changing the value note that the value is specified in milliseconds. This will add a delay to the response. :)

Read More

How to Completely Remove/Uninstall Apache2 Server (Apache HTTP Server)

After installing Apache Server using apt-get it can be quit difficult to remove all the packages that were installed together with Apache2. You can follow the following instructions to completely get rid of Apache2 Server.

1. Remove the apache server.

sudo apt-get --purge remove apache2


2. Now remove any remaining packages

a. Search whether there are any remains pakcages. To see this execute the following command

dpkg --get-selections | grep -v deinstall | grep apache*

This will give a simmilar output as following.

apache2-bin         install

apache2-data         install

apache2-utils         install

libapache2-mod-proxy-html install

libapache2-mod-svn install

libapache2-mpm-itk install

libapache2-svn install

b. Now you should remove each of these packages one by one as shown below.

 sudo apt-get --purge remove apache2-bin

sudo apt-get --purge remove apache2-data

sudo apt-get --purge remove apache2-utils              etc.

After completely removing each package you should be able to get rid of Apache2 server. You can confirm this by checking whether there are any services relevant to apache2.

e.g : sudo service apache2 start

The above command will give an error saying that the service is not available.

Read More