[WSO2] Clustering EI Analytics Profile

Setup

  • Enterprise Integrator 6.1.0
  • Analytics profile configure with Oracle 12 c database
  • Analytics two nodes in separate instances

Data source Configuration

  1. Install Oracle database server.
  2. Create databases in Oracle.

Create

  • REGISTRY database (ei610regana),
  • USER MANAGEMENT database (),
  • WSO2_ANALYTICS_EVENT_STORE_DB (anaevesto61),
  • WSO2_ANALYTICS_PROCESSED_DATA_STORE_DB (anaprodstor61) and
  • Two carbon databases (wso2carbonana1, wso2carbonana2) as we need 2 local databases for analytics nodes.
drop user ei610regana cascade;
Create user ei610regana identified by ei610regana account unlock;
grant connect to ei610regana;
grant resource, create session, dba to ei610regana;
grant unlimited tablespace to ei610regana;
grant create table to ei610regana;
commit;
drop user ei610user cascade;
Create user ei610user identified by ei610user account unlock;
grant connect to ei610user;
grant resource, create session, dba to ei610user;
grant unlimited tablespace to ei610user;
grant create table to ei610user;
commit;
drop user anaevesto61 cascade;
Create user anaevesto61 identified by anaevesto61 account unlock;
grant connect to anaevesto61;
grant resource, create session, dba to anaevesto61;
grant unlimited tablespace to anaevesto61;
grant create table to anaevesto61;
commit;
drop user anaprodstor61 cascade;
Create user anaprodstor61 identified by anaprodstor61 account unlock;
grant connect to anaprodstor61;
grant resource, create session, dba to anaprodstor61;
grant unlimited tablespace to anaprodstor61;
grant create table to anaprodstor61;
commit;
drop user wso2carbonana1 cascade;
Create user wso2carbonana1 identified by wso2carbonana1 account unlock;
grant connect to wso2carbonana1;
grant resource, create session, dba to wso2carbonana1;
grant unlimited tablespace to wso2carbonana1;
grant create table to wso2carbonana1;
commit;
drop user wso2carbonana2 cascade;
Create user wso2carbonana2 identified by wso2carbonana2 account unlock;
grant connect to wso2carbonana2;
grant resource, create session, dba to wso2carbonana2;
grant unlimited tablespace to wso2carbonana2;
grant create table to wso2carbonana2;
commit;

Configure data source related configs in EI Analytic profile

3. Download Oracle JDBC driver. Add driver jar in to lib folder ($EI_HOME/lib)

4. Add carbon data source details in to master-datasources.xml ($EI_HOME/wso2/analytics/conf/datasources)

Replace <ip >with instance IP of which installed oracle server.

<datasources>
<datasource>
<name>WSO2_CARBON_DB</name>
<description>The datasource used for registry and user manager</description>
<jndiConfig>
<name>jdbc/WSO2CarbonDB</name>
</jndiConfig>
<definition type="RDBMS">
<configuration>
<url>jdbc:oracle:thin:@<ip>:1521/pdbora12c</url>
<username>wso2carbonana1</username>
<password>wso2carbonana1</password>
<driverClassName>oracle.jdbc.driver.OracleDriver</driverClassName>
<maxActive>50</maxActive>
<maxWait>60000</maxWait>
<testOnBorrow>true</testOnBorrow>
<validationQuery>SELECT 1</validationQuery>
<validationInterval>30000</validationInterval>
<defaultAutoCommit>false</defaultAutoCommit>
</configuration>
</definition>
</datasource>
<datasource>
<name>WSO2_REG_DB</name>
<description>The datasource used for registry and user manager</description>
<jndiConfig>
<name>jdbc/WSO2RegDB</name>
</jndiConfig>
<definition type="RDBMS">
<configuration>
<url>jdbc:oracle:thin:@<ip>:1521/pdbora12c</url>
<username>ei610regana</username>
<password>ei610regana</password>
<driverClassName>oracle.jdbc.driver.OracleDriver</driverClassName>
<maxActive>50</maxActive>
<maxWait>60000</maxWait>
<testOnBorrow>true</testOnBorrow>
<validationQuery>SELECT 1</validationQuery>
<validationInterval>30000</validationInterval>
<defaultAutoCommit>false</defaultAutoCommit>
</configuration>
</definition>
</datasource>
<datasource>
<name>WSO2_UM_DB</name>
<description>The datasource used for registry and user manager</description>
<jndiConfig>
<name>jdbc/WSO2UMDB</name>
</jndiConfig>
<definition type="RDBMS">
<configuration>
<url>jdbc:oracle:thin:@<ip>:1521/pdbora12c</url>
<username>ei610user</username>
<password>ei610user</password>
<driverClassName>oracle.jdbc.driver.OracleDriver</driverClassName>
<maxActive>50</maxActive>
<maxWait>60000</maxWait>
<testOnBorrow>true</testOnBorrow>
<validationQuery>SELECT 1 FROM DUAL</validationQuery>
<validationInterval>30000</validationInterval>
</configuration>
</definition>
</datasource>
</datasources>

5. Add WSO2_ANALYTICS_EVENT_STORE_DB and WSO2_ANALYTICS_PROCESSED_DATA_STORE_DB data source details in to analytics-datasources.xml ($EI_HOME/wso2/analytics/conf/datasources).

Replace <ip >with instance IP of which installed oracle server.

<datasource>
<name>WSO2_ANALYTICS_EVENT_STORE_DB</name>
<description>The datasource used for analytics record store</description>
<definition type="RDBMS">
<configuration>
<url>jdbc:oracle:thin:@<ip>:1521/pdbora12c</url>
<username>anaevesto61</username>
<password>anaevesto61</password>
<driverClassName>oracle.jdbc.driver.OracleDriver</driverClassName>
<maxActive>50</maxActive>
<maxWait>60000</maxWait>
<validationQuery>SELECT 1 FROM DUAL</validationQuery>
<defaultAutoCommit>false</defaultAutoCommit>
<initialSize>0</initialSize>
<testWhileIdle>true</testWhileIdle>
<minEvictableIdleTimeMillis>4000</minEvictableIdleTimeMillis>
<defaultTransactionIsolation>READ_COMMITTED</defaultTransactionIsolation>
</configuration>
</definition>
</datasource>
<datasource>
<name>WSO2_ANALYTICS_PROCESSED_DATA_STORE_DB</name>
<description>The datasource used for analytics record store</description>
<definition type="RDBMS">
<configuration>
<url>jdbc:oracle:thin:@<ip>:1521/pdbora12c</url>
<username>anaprodstor61</username>
<password>anaprodstor61</password>
<driverClassName>oracle.jdbc.driver.OracleDriver</driverClassName>
<maxActive>50</maxActive>
<maxWait>60000</maxWait>
<validationQuery>SELECT 1 FROM DUAL</validationQuery>
<defaultAutoCommit>false</defaultAutoCommit>
<initialSize>0</initialSize>
<testWhileIdle>true</testWhileIdle>
<minEvictableIdleTimeMillis>4000</minEvictableIdleTimeMillis>
<defaultTransactionIsolation>READ_COMMITTED</defaultTransactionIsolation>
</configuration>
</definition>
</datasource>
</datasources>

6. To share the user store among the nodes, modify user-mgt.xml ($EI_HOME/wso2/analytics/conf) as follows:

[Change <Property name=”dataSource”> as given]

 <Property name=”dataSource”>jdbc/WSO2UMDB</Property>

7. Add or modify “govregistry” datasource attribute in registry.xml ($EI_HOME/wso2/analytics/conf) as follows:

Replace <ip >with instance IP of which installed oracle server.

<dbConfig name="wso2registry">
<dataSource>jdbc/WSO2CarbonDB</dataSource>
</dbConfig>
<dbConfig name="govregistry">
<dataSource>jdbc/WSO2RegDB</dataSource>
</dbConfig>
<remoteInstance url="https://localhost:9443/registry">
<id>gov</id>
<cacheId>jdbc:oracle:thin:@<ip>:1521/pdbora12c</cacheId>
<dbConfig>govregistry</dbConfig>
<readOnly>false</readOnly>
<enableCache>true</enableCache>
<registryRoot>/</registryRoot>
</remoteInstance>
<mount path="/_system/governance" overwrite="true">
<instanceId>gov</instanceId>
<targetPath>/_system/governance</targetPath>
</mount>
<mount path="/_system/config" overwrite="true">
<instanceId>gov</instanceId>
<targetPath>/_system/config</targetPath>
</mount>

Configure clustering related configs in EI Analytic profile

8. Edit axis2.xml ($EI_HOME/wso2/analytics/conf/axis2/) in each node as follows.

  • Enable clustering.
<clustering class = "org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent" enable="true">
  • Enable wka mode.
<parameter name="membershipScheme">wka</parameter>
  • Add other analytics profile nodes details under members tag.
<members><member><hostName>[node1 IP]</hostName><port>[node1 port]</port></member><member><hostName>[node2 IP]</hostName><port>[node2 port]</port></member></members>

Sample Config:

<members>
<member>
<hostName>192.168.**.***</hostName>
<port>4100</port>
</member>
</members>
  • For each node, enter the respective server IP address as the value for the localMemberHost property

<parameter name="localMemberHost">[Server_IP_Address]</parameter>

9. Configure the event-processor.xml ($EI_HOME/wso2/analytics/conf/) file as follows to cluster CEP in the Receiver.

  • Enable the HA mode

<mode name="HA" enable="true">

  • Disable the Distributed mode

<mode name="Distributed" enable="false">

  • For each node, enter the respective server IP address under the HA mode Config section.
<eventSync>
<hostName>*.*.*.*</hostName>

10. Update spark-defaults.conf ($EI_HOME/wso2/analytics/conf/analytics /spark/) file as follows to use the Spark cluster embedded within Analytic profile.

  • Keep the carbon.spark.master configuration as local. This instructs Spark to create a Spark cluster using the Hazelcast cluster.
  • Enter 2 as the value for the carbon.spark.master.count configuration. This specifies that there should be two masters in the Spark cluster. One master serves as an active master and the other serves as a stand-by master. (Change count according to number of nodes)
carbon.spark.master local
carbon.spark.master.count 2
  • Create a symbolic link to <$AnalyticProfile_Home>in both nodes, where paths of those symbolic links are identical. This ensures that if we use the symbolic link to access the analytic profile, we can use a common path.

In this scenario we are having analytics profile in two different instances which are having two different OSes. Therefore choose a common directory as follows:

/usr/share/das/das_symlink

Use following command to create symlink folder.

ln -s <$AnalyticProfile_Home> <$Symlink_Folder>

Sample config:

ln -s /home/debian/analytics2/wso2ei-6.1.0/wso2/analytics /usr/share/das/das_symlink61

Give permission to that folder to have access while creating analytics cluster. (User group and user name should be change accordingly). Use following command for that.

sudo chown -R username:group directory

Ex: sudo chown -R cloud-user:cloud-user /user/share/das/das_symlink61

Check for updated permission for the folder using command ls -al

[cloud-user@ei6s2-analytics1 das]$ ls -la

Sample output:

lrwxrwxrwx. 1 cloud-user cloud-user 69 මාර් 17 11:35 das_symlink61 -> /home/cloud-user/EI610Analytics1/wso2ei-6.1.0-SNAPSHOT/wso2/analytics

Now you are done with creating symlink.

Add symlink in spark-defaults.conf file.

carbon.das.symbolic.link /usr/share/das/das_symlink61/

11. Add followings in to wso2server.sh ($EI_Home/wso2/analytics/bin/)

-Dcarbon.das.symlink.config.dir.path=<SYM_LINK_PATH>/conf
-Dcarbon.das.symlink.external.lib.dir.path=<SYM_LINK_PATH>/../../lib
-Dcarbon.das.symlink.dropins.dir.path=<SYM_LINK_PATH>/../../dropins
-Dcarbon.das.symlink.internal.lib.dir.path=<SYM_LINK_PATH>/../lib
-Ddas.symlink.components.repo=<SYM_LINK_PATH>/../components/plugins

Sample config:

ANA_HOME=”/usr/share/das/das_symlink61"
...
-Dcarbon.das.symlink.config.dir.path=$ANA_HOME/conf \
-Dcarbon.das.symlink.external.lib.dir.path=$ANA_HOME/../../lib \
-Dcarbon.das.symlink.dropins.dir.path=$ANA_HOME/../../dropins \
-Dcarbon.das.symlink.internal.lib.dir.path=$ANA_HOME/../lib \
-Ddas.symlink.components.repo=$ANA_HOME/../components/plugins \

12. Add following in to load-spark-env-vars.sh file ($EI_Home/wso2/analytics/bin/).

export CARBON_CONFIG_DIR_PATH=/usr/share/das/das_symlink61/conf

Note: Replace path with given path.

You are done with configurations :) .

When you started, you can check for spark cluster to identify status. Default url for spark cluster : <ip_instance>:8082/

Sample URL:

http://192.168.**.**:8082/

Note: As we didn’t configure the deployment synchronizer, it is required to deploy any C-App use in the Analytics profile cluster set up to both the nodes.

Important Notes

  1. When changing datasources/clean up datasources within cluster, you should have to clean following folders too. ( rm -r * ).
  • solr/*
  • tmp/*
  • repository/data/*
  • repository/database/*

2. When you change symplinks, clear above folders and additionally followings too.

  • work/*

3. When you are going start cluster, start one node and in parallel you can start other node too. Otherwise you will get known issue as follows:

ERROR {org.wso2.carbon.event.processor.manager.core.internal.HAManager} -  CEP HA State syncing failed, No execution plans exist for tenant  -1234
org.wso2.carbon.event.processor.manager.core.exception.EventManagementException: No execution plans exist for tenant -1234
at org.wso2.carbon.event.processor.core.internal.CarbonEventProcessorManagementService.restoreState(CarbonEventProcessorManagementService.java:83)
at org.wso2.carbon.event.processor.manager.core.internal.HAManager.syncState(HAManager.java:336)
at org.wso2.carbon.event.processor.manager.core.internal.HAManager.access$100(HAManager.java:49)
at org.wso2.carbon.event.processor.manager.core.internal.HAManager$2.run(HAManager.java:276)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

Refer [1] for more details and for known issues.

References:

  1. https://docs.wso2.com/display/CLUSTER44x/Minimum+High+Availability+Deployment+-+DAS+3.1.0

--

--

--

Techie | Traveler | Writer

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Postgres and the Perils of Abstraction

Animating Zombies: Root Transforms

We created an Automated YouTube Channel in Python and it went viral 🚀

YouTube channel statistics showing amount of views & subscribers

The Art Of Technical Decision Making

What’s up, doc? Building Cytora’s Documentation system

edX Online Courses by Harvard, MIT, Berkeley, IBM Apk Download

edX  Online Courses by Harvard, MIT, Berkeley, IBM Apk Download

Designing Matched Bandstop Filter for Very High Frequency (VHF) Band

Unity3D: Cleaning Up Enemy Spawn

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Dilshani Subasinghe

Dilshani Subasinghe

Techie | Traveler | Writer

More from Medium

What is A/B Testing? Its Benefits and Steps to do Test

Costumer segmentation for an e-commerce retail

Application Lifecycle Management

Our Customer Satisfied or Not ?