Tomcat clustering configuration

The following steps assume that you have installed a Tomcat 5.5.x bundle or latest, i only tested on 5.5.27 but is should work for other configuration as well. The network configuration apply to Linux and may vary with the distribution. It should work as is for distributions based on Red Hat.

For Tomcat clustering we have two main things to configure:

Configure the network support for cluster

Opening Specific HTTP Ports (e.g. Port 45564, 4001)

The cluster class will start up a membership service (multicast) and a replication service (tcp unicast). See also http://www.cyberciti.biz/faq/howto-rhel-linux-open-port-using-iptables/ for a brief article on this. You will need to have root access as noted above to complete this.

Your server may or may not already have this entry. Open iptables:

> vi /etc/sysconfig/iptables

Add the following entries:

-A RH-Firewall-1-INPUT -p udp -m udp --dport 45564 -j ACCEPT
-A RH-Firewall-1-INPUT -p tcp -m tcp --dport 45564 -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 4001 -j ACCEPT

Save and close the above file and after restart the iptables

  > /etc/init.d/iptables restart

Configure the multicast address and routes.

Clustering membership is established using very simple multicast pings. Each Tomcat instance will periodically send out a multicast ping, in the ping message the instance will broad cast its IP and TCP listen port for replication. If an instance has not received such a ping within a given timeframe, the member is considered dead.

Add route  (the server’s ip address)

sudo /sbin/route add 228.0.0.4 gw 10.72.10.1 dev bond0

Edit rc.local to make the change persistent through restarts.

sudo vim /etc/rc.d/rc.local

Add this line at the end (the server’s ip address)

/sbin/route add 228.0.0.4 gw 10.72.10.1 dev bond0

Configure Tomcat to support clustering.

Application clustering with Tomcat has two steps:

Enable Tomcat clustering support

You need to enable the cluster support in Tomcat by editing the server.xml file. Open server.xml

sudo vim /usr/local/tomcat-5.5.27/conf/server.xml

Enable clustering configuration in the configuration file, notice that the default configuration is using the DeltaManager which will replicate only the session’s changes and not the entire object:

<Cluster className="org.apache.catalina.cluster.tcp.SimpleTcpCluster"
managerClassName="org.apache.catalina.cluster.session.DeltaManager"
	expireSessionsOnShutdown="false"
	useDirtyFlag="true"
	notifyListenersOnReplication="true">
<Membership className="org.apache.catalina.cluster.mcast.McastService"
	mcastAddr="228.0.0.4"
	mcastPort="45564"
	mcastFrequency="500"
	mcastDropTime="3000"/>
<Receiver className="org.apache.catalina.cluster.tcp.ReplicationListener"
	tcpListenAddress="10.72.10.1"
	tcpListenPort="4001"
	tcpSelectorTimeout="100"
	tcpThreadCount="6"/>
<Sender className="org.apache.catalina.cluster.tcp.ReplicationTransmitter"
	replicationMode="pooled"
	ackTimeout="15000"
	waitForAck="true"/>
<Valve className="org.apache.catalina.cluster.tcp.ReplicationValve"
  filter=".*\.gif;.*\.js;.*\.jpg;.*\.png;.*\.htm;.*\.html;.*\.css;.*\.txt;"/>
<Deployer className="org.apache.catalina.cluster.deploy.FarmWarDeployer"
	tempDir="/tmp/war-temp/"
	deployDir="/tmp/war-deploy/"
	watchDir="/tmp/war-listen/"
	watchEnabled="false"/>
<ClusterListener className="org.apache.catalina.cluster.session.ClusterSessionListener"/>
  </Cluster>

One main condition for replication to work is that your session content is serializable. Add a _jvmRoute_ to your Tomcat Engine section From

  <Engine name="Catalina" defaultHost="localhost">

To

  <Engine name="Catalina" defaultHost="localhost" jvmRoute="tomcat1">

jvmRoute identifies unique a Tomcat instance in a cluster. If multiple servers are used I recommend you to use descriptive names.

Make your application clusterizable

Configuring Tomcat clustering is not enough to cluster your application. For that you need to tell Tomcat which application you want to be clusterizable. This is achieved in two ways:

Enable application clustering by ROOT.xml

Edit ROOT.xml file

 sudo vim /usr/local/tomcat-5.5.27/conf/Catalina/localhost/ROOT.xml

Look for

 <Context path="" cookies="true" distributable="true" crossContext="true">

Change it to

 <Context path="" debug="0" reloadable="true"
cookies="true" crossContext="false" privileged="false" >

Enable application clustering by editing the web.xml

Edit the web.xml file

 sudo vim /usr/local/tomcat-5.5.27/webapps/ROOT/WEB-INF/web.xml

Look for:

 <web-app xmlns="http://java.sun.com/xml/ns/j2ee" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://java.sun.com/xml/ns/j2ee http://java.sun.com/xml/ns/j2ee/web-app_2_4.xsd" version="2.4">
	<context-param>
	<param-name>contextClass</param-name>
	.............

Change it to:

 <web-app xmlns="http://java.sun.com/xml/ns/j2ee" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://java.sun.com/xml/ns/j2ee http://java.sun.com/xml/ns/j2ee/web-app_2_4.xsd" version="2.4">
	<distributable/>
	<context-param>
	<param-name>contextClass</param-name>
	.............

Restart Tomcat

 cd /usr/local/tomcat-5.5.27/bin/
sudo ./shutdown.sh
sudo ./startup.sh
or if you have a init script
sudo /etc/init.d/tomcat5 restart

You need to configure all the nodes in the cluster as detailed above. Every node should have unique name provided by “jvmRoute” attribute.

Further reading

Cluster-howto | http://tomcat.apache.org/tomcat-5.5-doc/cluster-howto.html

June 9, 2009 | Filed Under Architecture, Tomcat | 5 Comments