The local Ehcache example shown in
part 2
implemented a local cache for a single Tomcat instance.
This section shows how the application can be extended to provide a
peer-to-peer-based shared cache across multiple Tomcat instances.
As shown below, Ehcache uses its listener architecture to provide a
fully replicated peer-to-peer cluster: Whenever a key/value pair is
written, updated, or deleted, a listener broadcasts the event to
all other instances in the network.
As the implementation below shows, Ehcache does not support atomic
operations in peer-to-peer mode. Therefore, the set-up below is only
useful in one of the following scenarios:
In scenarios where keys are modified by multiple nodes, Ehcache’s peer-to-peer
cluster will either throw an Exception or become inconsistent. In these cases,
Ehcache should be used in client-server mode,
as will be shown in part 4.
In order to test the configuration on one single machine, we use two
different configurations: One opening the RMI port 40001, and one
opening port 40002.
The name of the configuration file is passed via the system property
ehcache.config.filename, as shown in section “How to Run” below.
Apart from specifying a non-default config file name, initialization is
the same as with the simple Ehcache application.
@Override
public void contextInitialized(ServletContextEvent servletContextEvent) {
String cfg = "/" + System.getProperty("ehcache.config.filename", "ehcache1.xml");
URL url = getClass().getResource(cfg);
Cache cache = CacheManager.create(url).getCache("events");
ServletContext context = servletContextEvent.getServletContext();
context.setAttribute(CACHE, cache);
}As shown above, Ehcache uses its listener architecture to provide the
peer-to-peer cluster. The necessary listener classes are pre-defined and need
to be configured in ehcache1.xml or ehcache2.xml.
The following example shows how to configure the listeners for RMI-based
peer-to-peer networking:
<?xml version="1.0" encoding="UTF-8"?>
<ehcache xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="ehcache.xsd"
updateCheck="true" monitoring="autodetect"
dynamicConfig="true">
<cacheManagerPeerProviderFactory
class="net.sf.ehcache.distribution.RMICacheManagerPeerProviderFactory"
properties="peerDiscovery=automatic,
multicastGroupAddress=230.0.0.1,
multicastGroupPort=4446,
timeToLive=32"/>
<cacheManagerPeerListenerFactory
class="net.sf.ehcache.distribution.RMICacheManagerPeerListenerFactory"
properties="hostName=localhost,
port=40001,
socketTimeoutMillis=2000"/>
<cache name="events"
maxEntriesLocalHeap="1000"
eternal="true"
overflowToDisk="false"
memoryStoreEvictionPolicy="LFU">
<cacheEventListenerFactory
class="net.sf.ehcache.distribution.RMICacheReplicatorFactory"
properties="replicateAsynchronously=true,
replicatePuts=true,
replicateUpdates=true,
replicateUpdatesViaCopy=true,
replicateRemovals=true"/>
</cache>
</ehcache>The file ehcache2.xml is exactly the same, except that the port for the
PeerListenerFactory is 40002 instead of 40001.
@Override
public void contextDestroyed(ServletContextEvent servletContextEvent) {
CacheManager.getInstance().shutdown();
}When testing the initialization above with the write implementation from
the simple Ehcache application, an exception
is thrown on calling Cache.putIfAbsent(Element):
net.sf.ehcache.CacheException:
You have configured the cache with a replication scheme
that cannot properly support CAS operation guarantees.
at net.sf.ehcache.Cache.checkCASOperationSupported(Cache.java:3684)
at net.sf.ehcache.Cache.putIfAbsent(Cache.java:3564)
at net.sf.ehcache.Cache.putIfAbsent(Cache.java:3555)
at de.consol.research.cache.part03.ehcache.RestInterface.appendEvent(RestInterface.java:60)
All of ConcurrentMap’s atomic compare-and-swap operations are
not supported in Ehcache’s peer-to-peer configuration!
The unsupported
compare-and-swap (CAS) operations
are:
Therefore, Ehcace’s peer-to-peer mode cannot be used when multiple peers
modify the same key. However, it can be used if either keys are only written
once, or if each key is only modified by a single instance (for example,
if the key is a HTTP session, and a session aware load balancer is used).
In that case, regular
Cache.get(Object)
and
Cache.put(Element)
operations can be used without distributed atomicity guarantees:
@POST
@Path("{user}")
@Consumes(MediaType.APPLICATION_JSON)
public void appendEvent(@PathParam("user") String user, String msg) {
// NOT THREAD SAFE !!!
Element oldElement = cache.get(user);
UserEventList oldList = oldElement == null ? UserEventList.emptyList() : (UserEventList) oldElement.getObjectValue();
UserEventList newList = UserEventList.append(oldList, msg);
Element newElement = new Element(user, newList);
cache.put(newElement);
}However, using the implementation above the distributed cache will
become inconsistent if two nodes modify the same key at the same time.
Reading values is the same as with the
simple Ehcache application:
@GET
@Path("{user}")
@Produces(MediaType.APPLICATION_JSON)
public List<String> searchByUser(@PathParam("user") String user) {
Element result = cache.get(user);
if ( result == null ) {
return new ArrayList<>();
}
return ((UserEventList) result.getObjectValue()).getMessages();
}<dependency>
<groupId>net.sf.ehcache</groupId>
<artifactId>ehcache</artifactId>
<version>2.7.0</version>
</dependency>Our example code is hosted on GitHub.
The example can be run with maven.
Instance 1:
mvn tomcat7:run-war -pl part03.ehcache -am verify \
-Dmaven.tomcat.port=8080 -Dehcache.config.filename=ehcache1.xmlInstance 2:
mvn tomcat7:run-war -pl part03.ehcache -am verify \
-Dmaven.tomcat.port=9090 -Dehcache.config.filename=ehcache2.xmlThe Web interfaces are then available via
http://localhost:8080 and
http://localhost:9090.
The following pages will show how peer-to-peer clustering can be
implemented with Hazelcast and Infinispan: