In our series of articles about configuring remote JMX access for the jmx4perl proxy mode, this article tackles how to enable JMX remoting for Weblogic Server 9 and 10. It is not specific to jmx4perl and explains several different setups and possible problems.
As worked out in a previous post JBoss is a bit of a mess when it to comes to export the JDK’s
PlatfomMBeanServer. As it turns out for Weblogic things are not much easier (but there are ‘workarounds’). Weblogic even doubles the number of available MBeanServers to four.
Weblogic comes with three own MBeanServers, which are exported via RMI/IIOP as JSR-160 connectors. They can be looked up via a certain JNDI name as shown in the table below. Additionally, there is the ubiquitous PlatformMBeanServer which can be exported the usual way as described later in this post.
|MBean Server||JNDI Name|
|Domain Runtime MBean Server|| |
|Runtime MBean Server|| |
|Edit MBean Server|| |
The Runtime MBean Server specifies an individual application server, whereas the Domain Runtime MBean Server exposes the MBeans for all servers in a cluster. The Edit MBean Server is used for accessing and modifying the domain configuration. The Weblogic documentation contains further details.
When using jmx4perl’s proxy mode you have to choose the MBeanServer in advance (in contrast to the agent mode, where MBeanServers are merged to one virtual MBeanServer).
There are two ways how the MBeanServers mentioned above can be exported for remote access:
Via RMI/IIOP exported by Weblogic.
This way, the three Weblogic MBeanServers (those with an JNDI name) can
be exported (but not the PlatformMBeanServer directly). Advantage of
this export is that it can be enabled with the admin console and
that it includes the complete Weblogic security stack.
Via RMI/JRMP exported by the JVM.
This allows for the PlatformMBeanServer to be exported (but not the
other, Weblogic specific MBeanServers). It gets enabled as usual by
setting certain java defines as startup options and is secured the JDK
way. A forthcoming blog will clarify how to setup security for JDK
exported JSR-160 connectors.
Both methods are explained in detail in the following sections.
First of all, some configuration items must be set to enable IIOP exported MBeans. In the admin console, check that following properties are set:
First, IIOP must be enabled: Domain (‘wl_server’) / Environment / Servers -> Server (‘examplesServer’) -> Protocols -> IIOP -> Enable IIOP
Allow for anonymous read access, if you want to monitor without sending credentials: Domain (‘wl_server’) -> Security -> General -> Anonymous Admin Lookup Enabled
If you want to secure IIOP access (or want to have write access), set the name and password of the default user: Domain (‘wl_server’) / Environment / Servers -> Server (‘examplesServer’) -> Protocols -> IIOP -> (Advanced) -> “Default IIOP Username” and “Default IIOP Password”
You need to restart the server if you change one of the options above.
The JMX service URL for accessing Weblogic JMX connectors via IIOP looks like
service:jmx:rmi:///jndi/iiop://<server address>:<port>/<jndi name>
where the JNDI name is one of the MBeanServer’s JNDI name as described above. For example:
Now you are ready to access you Weblogic MBeans via jmx4perl:
jmx4perl http://proxy:8888/j4p \ --target \ service:jmx:iiop:///jndi/iiop://target:7001/weblogic.management.mbeanservers.runtime \ --target-user weblogic \ --target-password weblogic \ list
Here proxy:8888 specifies the server, where
j4p.war is running as a proxy servlet and target is the remote JMX enabled Weblogic server. Please note, that the usual
info command of jmx4perl won’t work here, since we don’t have access to any Java 5 MXBeans (like the MemoryMBean). But see below for how to make this work, too.
There are some issues, though.
There is one important point you should take care of if you connect to the target platform via IIOP. As described in this bug report, there are issues when Java 5 and Java 6 virtual machines are communicating via IIOP. A typical error happening in such a scenario looks like:
026030 ERROR STDERR| Nov 28, 2009 1:21:51 PM com.sun.corba.se.impl.io.InputStreamHook throwOptionalDataIncompatibleException WARNING: "IOP00800008: (MARSHAL) Not enough space left in current chunk" org.omg.CORBA.MARSHAL: vmcid: OMG minor code: 8 completed: No at com.sun.corba.se.impl.logging. OMGSystemException.rmiiiopOptionalDataIncompatible2 (OMGSystemException.java:2709) ..... at org.omg.stub.javax.management. remote.rmi._RMIConnection_Stub.getMBeanInfo(Unknown Source) at javax.management.remote.rmi.RMIConnector $RemoteMBeanServerConnection.getMBeanInfo(RMIConnector.java:1052)
The only known workaround for this problem is to run the proxy and the target server with the same JDK version (either both Java 5 or both Java 6). Since you have normally the freedom to choose the JDK for at least the proxy, this shouldn’t be a big problem.
As with every RMI based connection, you need to be sure that type definitions are available on both sides of the connection. As Weblogic export Weblogic specific data types via JMX you will encounter
ClassCastExceptions when you access these remotely from a generic client (without Weblogic specific classes in the classpath):
Caused by: java.lang.ClassNotFoundException: weblogic.wsee.ws.dispatch.server.OneWayHandler (no security manager: RMI class loader disabled) [exec] at sun.rmi.server.LoaderHandler.loadClass ...
I.e. you won’t be able to access the MBeanServer via RMI/IIOP with
jconsole due to missing local classes.
As an alternative the usual way for exporting MBeans via RMI/JRMP can be used. This includes to set some java defines as startup options (probably within
JAVA_OPTIONS="$JAVA_OPTIONS \ -Dcom.sun.management.jmxremote \ -Dcom.sun.management.jmxremote.port=9999 \ -Dcom.sun.management.jmxremote.ssl=false \ -Dcom.sun.management.jmxremote.authenticate=false"
In this example we switched of security complete for sake of demonstration. You can now connect with jmx4perl or with your favorite JMX console as usual:
jmx4perl http://proxy:8888/j4p \ --target service:jmx:rmi:///jndi/rmi://target:9999/jmxrmi \ read java.lang:type=Memory HeapMemoryUsage jconsole service:jmx:rmi:///jndi/rmi://target:9999/jmxrmi
Unfortunately, by default, only the
PlatformMBeanServer gets exported without any Weblogic specific MBean. However, there is a way to get to the Weblogic MBeans as described in the next section.
For monitoring purposes having Weblogic runtime MBeans and JVM MXBeans in different MBeanServers which are exportable in different ways is quite annoying. However, for Weblogic there is a solution by configuring WLS to use the
PlatformMBeanServer as it’s MBeanServer. With this configuration, your are able to access Weblogic MBeans from RMI/JRMP Service URLs (as it is used with ‘normal’ JMX clients like jconsole) and Java MXBeans from the RMI/IIOP connector used by Weblogic.
For this to work, you need to set Domain (‘wl_server’) -> Configuration -> General -> (Advanced) -> Platform MBeanServer enabled in the admin console for both WLS 9 and 10. Afterwards a server restart is required.
For WLS 10 an additional configuration parameter has to be set in order to let WLS use the
PlatformMBeanServer for its runtime MBeans. The attribute
PlatformMBeanServerUsed needs to be set to true on the
JMXMBean (it is
false by default). Unfortunately I didn’t find a way to set this attribute via the admin console, but only via the Weblogic scripting environment WLST. Assuming that your current working directory is you WLS 10 root directory, you should use
wlst.sh to fire up the WLST in interactive mode and when the server is not running:
$ common/bin/wlst.sh ..... wls:/offline> readDomain('samples/domains/wl_server') wls:/offline/wl_server>cd('JMX') wls:/offline/wl_server/JMX>ls() drw- NO_NAME_0 wls:/offline/wl_server/JMX>cd('NO_NAME_0') wls:/offline/wl_server/JMX/NO_NAME_0>ls() -rw- CompatibilityMBeanServerEnabled true -rw- DomainMBeanServerEnabled true -rw- EditMBeanServerEnabled true -rw- InvocationTimeoutSeconds 0 -rw- ManagementEJBEnabled true -rw- Name null -rw- Notes null -rw- PlatformMBeanServerEnabled false -rw- PlatformMBeanServerUsed false -rw- RuntimeMBeanServerEnabled true wls:/offline/wl_server/JMX/NO_NAME_0>set('PlatformMBeanServerUsed','true') wls:/offline/wl_server/JMX/NO_NAME_0>set('PlatformMBeanServerEnabled','true') wls:/offline/wl_server/JMX/NO_NAME_0>updateDomain() wls:/offline/wl_server/JMX/NO_NAME_0>closeDomain() wls:/offline>exit()
The path to the domain (
samples/domains/wl_server), an the name of the JMX-Bean (
NO_NAME_0) might differ at your side, but I think, the idea is clear. As shown above you also can set
PlatformMBeanServerEnabled via WLST as well.
Fire up the server and you should be ready for accessing WLS MBeans and JDK MXMBeans from within the same
MBeanServer on Weblogic 10.
PlatformMBeanServerUsed via jmx4perl itself doesn’t work, because Weblogic needs some extra boilerplate (which I didn’t dive into) before configuration can be changed via JMX:
jmx4perl http://proxy:8888/j4p \ --target \ service:jmx:iiop:///jndi/iiop://target:7001/weblogic.management.mbeanservers.edit \ --target-user weblogic \ --target-password weblogic \ write com.bea:Name=wl_server,Type=JMX PlatformMBeanServerUsed true ERROR: org.omg.CORBA.UNKNOWN: vmcid: 0x0 minor code: 0 completed: Maybe
and on the WLS console:
<Warning> <RMI> <BEA-080003> RuntimeException thrown by rmi server: javax.management.remote.rmi.RMIConnectionImpl.setAttribute weblogic.management.NoAccessRuntimeException: Operation can not be performed as caller has not started an edit session. at weblogic.management.mbeanservers.edit.internal.EditLockInterceptor.checkEditLock ...
For setting up Weblogic for remote JMX access quite some configuration is needed. There are traps, but at least for normal monitoring needs everything should work fine at the end. Nevertheless, if you have the chance to operate jmx4perl in agent mode (without the need for remote JSR-160 connectors), I still recommend to connect directly to the server via the
j4p.war running as an agent and not as a proxy ;-).