jmx4r 0.0.3, documentation and multiple connections

jmx4r 0.0.3 has just been released.
jmx4r is a JRuby library which makes it super easy to write simple Ruby scripts to manage Java applications using JMX.

Two new features in this release:

1. some much-needed documentation
2. as requested by Brian McCallister, I've modified the code so that it is now possible to write a script to manage many Java applications at the same time.

For example, the script to trigger a garbage collection on a cluster of Java applications at the same time (quite a bad idea but a simple one):

hosts = ["node1", "node2", "node3", "node4"]
hosts.each do |h|
    memory = JMX::MBean.find_by_name "java.lang:type=Memory", :host => h, :port => port
    memory.gc
end

Quite simple, isn't it?

JMX::MBean#find_by_name and JMX::MBean#find_all_by_name now accept an optional hash where you can specify specific information related to the connection to use (:host, :port, :username, :password or :connection) as described in the documentation. Otherwise, a global connection will be used.

A complete example shows how to display memory usage on 3 Java applications before and after a garbage collection is triggered.

$ jruby -Ilib examples/memory_on_many_nodes.rb    

Before GC:
+----------------+----------------+----------------+
| Node           |      Heap Used |  Non Heap Used |
+----------------+----------------+----------------+
| localhost:3000 |        1264328 |       14414808 |
| localhost:3001 |        1345632 |       14465192 |
| localhost:3002 |        1405072 |       14501936 |
+----------------+----------------+----------------+
After GC:
+----------------+----------------+----------------+
| Node           |      Heap Used |  Non Heap Used |
+----------------+----------------+----------------+
| localhost:3000 |        1142304 |       14421184 |
| localhost:3001 |        1205040 |       14476312 |
| localhost:3002 |        1218928 |       14506672 |
+----------------+----------------+----------------+

As usual, the simplest way to install it is by using RubyGems: jruby -S gem install **jmx4r**.

Gem for jmx4r + authentication

jmx4r is a library for JRuby to make it super easy to write Ruby scripts to manage remote Java applications through JMX.

Thanks to RubyForge, installing jmx4r is now as simple as typing

jruby -S gem install jmx4r

and its use is straightforward

#!/usr/bin/env jruby
require 'rubygems'
require 'jmx4r'    

# optional since by default, jmx4r tries to connect to 
# a JMX Server on localhost which listens to port 3000
JMX::MBean.establish_connection :host => "localhost", :port => 3000    

memory = JMX::MBean.find_by_name "java.lang:type=Memory"
# trigger a Garbage Collection
memory.gc

Since my previous post on jmx4r, I've added unit tests and some examples to highlight its features. It still needs to be properly documented though...

However, one new feature is worth mentioning: jmx4r now supports connection authentication

JMX::MBean.establish_connection :host => "localhost",
:username => "jeff", :password => "secret"

If you're using it, I'm very interested to now what you think about it.
And if you encounter any problem, do not hesitate to submit a bug.

jmx4r, a JMX Libary for JRuby

Just in time for the release of JRuby 1.0 and following my experiments with writing JRuby scripts to manage remote applications using JMX (part I & II), I created jmx4r, a simple library which makes it super easy to write such scripts.

For example, to trigger a Garbage Collection on a remote Java application , the whole script is:

require 'java'
require 'jmx4r'    

memory = JMX::MBean.find_by_name "java.lang:type=Memory"
memory.verbose = true
memory.gc

Simple enough, isn't it?

JMX Connection

By default, jmx4r expects to connect to the remote MBean Server on localhost using the standard JMX Service URL. Otherwise, it is possible to set the host and port of the remote MBean Server:

JMX::MBean.establish_connection :host => "localhost", :port => 3000

Attributes & Operations naming convention

JMX attributes are available on the JRuby objects but their names are changed to be snake_cased instead of CamelCased.
For example, the LoadedClassCount attribute of the java.lang:type=ClassLoading MBean is available as the loaded_class_count attribute on the corresponding JRuby object.

Ditto for the operations:

logging = JMX::MBean.find_by_name "java.util.logging:type=Logging"
logging.logger_names.each do |logger_name|
    logging.set_logger_level logger_name, "INFO"
end

The set_logger_level method corresponds to the setLoggerLevel MBean operations.

Features List

For now, the features of jmx4r are:

• read MBean attributes:

memory = JMX::MBean.find_by_name "java.lang:type=Memory"
puts "verbose : #{memory.verbose}"

• write MBean attributes (provided they are writable):

memory = JMX::MBean.find_by_name "java.lang:type=Memory"
memory.verbose != memory.verbose

• invoke MBean operations (provided the parameters types are simple):

memory = JMX::MBean.find_by_name "java.lang:type=Memory"
memory.gc

logging = JMX::MBean.find_by_name "java.util.logging:type=Logging"
logging.set_logger_level "global", "INFO"

• query for MBeans:

memory_pools = JMX::MBean.find_all_by_name "java.lang:type=MemoryPool,*"
memory_pools.each { |mem_pool| puts mem_pool.name }

• get the ObjectName correponding to a MBean:

memory_pools = JMX::MBean.find_all_by_name "java.lang:type=MemoryPool,*"
memory_pools.each { |mem_pool| puts mem_pool.object_name }

Next steps are:

• provide comprehensive tests
• document the library
• make a gem to simplify its deployment

eclipse-jmx 0 .2.1 is released

eclipse-jmx 0.2.1 has been released.

Compared to previous release, the main new feature is the support for MBean notifications.

Under the hood, the code which displays MBean attribute values has been completely rewritten by Ben Walstrum using extension points. Thanks to his contribution, any MBean attribute display can now be customized (e.g. using charts, hyperlinks, ...)

See what's new & noteworthy and the release notes.
You can download it or update it from Eclipse using the update site.

As usual, one word of caution: it is still in alpha stage and nowhere near as functional or usable as jconsole.

Do not hesitate to send feedback on the dev mailing list or to create issues in its tracker if you encounter a bug.

Enjoy!

JMX Scripts using JRuby -- Part II

update: a more Ruby-esque version using Rails idioms by Aaron Batalion

update: updated Ruby script to use instance_eval and define_method instead of eval based on a poignant explanation of eval-less metaprogramming.

In Part I, I created a JRuby script to manage a Java application using JMX.

In this entry, I'll explain how to remove the dependency on the MBean proxies by taking advantage of Ruby to dynamically add the MBean attributes to a Ruby object representing the MBean.

MBean proxies

Using MBean proxies (using either ManagementFactory.newPlatformMXBeanProxy() or MBeanServerInvocationHandler.newProxyInstance()) is always the simplest way to interact with MBeans. However it also means that the MBean interfaces must be available to the JMX client. This can be sometimes problematic.
For example, to use a proxy on your MBean (as opposed to the JVM MBeans that I'm using in my examples for convenience), you must add the jar containing your MBeans in JRuby classpath.
If it is not a problem, good news, go for it.
However if it makes the setup too complex, let's try to break this dependency.

MBean attributes

To make it simple, I'll focus on the MBean attributes. We will create a Ruby object which makes it simple and elegant to read the value of a MBean attribute without creating a proxy of the MBean (by simple & elegant, I mean something else than calling directly MBeanServerConnection.getAttribute(ObjectName, String)).

We will again use jconsole as our managed java application (see Part I to start jconsole with all System properties required to manage it remotely).

Going directly to the conclusion, the code to display a list of all loggers defined by java.util.logging is

# connect to jconsole MBeanServer
url = JMXServiceURL.new "service:jmx:rmi:///jndi/rmi://localhost:3000/jmxrmi"
connector = JMXConnectorFactory::connect url, HashMap.new
mbsc = connector.mbean_server_connection

logging = MBean.new mbsc, "java.util.logging:type=Logging"
# list all Loggers
logging.LoggerNames.each do | logger_name |
    puts logger_name
end

Running this code will display all loggers:

sun.rmi.client.call
java.awt.ContainerOrderFocusTraversalPolicy
javax.management.remote.rmi
javax.swing.SortingFocusTraversalPolicy
sun.rmi.server.call
sun.rmi.transport.tcp
...

Where is the magic? It is in

logging = MBean.new mbsc, "java.util.logging:type=Logging"

JRuby's MBean class

MBean is a Ruby class which creates objects with specific methods to access all the attributes associated to the ObjectName passed in parameter using the MBeanServerConnection to connect to a remote MBeanServer.

What does its code look like?

class MBean
    include_class 'javax.management.ObjectName'

    def initialize(mbsc, object_name)
        @mbsc = mbsc
        @object_name = ObjectName.new object_name
        info = mbsc.getMBeanInfo @object_name
        info.attributes.each do | mbean_attr |
            self.class.instance_eval do 
                define_method mbean_attr.name do
                    @mbsc.getAttribute @object_name, "#{mbean_attr.name}"
                end
            end
        end
    end
end

This class only defines a constructor initialize which accepts a MBeanServerConnection and a String representing an ObjectName which is used to create a @object_name field.

It retrieves the MBeanInfo associated to the @object_name thanks to @mbsc. It then iterates on the MBeanAttributeInfo using info.attributes.

That's in this iterator that the magic happens:

self.class.instance_eval do 
    define_method mbean_attr.name do
        @mbsc.getAttribute @object_name, "#{mbean_attr.name}"
    end
end

It calls instance_eval to add a method to the instance class.
This method is created using define_method with mbean_attr.name as the method name. It returns the value of the mbean attribute by calling @mbsc.getAttribute for the given mbean_attr.name of the @object_name of the MBean.

What does this mean? Executing logging = MBean.new mbsc, "java.util.logging:type=Logging" will create the logging object and add a method to this object to access the LoggerNames attribute:

def LoggerNames
    @mbsc.getAttribute @object_name, "LoggerNames"
end

Conclusion

Complete code is in jmx-attr.rb.

With few lines of code, it is possible to access MBean attributes in a simple way.

However, to have the same level of functionalities than using MBean proxies, we still need to be able to write attributes and invoke operations. Let's keep that for other entries.

Google Gears & Alchemy

Google just released Gears which is :

"an open source browser extension that enables web applications to provide offline functionality:

• Store and serve application resources locally
• Store data locally in a fully-searchable relational database
• Run asynchronous Javascript to improve application responsiveness".

At first glance, it seems to be the implementation of what Adam Bosworth envisioned when he was talking about BEA's Alchemy and browser connectivity (search for "When connectivity isn't certain" at the middle of the page).

Gears is already enabled for Google Reader. I wonder how long it will take to have it supported in Gmail...

Another potential use case for Gears is to develop browser-based desktop applications. Their main data store will be provided by Gears database which could be synchronized from time to time to a web store. Office applications (document, spreadsheets, calendar) seem like good candidate for that approach: they'll feel faster and more responsive that way. Synchronizing them will only be done when (auto) saving the document/spreadsheet/event.

JMX Scripts with Eclipse Monkey

Continuing the series about "writing JMX scripts in a dynamic language", after Ruby (part I & II), let's do that in JavaScript.

Aside of the use of a different scripting language, this example differs completely from the Ruby one by its context of execution: it will be integrated into Eclipse and called directly from its user interface (using Eclipse Monkey as the glue).

The example will:

1. ask graphically the user for a logging level
2. update all the JVM's loggers with this level
3. display all the loggers of the JVM

in 50 lines of code.

This example is simple but it implies several interesting steps:

• connect to a JMX Server
• retrieve a MBean
• retrieve value of MBean attributes
• invoke operations on the MBean

There are many use cases where you have to perform theses steps in repetition. It's tedious to do that in a JMX console (e.g. jconsole or eclipse-jmx) and most of the time, it is not worth writing a Java application.

These use cases beg to be scripted.

We will again use jconsole as our managed java application (see this previous post to start jconsole with all System properties required to manage it remotely).

To run the example:

1. install Eclipse Monkey
2. Make sure that Eclipse is running on Java 1.5 or above
3. copy the script just below and save it as logging_change_level.js in the scripts directory of an Eclipse project
4. click on Scripts > Logging > Change level in Eclipse menu
5. follow the instruction to install the required DOM
6. only the JMX Monkey feature is required
7. restart Eclipse
8. click on Scripts > Logging > Change level
9. in the dialog, input the logging level
10. all the loggers have been updated with the given level

logging__change__level.js

/*
 * Menu: Logging > Change level
 * DOM: http://eclipse-jmx.googlecode.com/svn/trunk/net.jmesnil.jmx.update/net.jmesnil.jmx.monkey.doms
 */
    
function main() {
   mbsc = jmx.connect("localhost", 3000)
   logging = mbsc.getMBean("java.util.logging:type=Logging")
   
   level = ask_level()
   set_all(mbsc, logging, level)
   text = "All loggers are set to " + level + "\n\n"
   text += logger_levels(mbsc, logging)
   show(text)
}
    
function logger_levels(mbsc, logging) {
   var out = ""
   for each(loggerName in logging.LoggerNames) {
       lvl = mbsc.invoke(logging, "getLoggerLevel",
            [loggerName],
            ["java.lang.String"]);
       out += loggerName + " is at " + lvl + "\n"
   }
   return out
}
    
function set_all(mbsc, logging, level) {
   for each(loggerName in logging.LoggerNames) {
       mbsc.invoke(logging, "setLoggerLevel",
            [loggerName, level],
            ["java.lang.String", "java.lang.String"])
   }
}
    
function ask_level() {
   dialog = new Packages.org.eclipse.jface.dialogs.InputDialog(
          window.getShell(), 
          "Logging Level",
          "Which level? (SEVERE, WARNING, INFO, CONFIG, FINE, FINER, FINEST)",
          "INFO", null)
   result = dialog.open()
   if (result == Packages.org.eclipse.jface.window.Window.OK) {
       return dialog.getValue()
   }
}
    
function show(text) {
   Packages.org.eclipse.jface.dialogs.MessageDialog.openInformation(
           window.getShell(),
           "Update Logging",
           text
       )
}

code walkthrough

The main() function of the script is:

function main() {
  mbsc = jmx.connect("localhost", 3000)
  logging = mbsc.getMBean("java.util.logging:type=Logging")
     
   level = ask_level()
   set_all(mbsc, logging, level)
   text = "All loggers are set to " + level + "\n\n"
   text += logger_levels(mbsc, logging)
   show(text)
} 

where

• mbsc = jmx.connect("localhost", 3000) creates a mbsc object connected to a local JMX server using the standard JMX Service URL.
• logging = mbsc.getMBean("java.util.logging:type=Logging") creates a logging object representing the LoggingMXBean used to manage the JVM logging.
• level = ask_level() opens an Eclipse input dialog where you can input the logging level and returns it as a String.
• set_all(mbsc, logging, level) sets all the loggers at the given level
• text += logger_levels(mbsc, logging) returns a String representation of all the loggers and their level
• show(text) opens an Eclipse message dialog

The interesting methods are logger_levels() and set_all(). In both methods, we retrieve an attribute of a MBean and invoke an operation.

Let's focus on set_all():

function set_all(mbsc, logging, level) {
   for each(loggerName in logging.LoggerNames) {
      mbsc.invoke(logging, "setLoggerLevel",
            [loggerName, level],
            ["java.lang.String", "java.lang.String"])
   }
}  

logging represents a MBean (it is not a "real" MBean, more on that later) and mbsc represents a MBeanServerConnection (but it is not a "real" MBeanServerConnection, more on that later).
logging.LoggerNames returns the value of the LoggerNames attribute of the MBean (note that the first letter must be in upper case) which is an array of strings.
For each element of this array, we invoke the setLoggerLevel operation using mbsc.invoke().
This method is very similar to the "real" MBeanServerConnection.invoke() method:

1. something representing a MBean (instead of an ObjectName)
2. the MBean operation name
3. the parameters of the operation
4. the types of the parameters

jmx: an Eclipse Monkey DOM

What do I mean when I write that logging is not the "real" LogginMXBean and that mbsc is not the real MBeanServerConnection?

This 2 types of objects are created by the jmx object in the main() method. This jmx object is in fact an Eclipse Monkey DOM that is contributed by the plug-in listed in the DOM directive at the top of the script:

DOM: http://eclipse-jmx.googlecode.com/svn/trunk/net.jmesnil.jmx.update/net.jmesnil.jmx.monkey.doms

This plug-in was included in the "JMX Monkey" feature which was installed the first time you ran this script.

The jmx DOM has a single method connect(host, port) which connects to a JMX Server using the standard JMX Service URL.
The object returned by this method is a ScriptableMBeanServerConnection. This class encapsulates the "real" MBeanServerConnection (still available using the getObject() method) but only exposes its invoke().

It also exposes a getMBean() method which returns a ScriptableMBean. In turn this class exposes the attributes of the MBean as JavaScript attributes.

To sum up, these are the operations you can perform using the jmx DOM:

• connect to a JMX server: mbsc = jmx.connect(host, port)
• get a mbean: mbean = mbsc.getMBean(objectName)
• get the value of a mbean attribute: val = mbean.AttributeName
• get the "real" mbean server connection and use it: objectNames = mbsc.getObject().queryNames(name, query)
• invoke an operation on a mbean: mbsc.invoke(mbean, operation, params, param_types)

Conclusion

This script example is simple but quite interesting thanks to its integration with Eclipse.

I believe there is an use for such scripts: repeatable management operations that needs to be tweaked from time to time. It's tedious to do that with a GUI and it's even more tedious to write Java applications to do so.

Last year, during EclipseCon'06, I blogged about an use case for scripting a RCP application using Eclipse Monkey. This is a concrete example: I'm using eclipse-jmx to manage Java applications that I develop. When I realize that I perform the same kind of management task, I write a monkey script which automates it.

Next time, you have to perform the same operation on many MBeans or many operations on the same MBean but you think it is not worth to write a Java application to automate it, ask yourselves if it can not be simply automated by a script such as the one in this post.

eclipse-jmx 0.2.0 is released

I've just released eclipse-jmx 0.2.0.

See what's new & noteworthy and the release notes.
You can download it or update it from Eclipse using the update site.

One word of caution: it is still in alpha stage and nowhere near as functional or usable as jconsole. Having said that, I hope it can still be useful.

Do not hesitate to send feedback on the dev mailing list or to create issues in its tracker if you encounter a bug.

Enjoy!

JMX Scripts using JRuby

[update: fixed Ruby script by using ManagementFactory::newPlatformMXBeanProxy instead of MBeanServerInvocationHandler::newProxyInstance based on Daniel comment]

I needed to write a script to automate the management of a Java application using JMX which I could put in a crontab.

I found some explanation on how to script JMX in jconsole with BeanShell or with Groovy. It's a good approach but:

• I want the scripts to be executable in a cron (and not from jconsole GUI)
• I'd prefer to write them in Ruby

It turns out that it is straightforward to write such scripts using JRuby and Java 5 or later.

To keep things simple, let say I want to write a script which will be called periodically to run the garbage collector on a long-running Java application.

First let use jconsole as our long-running application. We will start jconsole with all the System properties required to manage its JVM through JMX/RMI connector:

jconsole -J-Dcom.sun.management.jmxremote \
         -J-Dcom.sun.management.jmxremote.port=3000 \
         -J-Dcom.sun.management.jmxremote.ssl=false \
         -J-Dcom.sun.management.jmxremote.authenticate=false

gc.rb

The script to run the garbage collector is simple:

module JMX
    require 'java'
    
    include_class 'java.util.HashMap'
    include_package 'java.lang.management'
    include_package 'javax.management'
    include_package 'javax.management.remote'
    
    url = JMXServiceURL.new "service:jmx:rmi:///jndi/rmi://localhost:3000/jmxrmi"
    connector = JMXConnectorFactory::connect url, HashMap.new
    mbsc = connector.mbean_server_connection
    
    memory_mbean = ManagementFactory::newPlatformMXBeanProxy mbsc, "java.lang:type=Memory", MemoryMXBean::java_class
    
    memory_mbean.verbose = !memory_mbean.verbose
    puts "verbose = #{memory_mbean.verbose}"
    
    memory_mbean.gc
end

This gc.rb script is doing several things:

1. it require 'java' so our scripts will run only on JRuby.
2. it includes all the required classes and packages .
3. it connects to jconsole's MBeanServer using the "standard" JMX URL service:jmx:rmi:///jndi/rmi://localhost:3000/jmxrmi
4. it creates a proxy of the MemoryMXBean based on the MBeanServerConnection mbsc and the ObjectName java.lang:type=Memory
5. for "fun", it toggles the verbose boolean of the memory_mbean
6. and finally, it runs the garbage collector by calling memory_mbean.gc

When we run the script:

$ jruby gc.rb
verbose = true

we see that a gc has been performed by jconsole's JVM:

[Full GC 3754K->3172K(6076K), 0.0895140 secs]
[GC 3680K->3189K(6076K), 0.0008180 secs]
[GC 3701K->3281K(6076K), 0.0016410 secs]
[GC 3793K->3320K(6076K), 0.0014080 secs]
...

I just need to put the call to this script in my crontab and I'm done.

MBeans in JRuby classpath

Writing such scripts works fine for most cases but there is a caveat: it requires to have the MBean Java class in JRuby classpath to be able to create a proxy using either MBeanServerInvocationHandler::newProxyInstance or ManagementFactory::newPlatformMXBeanProxy (if the MBean is a MXBean as it is the case in this script).

If you have access to the jar containing the MBeans, you just need to put it $JRUBY_HOME/lib/ directory to be able to access it from Ruby scripts.

Still it should possible to map JMX API with Ruby metamodel to get rid of this requirement and develop "JMX-generic" scripts.

Next Step: Generic JMX Scripts

The next step would be to create Ruby objects representing JMX MBeans where their attributes & operations will be added dynamically to the corresponding Ruby instances by examining the MBeanInfo.
The previous example would look like:

memory_mbean = MBeanObject.new mbsc, "java.lang:type=Memory"
memory_mbean.verbose = !memory_mbean.verbose
memory_mbean.gc

The big change will be that we will no longer require to have the MBeans in JRuby classpath making it simpler to manage many Java applications (e.g. Tomcat, JBoss, etc.)
To be continued...

Add a filter to a TreeViewer

In eclipse-jmx, a plug-in to manage Java applications through JMX, I have a view which displays registered MBeans using a TreeViewer.
Since there can be many MBeans to display (e.g. more than 200 just for Tomcat), it is tedious to navigate in the Tree by expanding many nodes before finding the MBeans I want to manage. To make it more usable, I wanted to add a filter text to the view to show only the MBeans which are interesting to me.

UI-wise, I wanted something like the Filtered Extensions of the PDE Editor:

It shows only the extensions matching the filter text (and also displays the extensions categories).

Since this screenshot was from Eclipse 3.3M1 New & Noteworthy, I thought that this filter was not available for Eclipse 3.2 (or that it was specific to PDE).
It turns out that I was wrong, it works on Eclipse 3.2, and it takes only 2 lines to add such behavior to any TreeViewer using a PatternFilter and a FilteredTree (which are from the org.eclipse.ui.workbench plug-in).

Plain Old TreeViewer

The code to create the TreeViewer is straightforward:

public void createPartControl(Composite parent) {
    ...
    viewer = new TreeViewer(parent, SWT.MULTI | SWT.H_SCROLL | SWT.V_SCROLL);
    ...
}

which, in eclipse-jmx, gives:

PatternFilter & FilteredTree

To filter the tree's nodes, the code to change is minimal:

public void createPartControl(Composite parent) {
    ...
    PatternFilter patternFilter = new PatternFilter();
    final FilteredTree filter = new FilteredTree(parent, SWT.MULTI
            | SWT.H_SCROLL | SWT.V_SCROLL, patternFilter);
    viewer = filter.getViewer();
    ...
}

With these 2 lines, a filter widget is displayed on top of the tree and the tree's nodes are automatically filtered based on the filter text:

Screenshot has been taken on Eclipse 3.3M5 which uses native search widgets (if one is available) to display the filter control. It works also on Eclipse 3.2 but does not look so good.

By default, PatternFilter will display any nodes of the Tree whose label matches the filter text. By extending it, it is very simple to have more complex filter matching algorithm.
It is the case in eclipse-jmx: it displays any node whose descendant's leaves have an ObjectName which matches the filter text.

Kudos to the workbench team for this little UI nugget!