Chapter 3. Agents

The WAR agent is the most popular variant, and can be deployed in a servlet container just like any other Java EE web application.

Often, installation is simply a matter of copying the agent WAR to a deployment directory. On other platforms an administrative Web GUI or a command line tool need to be used for deployment. Providing detailed installation instructions for every servlet container is out of scope for this document.

The servlet itself can be configured in two ways:

The configuration options discoveryEnabled and discoveryAgentUrl can be provided via environment variables or system properties, too. See the below for details.

Jolokia has various detectors which can detect the brand and version of an application server it is running in. This version is revealed with the version command. With the configuration parameter detectorOptions extra options can be passed to the detectors. These options take the form of a JSON object, where the keys are productnames and the values other JSON objects containing the specific configuration. This configuration is feed to a successful detector which can do some extra initialization on agent startup. Currently the following extra options are supported:

The WAR agent comes in two flavors:

Java EE security is enabled by default by adding the required information within the web.xml.

All current client libraries are able to use BASIC HTTP authentication with user and password. The <login-config> should be set accordingly. The <security-constraint> specifies the URL pattern (which is in the default setup specify all resources provided by the Jolokia servlet) and a role name "jolokia" which is used to find the proper authentication credentials. This role must be referenced outside the agent WAR within the servlet container, e.g. for Tomcat the role definition can be found in $TOMCAT/config/tomcat-users.xml.

The Jolokia agent servlet can be integrated into one's own web-applications as well. Simply add a servlet with the servlet class org.jolokia.http.AgentServlet to your own web.xml. The following example maps the agent to the context /jolokia:

    <servlet>
      <servlet-name>jolokia-agent</servlet-name>
      <servlet-class>org.jolokia.http.AgentServlet</servlet-class>
      <load-on-startup>1</load-on-startup>
    </servlet>

    <servlet-mapping>
      <servlet-name>jolokia-agent</servlet-name>
      <url-pattern>/jolokia/*</url-pattern>
    </servlet-mapping>
    

Of course, any init parameter as described in Table 3.1, “Servlet init parameters” can be used here as well.

In order for this servlet definition to find the referenced Java class, the JAR jolokia-core.jar must be included. This jar can be found in Maven central . Maven users will can declare a dependency on this jar artifact:

    <project>
      <!-- ....  -->
      <dependencies>
        <dependency>
          <groupId>org.jolokia</groupId>
          <artifactId>jolokia-core</artifactId>
          <version>${jolokia.version}</version>
        </dependency>
      </dependencies>
      <!-- .... -->
    </project>

The org.jolokia.http.Agent can be subclassed, too in order to provide a custom restrictor or a custom log handler. See Section 4.2, “Jolokia Restrictors” for details.^[5]

Also, multiple Jolokia agents can be deployed in the same JVM without problem. However, since the agent deploys some Jolokia-specific MBeans on the single PlatformMBeansServer, for multi-agent deployments it is important to use the mbeanQualifier init parameter to distinguish multiple Jolokia MBeans by adding an extra propery to those MBeans' names. This also needs to be done if multiple webapps containing Jolokia agents are deployed on the same Java EE server.

This bundle depends mostly on a running OSGi HttpService which it uses for registering the agent servlet.

All package imports of this bundle are listed in Table 3.3, “Package Imports of jolokia-osgi.jar (SB: exported by system bundle)”. Note that the org.osgi.framework.* and javax.* packages are typically exported by the system bundle, so no extra installation effort is required here. Whether the org.osgi.service.* interfaces are available depends on your OSGi container. If they are not provided, they can be easily fetched and installed from e.g. maven central. Often the LogService interface is exported out of the box, but not the HttpService. You will notice any missing package dependency during the resolve phase while installing jolokia-osgi.jar.

This agent bundle consumes two services by default: As stated above, an org.osgi.service.http.HttpService which is used to register (deregister) the Jolokia agent as a servlet under the context /jolokia by default as soon as the HttpService becomes available (unavailable). Secondly, an org.osgi.service.log.LogService is used for logging, if available. If such a service is not registered, the Jolokia bundle uses the standard HttpServlet.log() method for its logging needs.

The Jolokia OSGi bundle can be configured via the OSGi Configuration Admin service using the PID org.jolokia.osgi (e.g. if using Apache Karaf, place properties in etc/org.jolokia.osgi.cfg), or alternatively via global properties which typically can be configured in a configuration file of the OSGi container. All properties start with the prefix org.jolokia and are listed in Table 3.4, “Jolokia Bundle Properties”. They are mostly the same as the init-param options for a Jolokia servlet when used in a Java EE WAR artifact.

This bundle also exports the service org.jolokia.osgi.servlet.JolokiaContext which can be used to obtain context information of the registered agent like the context path under which this servlet can be reached. Additionally, it exports org.osgi.service.http.HttpContext, which is used for authentication. Note that this service is only available when the agent servlet is active (i.e. when an HttpService is registered).

You have a couple of choices when running jolokia on Glassfish v3 and up, since Glassfish is a both a fully fledged Java EE container and an OSGi container. If you choose to run the Section 3.1, “Java EE Agent (WAR)” then it is completely straight forward just deploy the war in the normal way. If you choose to deploy the Section 3.2, “OSGi Agents” then you will need to configure the org.jolokia.httpServiceFilter option with a filter to select either the Admin HttpService (4848 by default) or the Default HttpService which is where WAR files are deployed to.

In Glassfish 3.1.2 the OSGi bundle configuration is done in glassfish/conf/osgi.properties in version's prior to this the configuration is by default in glassfish/osgi/felix/conf/config.properties or if you are using Equinox glassfish/osgi/equinox/configuration/config.ini

# Restrict the jolokia http service selection to the admin host
org.jolokia.httpServiceFilter=(VirtualServer=__asadmin)
# Or alternatively to the normal http service use : (VirtualServer=server)

Deploying the bundle can be either be done by coping the jolokia-osgi.jar into the domain glassfish/domains/<domain>/autodeploy/bundles directory or it can be added to all instances by copying the jar to glassfish/modules/autostart

By default the agent will be available on http://localhost:<port>/osgi/jolokia rather than http://localhost:<port>/jolokia as with WAR deployment.

The all-in-one bundle includes an implementation of org.osgi.service.http.HttpService, i.e. the Felix implementation. The HttpService will be registered as OSGi service during startup, so it is available for other bundles as well. The only package import requirement for this bundle is org.osgi.service.LogService, since the Felix Webservice requires this during startup. As mentioned above, normally the LogService interface gets exported by default in the standard containers, but if not, you need to install it e.g. from the OSGi compendium definitions.

This bundle can be configured the same way as the pure bundle as described in Section 3.2.1, “jolokia-osgi.jar”. Additionally, the embedded Felix HttpService can be configured as described in its documentation. e.g. setting the port to 9090 instead of the default port 8080, a property org.osgi.service.http.port=9090 needs to be set. This might be useful, if this bundle is used within containers which already occupy the default port (Glassfish, Eclipse Virgo) but don't expose an OSGi HttpService.

It is also possible to register the Jolokia agent servlet manually instead of relying of the OSGi bundle activator which comes with the agents. For this use case jolokia-osgi.jar should be used. This bundle exports the package org.jolokia.osgi.servlet which includes the servlet class JolokiaServlet. This class has three constructors: A default constructor without arguments, one with a single BundleContext argument and finally one with an additional Restrictor (see Section 4.2, “Jolokia Restrictors” for details how access restrictions can be applied). The constructor with a BundleContext as its argument has the advantage that it will use an OSGi LogService if available and adds various OSGi server detectors which adds server information like product name and version to the version command. Refer to Section 6.2.6, “Getting the agent version (version)” for details about the server infos provided.

Please note that for this use case the bundle org.jolokia.osgi should not be started but left in the state resolved. Otherwise, as soon as an OSGi HttpService registers, this bundle will try to add yet another agent servlet to this service, which is probably not what you want. Alternatively, the bundle property org.jolokia.listenForHttpService can be set to false in which case there will be never an automatic servlet registration to an HttpService.

As described in Section 4.2, “Jolokia Restrictors”, the Jolokia agent can use custom restrictors implementing the interface org.jolokia.restrictor.Restrictor. If the bundle property org.jolokia.useRestrictorService is set to true and no restrictor is configured by other means, the agent will use one or more OSGi service which register under the name org.jolokia.restrictor.Restrictor. If no such service is available, access to the agent is always denied. If one such restrictor service is available, the access decision is delegated to this service. When more than one restrictor service is available, access is ony granted if all of them individually grant access. A sample restrictor service as a maven project can be found in the Jolokia source at agent/osgi/restrictor-sample.

The JVM agent uses the JVM Agent interface for linking into any JVM. Under the hood it uses an HTTP-Server, which is available on every Oracle/Sun JVM from version 1.6 upwards.

java -javaagent:agent.jar=port=7777,host=localhost

A Jolokia agent can be attached to any running Java process as long as the user has sufficient access privileges for accessing the process. This agent uses the Java attach API for dynamically attaching and detaching to and from the process. It works similar to JConsole connecting to a local process. The Jolokia advantage is, that after the start of the agent, it can be reached over the network.

The JAR containing the JVM agent also contains a client application which can be reached via the -jar option. Call it with --help to get a short usage information:

$ java -jar jolokia-jvm-1.3.4-agent.jar --help

Jolokia Agent Launcher
======================

Usage: java -jar jolokia-jvm-1.3.4-agent.jar [options] <command> <pid/regexp>

where <command> is one of
    start     -- Start a Jolokia agent for the process specified
    stop      -- Stop a Jolokia agent for the process specified
    status    -- Show status of an (potentially) attached agent
    toggle    -- Toggle between start/stop (default when no command is given)
    list      -- List all attachable Java processes (default when no argument is given at all)
    encrypt   -- Encrypt a password which is given as argument or read from standard input

[options] are used for providing runtime information for attaching the agent:

    --host <host>                   Hostname or IP address to which to bind on
                                    (default: InetAddress.getLocalHost())
    --port <port>                   Port to listen on (default: 8778)
    --agentContext <context>        HTTP Context under which the agent is reachable (default: /jolokia)
    --agentId <agent-id>            VM unique identifier used by this agent (default: autogenerated)
    --agentDescription <desc>       Agent description
    --authMode <mode>               Authentication mode: 'basic' (default), 'jaas' or 'delegate'
    --authClass <class>             Classname of an custom Authenticator which must be loadable from
                                    the classpath
    --authUrl <url>                 URL used for a dispatcher authentication (authMode == delegate)
    --authPrincipalSpec <spec>      Extractor specification for getting the principal
                                    (authMode == delegate)
    --authIgnoreCerts               Whether to ignore CERTS when doing a dispatching authentication
                                    (authMode == delegate)
    --user <user>                   User used for Basic-Authentication
    --password <password>           Password used for Basic-Authentication
    --quiet                         No output. "status" will exit with code 0 if the agent is running,
                                    1 otherwise
    --verbose                       Verbose output
    --executor <executor>           Executor policy for HTTP Threads to use (default: single)
                                     "fixed"  -- Thread pool with a fixed number of threads (default: 5)
                                     "cached" -- Cached Thread Pool, creates threads on demand
                                     "single" -- Single Thread
    --threadNr <nr threads>         Number of fixed threads if "fixed" is used as executor
    --backlog <backlog>             How many request to keep in the backlog (default: 10)
    --protocol <http|https>         Protocol which must be either "http" or "https" (default: http)
    --keystore <keystore>           Path to keystore (https only)
    --keystorePassword <pwd>        Password to the keystore (https only)
    --useSslClientAuthentication    Use client certificate authentication (https only)
    --secureSocketProtocol <name>   Secure protocol (https only, default: TLS)
    --keyStoreType <name>           Keystore type (https only, default: JKS)
    --keyManagerAlgorithm <name>    Key manager algorithm (https only, default: SunX509)
    --trustManagerAlgorithm <name>  Trust manager algorithm (https only, default: SunX509)
    --caCert <path>                 Path to a PEM encoded CA cert file (https & sslClientAuth only)
    --serverCert <path>             Path to a PEM encoded server cert file (https only)
    --serverKey <path>              Path to a PEM encoded server key file (https only)
    --serverKeyAlgorithm <algo>     Algorithm to use for decrypting the server key (https only, default: RSA)
    --clientPrincipal <principal>   Allow only this principal in the client cert (https & sslClientAuth only)
                                    If supplied multiple times, any one of the clientPrincipals must match
    --extendedClientCheck <t|f>     Additional validation of client certs for the proper key usage
                                    (https & sslClientAuth only)
    --discoveryEnabled <t|f>        Enable/Disable discovery multicast responses (default: true)
    --discoveryAgentUrl <url>       The URL to use for answering discovery requests. Will be autodetected
                                     if not given.
    --sslProtocol <protocol>        SSL / TLS protocol to enable, can be provided multiple times
    --sslCipherSuite <suite>        SSL / TLS cipher suite to enable, can be provided multiple times
    --debug                         Switch on agent debugging
    --debugMaxEntries <nr>          Number of debug entries to keep in memory which can be fetched from the
                                    Jolokia MBean
    --maxDepth <depth>              Maximum number of levels for serialization of beans
    --maxCollectionSize <size>      Maximum number of element in collections to keep when serializing the
                                    response
    --maxObjects <nr>               Maximum number of objects to consider for serialization
    --restrictorClass <class>       Classname of an custom restrictor which must be loadable from the classpath
    --policyLocation <url>          Location of a Jolokia policy file
    --mbeanQualifier <qualifier>    Qualifier to use when registering Jolokia internal MBeans
    --canonicalNaming <t|f>         whether to use canonicalName for ObjectNames in 'list' or 'search'
                                    (default: true)
    --includeStackTrace <t|f>       whether to include StackTraces for error messages (default: true)
    --serializeException <t|f>      whether to add a serialized version of the exception in the Jolokia
                                    response (default: false)
    --config <configfile>           Path to a property file from where to read the configuration
    --help                          This help documentation
    --version                       Version of this agent (it's 1.3.4 btw :)

<pid/regexp> can be either a numeric process id or a regular expression. A regular expression is matched
against the processes' names (ignoring case) and must be specific enough to select exactly one process.

If no <command> is given but only a <pid> the state of the Agent will be toggled
between "start" and "stop"

If neither <command> nor <pid> is given, a list of Java processes along with their IDs
is printed

There are several possible reasons, why attaching to a process can fail:
   * The UID of this launcher must be the very *same* as the process to attach too. It not sufficient
     to be root.
   * The JVM must have HotSpot enabled and be a JVM 1.6 or larger.
   * It must be a Java process ;-)

For more documentation please visit www.jolokia.org

Every option described in Table 3.6, “JVM agent configuration options” is reflected by a command line option for the launcher. Additionally, the option --quiet can be used to keep the launcher silent and --verbose for adding some extra logging.

The launcher knows various operational modes, which needs to be provided as a non-option argument and possibly require an extra argument.

The launcher is especially suited for one-shot, local queries. For example, a simple shell script for printing out the memory usage of a local Java process, including (temporarily) attaching an Jolokia agent looks simply like in the following example. With a complete client library like Jmx4Perl even more one shot scripts are possible^[6].

#!/bin/sh

url=`java -jar agent.jar start $1 | tail -1`

memory_url="${url}read/java.lang:type=Memory/HeapMemoryUsage"
used=`wget -q -O - "${memory_url}/used" | sed 's/^.*"value":\([0-9]*\).*$/\1/'`
max=`wget -q -O - "${memory_url}/max" | sed 's/^.*"value":\([0-9]*\).*$/\1/'`
usage=$((${used}*100/${max}))
echo "Memory Usage: $usage %"

java -jar agent.jar --quiet stop $1