Thursday, February 13, 2014

Running the RHQ-agent on a Raspberry PI [updated]

I finally got a Raspberry Pi too. After the hurdles of initial installation, got it hooked up to my LAN and of course I had to install an RHQ agent on it.

And it turned out that this was dead simple, as the Pi already has Java 1.7 installed (in the Raspbian Wheezy distro that I am using). Thus it was only a matter of laying down the rhq-agent, and starting it the usual way.

Now there was one caveat: the agent did not find any file systems or network interfaces etc. This is due to the fact that there is no native library for Sigar on arm v6 cpus supplied with the agent.

I cloned Sigar from its git repository, changed into the 1.6 branch and built that library myself.

Now after dropping libsigar-arm-linux.so into the agent's lib/ directory, the native library is available and on agent restart all the native stuff could be found.

Screenshot of platform details in RHQ
(Platform details in RHQ)

If you don't want to compile that library yourself, you can take my version from https://sourceforge.net/projects/rhq/files/rhq/misc/.

I will try to get that library into the upcoming RHQ 4.10 release, but can't promise anything.

Update

If you run the agent from current master (or upcoming RHQ 4.10), you can configure a list of plugins to be enabled, so that the agent only uses these plugins (and thus uses less memory and starts faster).
This property can be found in the file conf/agent-configuration.xml:


<entry key="rhq.agent.plugins.enabled" value="Platforms,JMX,RHQAgent"/>

The entries are a comma separated list of plugin (short) names. To determine those, you can run
plugins info at the agent command prompt:


> plugins info
Details of the plugins that are currently installed:

rhq-agent-plugin-4.10.0-SNAPSHOT.jar
Plugin Name: RHQAgent
Display Name: RHQ Agent
Last Updated: 14. Februar 2014 11:03:35 MEZ
File Size: 51.558 bytes
MD5 Hashcode: f7eb7577af667ee4883437230e4b2d8c
[...]

Summary of installed plugins:
[RHQAgent, Platforms, JMX]

The short names are the ones encoded as "Plugin Name" and which are also shown on the summary line. There has actually been a property to disable unwanted plugins for a longer time, but just enabling the ones needed is probably easier.


The other thing you should do it to remove the -Xms setting in the rhq-agent.sh script- the default of a 64MB minimum heap is just too large here.

With those 3 plugins above and the removed Xms setting, my agent has a committed heap of ~14MB and a used heap of ~11MB. A dump is/was 4MB in size.

P.S.: “Raspberry Pi" is a trademark of the Raspberry Pi Foundation

Saturday, December 21, 2013

Money for nothing and memory for free: Java 7u40

I recently started playing with tools like MAT and also reading up about memory usage, performance (tuning) etc.

One of the interesting blog posts I came by was a comparison of JVM versions, that mentions that in 7u40, the default on how the JVM allocates the backing memory for ArrayList and HashMap has changed.

When you do

List<Foo> bla = new ArrayList<Foo>();

the VM will allocate memory for the base object and in vm versions prior to 7u40 also 10* 4bytes for the references to Foo objects, which (with alignment and so on) accounts to 80 bytes per empty ArrayList.

In 7u40, the array for the references is no longer eagerly allocated so that an empty ArrayList now only occupies the base 24 bytes, which account for 56bytes saving.

You may say so what, that are only 56 bytes, but remember that memory not allocated does not fill the heap, does not need to be garbage collected and also does not require memory bandwidth for the initial nulling out.

And so often an empty list/map does not come alone as in a case like this (output from MAT):

Screenshot 2013 11 15 17 51 25

With 225k empty ArrayLists, 56 bytes matter: 225k*56 bytes are 12 MB that you would save just by switching from a JVM pre 7u40 to 7u40 or later without a code change (of course not allocating those lists at all would save even more).

The situation for HashMap is similar: before 7u40 an empty one uses 136 bytes while in 7u40+ is uses 48, a saving of 88 bytes per empty HashMap or with the 235k empty HashMaps of the above example a saving of 20 MB.

Another (older) change is that in 7u06 the minimum size of String objects also has been reduced by 8 bytes, which matters a lot if you have millions of Strings in your VM.

Of course (as I've mentioned already) if you have access to the source code and can prevent the allocation of those empty objects altogether you would save even more memory.

Saturday, November 02, 2013

Back from OneDayTalk

[ I should have already written this a bit earlier, but I had some trouble with my left knee and had to go through some surgery (which went well). ]

As in the previous years I have been invited to give a talk at the OneDayConference in Munich. This year it was in a new location in a suburb of Munich called Germering. Getting there was easy for me, as there is a S-Bahn stop almost in front of the conference location.

The new location featured more and larger rooms and especially an area to sit down between talks or during lunch time. As in the last years the conference featured three parallel tracks.

As I said before I like that conference as everything is like a big family event with the organizers and also the presenters which featured many JBoss Colleagues; while I wrote that Ray would be there, Andrew Rubinger replaced him. The only talk that I really attended was the Wildfly one from Harald Pehl, which was full house. In the remaining part of the conference I talked to various attendees and colleagues from Jan Wildeboer to Gavin King and Heiko Braun. Heiko gave me an introduction about his (and Harald's) work to generate UIs from descriptors (which they use in the Wildfly console), which looks very interesting and where I think we could use some of that inside of RHQ to create "wizards" for several types of target resources.

In my talk, which was in the last slot, I had around 30 attendees (which was around 1/3 of the attendees still present). To my surprise I found out that the large majority did not yet know or use RHQ, so I had to switch from my original agenda and gave a brief introduction into RHQ first. Next I talked about the recent changes in RHQ and tried to gather feedback for future use case, but that was of course harder with attendees not knowing too much about RHQ. So much for "know your audience".

How do others try to find out their audience when the only thing they know is "This conference is all about JBoss projects" ?

You can find my slides in AsciiDoc format on GitHub that you can render via AsciiDoctor to html presentation.

Friday, November 01, 2013

Beware of the empty tag

I started playing with AngularJS lately and made some progress with RHQ as backend data source (via the REST api). Now I wanted to write a custom directive and use it like this:


<rhq-resource-picker/>
<div id="graph" style="padding: 10px"></div>

Now the very odd thing (for me) was that in the DOM this looked like this:


<rhq-resource-picker>
<div id="graph" style="padding: 10px"></div>
</rhq-resource-picker>

My custom directive wrapped around the <div> tag.

To fix this I had to turn my tag in a series of opening and closing tags instead of the empty one:


<rhq-resource-picker></rhq-resource-picker>
<div id="graph" style="padding: 10px"></div>

And in fact it turns out that a few more tags like the popular <div> tag show the same property of not allowing to write an empty tag, but requiring a tag pair.

Thursday, October 10, 2013

OneDayTalk conference in Munich

I will as in the previous years be at this years JBoss OneDayTalk conference in Munich and talk about recent and future developments in RHQ.

OneDayTalk is a pretty nice little conference organized by the JBoss User Group Munich, that offers three tracks with 6 sessions each, where I usually have the problem that I can't divide myself into three to visit them all. And for 99 Euro you also get some good food and many opportunities to meet myself and other JBossians like Eric Schabell, Gavin King, Emanuel Muckenhuber, Heiko Braun or Ray Ploski. The conference web site has the full listing of speakers as well as the scheduled program.

Friday, September 13, 2013

More clever builds and tests ?

Hey,

when you have already tried to build RHQ and run all the tests you probably have seen that this may take a huge amount of time, which is fine for the first build, but later when you e.g. change a typo in the as4-plugin, you don’t want GWT being compiled again.
Of course as a human developer it is relatively easy to just go into the right folder and only run the build there.

Now on build automation like Jenkins this is less easy, which is why I am writing
this post.

What I want to have is similar to

  • if a class in GWT has changed, only re-build GWT

  • if a class in a plugin has changed, only rebuild and test that plugin
    (and perhaps dependent plugins like hibernate on top of jmx)

  • if a class in enterprise changes, only build and test enterprise

  • if a class in server/jar changes, only rebuild that and run server-itests

  • if a class in core changes, rebuild everything (there may be more fine grained rules as e.g. a change in core/plugin-container does not require compiling GWT again)

This is probably a bit abbreviated, but you get the idea.

What I can imagine is that we compile the whole project (actually we may even do incremental compiles to get build times further down and may also only go into a specific module (and deps) and just build those).
And then instead of running mvn install we run mvn install -DskipTests
and afterwards analyze what has changed, throw the above rules
at it and only run the tests in the respective module(s).

We could perhaps have a little DSL like (which would live in the root of the project tree and be in git)

    rules: {
rule1: {
if :modules/enterprise/,
then: {
compile: modules/enterprise,
skip: modules/enterprise/gui/coregui,
test: ["modules/enterprise"]
}
},
rule2: {
if: modules/plugins/as7-plugin,
then: {
compile:[ modules/plugins/as7-plugin,
modules/integration-tests/as7-plugin],
test: [ modules/plugins/as7-plugin,
modules/integration-tests/as7-plugin]
}
}

And have them evaluated in a clever way by some maven build helper
that parses that file and also the list of changes since the last build to
figure out what needs testing.

We can still run a full build with everything to make sure that we don’t loose coverage
by those abbreviations

There may be build systems like gradle that have this capability built in; I think for maven
this requires some additional tooling

QUESTIONS:

  • Are there any "canned" solutions available?

  • Has anyone already done something like "partial tests" (and how)?

  • Anyone knows of maven plugins that can help here?

  • Anyone interested in getting that going? This is for sure not only interesting for RHQ but for a lot of projects

Having such an infrastructure will also help us in the future to better integrate
external patches, as faster builds / tests can allow to automatically test those and
"pre-approve" them.




Wednesday, September 11, 2013

RHQ 4.9 released

It is a pleasure for me to announce on behalf of the RHQ development team the immediate availability of RHQ 4.9

Features

Some of the new features since RHQ 4.8 are:

Be sure to read the release notes and the installation documents.

Note

For security reasons we have made changes to the installation in the sense that there is no more default password for the rhqadmin super user. Also the default bind address of 0.0.0.0 has been removed. You need to set them before starting the installation.

If you are upgrading from RHQ 4.8 you need to run a script to remove the native components from Cassandrabefore the upgrade. Otherwise RHQ 4.9 will fail to start.

Thanks

Special thanks goes to

  • Elias Ross

  • Jérémie Lagarde

  • Michael Burman

for their code contributions for this release and to Stian Lund for his repeated testing of the new graphs implementation.

Downloads

As usual you can download the release from the RHQ-project downloads on SourceForge