Java Moods

The Butler Dispute, Round 2

2011-05-05T23:46:00.007+02:00

Nearly 4 month have passed by since the renaming of Hudson project to Jenkins, which marks the climax of the dispute between the old Hudson developers and the guys from Oracle and Sonatype.

Jenkins has made a great job since then, delivering 15 releases in a weekly schedule. Build number 1.400 was hit in March, wich is not a particular significant release but shows how well things go on. The way from Hudson to Jenkins is as easy as it could be, and it seems like many users are going it.

Indeed there are a lot of reasons why to choose Jenkins over Hudson, just to name a few:

Support by the fabulous Hudson core development team – with Kohsuke Kawaguchi, the creator of Hudson, and other brave guys.
Strong community activity – measured in figures like commit counts and mailing list traffic, see this post for some numbers.
Most of the plugins moved over to Jenkins – 5 of the top 5 and 19 of the top 25 plugins continue primary development with Jenkins, see here for some statistics.
High quality and regular releases – the weekly schedule led to 15 high quality releases, each of them providing a couple of bug fixes and new features (see changelog). Moreover, a few weeks ago, Jenkins governance board proposed to start another release line for most stable baselines with a 3-months schedule.

Even the Hudson board seems to have observed that Jenkins outperforms Hudson in many ways, at least they are thinking about "how to make it more attractive for plug-in developers to support both Hudson and Jenkins" (see this message on Hudson-Dev list). The author's perception is that "Hudson also appears to be slowing down development wise" and "another place where Hudson appears to be slowing down, is when you compare changelogs". Some of the ideas deal with copying approaches that are working fine for the Jenkins project.

Hence, it seems Jenkins is the winner of the battle and has in fact benefited from the fork.... until today.

Because today, Oracle submitted a proposal to move Hudson to the Eclipse Foundation. This is, well, somewhat astonishing since that means Oracle will lose both control and the Hudson trademarks – which was the main background of the original dispute with the community.

As part of the proposal, other big players have announced support for the project, including IBM, VMware, Tasktop and Intuit. That means, moving the Hudson project to Eclipse will for sure result in higher attention and more resources (developers).

Does this change anything? Will Jenkins be the unlucky loser, after all? I don't think so. The heavens didn't really smile on Hudson since the fork (kind of bad karma) and I don't see why the move to Eclipse should change that. It's all about people, not code.

Moreover, Jenkins has been invited by Sonatype to reunite with Hudson. But... why should they do that? Jenkins is a vibrant project today, so what is the benefit? Also, there have been some deep disappointments on personal level that are not forgotten yet.

It's going to be interesting!

Jenkins: Pimp It Up!

2011-04-07T15:07:00.005+02:00

Some days ago, I started to review what plugins are available for Jenkins, my favorite CI server. I haven't done so for a long time, so I was somewhat surprised to see a full universe of plugins (380+) listed in the Wiki...

There is next to everything you can imagine. Among the plugins I would like to suggest for consideration are these:

Dependency Graph View Plugin – Shows the dependency graph of the Jenkins projects using graphviz. This greatly helps to keep track of dependencies between all your Jobs.

Disk Usage Plugin – This plugin calculates and records disk usage (space for builds and workspace) per project and per build, and can display trend graphs.

Global Build Stats Plugin – can be used to gather and display global build result statistics, monitoring over time, and show nice graphics.

And, of course, all the static analyzing plugins that scan result files of several static code analysis tools and visualize the results as trend graphs:

Have fun!

DocBook with Maven Issue

2011-03-29T09:07:00.002+02:00

We are using DocBook for writing technical documentation for all our projects and in-house frameworks. We are actually quite happy with this approach, especially because we are able to automatically publish the docs in a number of formats, including HTML and PDF. To do so, we use the docbkx-maven-plugin in the project's nightly build.

So, all has been in best order... until I decided to upgrade this docbkx-maven-plugin from version 2.0.8 to current version 2.0.11 (due to some issues we had). After doing so, the document conversion issues an error which breaks the build:

[ERROR] Failed to execute goal com.agilejava.docbkx:docbkx-maven-plugin:2.0.11:generate-pdf (pdf) on project builddoc-ma
ven-plugin: Failed to transform to PDF: org.apache.fop.fo.ValidationException: null:30:723: Error(30/723): fo:table-body
 is missing child elements.
[ERROR] Required Content Model: marker* (table-row+|table-cell+)

Well, this is somewhat unexpected because I didn't change anything but the plugin version, and I don't see any reason it could not work as before. In particular, we are still using the same docbook version in our POM. Here is the relevant snippet:

<plugin>
  <groupId>com.agilejava.docbkx</groupId>
  <artifactId>docbkx-maven-plugin</artifactId>
  <version>2.0.11</version>
  <dependencies>
    <!-- the DocBook XML DTD and catalog files (see http://www.oasis-open.org/docbook) -->
    <dependency>
      <groupId>org.docbook</groupId>
      <artifactId>docbook-xml</artifactId>
      <version>4.4</version>
      <scope>runtime</scope>
    </dependency>
  </dependencies>

  <executions>
    <execution>
      <id>pdf</id>
      <goals>
        <goal>generate-pdf</goal>
      </goals>
      <phase>post-site</phase>
      <configuration>
        ...
      </configuration>
    </execution>
    ...
  </executions>

  <configuration>
    <htmlStylesheet>css/html.css</htmlStylesheet>
    <htmlCustomization>${basedir}/src/doc/xsl/html_chunk_customization.xsl</htmlCustomization>
    <foCustomization>${basedir}/src/doc/xsl/fopdf_customization.xsl</foCustomization>
    ...
  </configuration>
</plugin>

It's important to understand that we are using the advanced customizing capabilities of DocBook, i.e. we customized the stylesheets used for rendering HTML and PDF. The created custom stylesheets contain an import to urn:docbkx:stylesheet, and in the Maven POM the htmlCustomization and foCustomization properties point to those custom stylesheets. This is how it's supposed to be, and this is how it worked all along.

I found out that the error message is correct when building with plugin version greater than 2.0.8, since the for-each element indeed does not return any element which results in an empty fo:table-body. In fact, none of the xsl:value-of in our customized stylesheet returned any value any more....

So here is why: since docbkx-maven-plugin version 2.0.9, the plugin is using namespaced stylesheets. That is, we must use a namespace in our custom stylesheet to be able to select any docbook element! See this or this post for related comments.

Thus, all I have to do is to add the docbook namespace declaration at the top and add the docbook namespace prefix to all references to element names in my customization layer. See highlighted lines in this XSL snippet:

<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
                xmlns:fo="http://www.w3.org/1999/XSL/Format"
                xmlns:db="http://docbook.org/ns/docbook"
                exclude-result-prefixes="date"
                version="1.0">

  <xsl:template name="book.titlepage.separator">
    <fo:block>
      <fo:table table-layout="fixed" width="163mm">
        ...
        <fo:table-body text-align="left">
          <xsl:for-each select="/db:book/db:bookinfo/db:revhistory/db:revision">
            ...
          </xsl:for-each>
        </fo:table-body>
      </fo:table>
    </fo:block>
  </xsl:template>
  ...
</xsl:stylesheet>

Well, that did the trick – after spending a couple of hours of investigation... I think that issue should be clearly noted with the docbkx-maven-plugin Maven plugin, because in the end it is an incompatability between versions 2.0.8 and 2.0.9. Alas, I did not find this information on the plugin's Changes Report page. At least, nothing that pointed me (not being a DocBook expert) into this direction... :-(

The Way From Hudson To Jenkins

2011-03-21T20:08:00.006+01:00

Some time has gone by since the Hudson/Jenkins fork... and there has been even more talk in the community. However, slowly the dust settles, everybody is getting back to business. And finally, we decided to switch from Hudson to Jenkins! This is about why and how.

Why move to Jenkins?

But wait: who has forked, anyways? Is it Jenkins that forked Hudson, or is it Hudson that did the fork of Jenkins? There is some evidence that the community just did a rename of the project (due to trademark conflicts), and after that Oracle forked Jenkins, using the Hudson name they claim holding the trademark on.

You may think this question is a purely theoretical one, but actually it's not. I'll have to legitimate the decision to move to Jenkins to my stakeholders, and using a fork would be a "smell". Project forks are usually not as good as the "original", are possibly done out of selfish reasons, are considered to harm the community etc. Hence, not moving to a fork but instead following the "real" project is a good reason for the move to Jenkins.

An even better one is "project vibrancy", that is the pace of development and level of support provided by the community. This is usually measured by indicators such as the number of commits, the mailing list traffic, the quantity, quality and regularity of releases etc. See this post for such an analysis on commit counts and mailing lists post counts. This is more than four weeks old now and covers not more than two weeks, but nevertheless the result is obvious: Jenkins moves much faster than Hudson does, and community is much more agile. This is confirmed by following the dev mailing lists of both: for Hudson, most of the relevant posts are by either Oracle or Sonatype engineers – seems the Hudson community has become pretty small... Moreover, as this post shows, most of the top plugins will continue primary development under Jenkins.

Last not least, I really respect Kohsuke Kawaguchi (the original creator of Hudson) and what he has done for us. I feel ashamed by how Oracle is dealing with him and the rest of the core team, that's why I have a strong tendency to follow the "good guys" with Jenkins.

As I blogged before, Maven integration is probably one of the most important features of any CI server (at least for me). I guess Sonatype is doing better with Maven integration – it's "The Maven Company", right? – and they are working with Oracle on Hudson. At least, they are putting huge efforts into rock-solid integration. However, after having seen a Sonatype Webinar about their plans with Hudson, I'm not that convinced any more. Current features looked a bit awkward and also does the GWT based UI they are using. So, from my point of view, this point is not yet decided.

Counting it all together, there are some really good reasons to move from Hudson to Jenkins, so we did.

How to upgrade

Now... how do you actually migrate from Hudson to Jenkins? Well, it couldn't be easier. There is a Wiki page about Upgrading from Hudson to Jenkins. To make it short, the involved steps are:

Backup your current installation – just for the good feeling.

Change Update Site: In your Hudson, go to Manage Hudson > Plugin Management > Advanced > Update Site and enter "http://updates.jenkins-ci.org/update-center.json" as URL for Jenkins update site.

Choose to upgrade automatically on Manage Hudson page, just as you did so many times to update Hudson. This will download the new JAR.

Restart Hudson, eh, Jenkins.... and there it is!

That's it. Took less than 5 min! Hudson indeed is a drop-in replacement, so you usually do not have to change anything (environment variables, system properties, start scripts, job configuration etc).

Well, there is only one thing: the name of the WAR file is still hudson.war! Is Oracle aware of this? ;-)

The Butler Dispute

2011-02-27T21:37:00.008+01:00

I thought it would be time to resurrect my blog, after not having posted for a couple of months. That was mainly because I have been really busy with some interesting stuff I should post about in the future, like Xtext upgrade...

Oracle vs. the Community

But today, I just bumped the ongoing dispute between Hudson and Jenkins guys. We are using Hudson since around 2008, coming from Cruise Control. We really liked the web interface, being able to setup everything by just using your browser. Of course, the features also have been impressive since then. Setting up a build farm is just fun with Hudson.

You probably know that there has been a fork of Hudson which is named Jenkins (others say Hudson has been renamed to Jenkins and then forked into Hudson). This all started with the Hudson team being unhappy with the infrastructure provided by java.net (which is driven by Oracle since Sun acquisition), due to its poor reliability. The community talked about moving parts of the project to other servers, and first candiate was issue tracking. Suddenly, the project is locked due to the migration of java.net projects to new Kenai infrastructure, which was announced by Oracle but somehow missed by the project owners. Frustrated by the migration, the community decides to move code to GitHub and mailing list to Google Groups. See "Who's driving this thing?" for the facts.

This is the point where Oracle steps in, claiming to have a trademark on the name. If the project decides to move, it must be using another name: "Because it is open source, we can't stop anybody from forking it. We do however own the trademark to the name so you cannot use the name outside of the core community. We acquired that as part of Sun." (BTW, that might not be true after all). Later he stated that "the final decision of what to do w.r.t. infrastructure belongs to Oracle".

Guess what: this really concerned the community. There have been some talks between key community members and Oracle representives, in an attempt to agree on a "proposal for a stable structure and arrangement" which later would be proposed to the community. But, with no success. That finally led to the decision of the community to move to GitHub and at the same time rename the project to another butler's name: "Jenkins".

Of course, Oracle's view on the subject is a bit different...

Welcome Jenkins!

Well, so now you have the choice: use Hudson, or use Jenkins. You know, competition is usually a good thing, so let the race begin. The majority of the community seems to have made the switch to Jenkins (given the blogs and mailing list traffic). This is because Oracle's behavior is not quite understood and does not cast a positive light on their comprehension of Oracle's role in the Hudson community.

However, Oracle is putting enormous resources (people and hardware) into the Hudson project. And what's even more important, Sonatype is helping to drive Hudson to the next level. Sonatype? Right, that's the company behind Maven and all the great Maven tools like Nexus and m2eclipse.

Maven Support – the Killer Feature?

One of the most important features (for me, but also for possibly the majority of other users) is Maven 3 support. Sure, Hudson/Jenkins already support Maven 3 since version 1.392 (end of 2010, see changelog). But hey, Sonatype entered the scene, and they will surely do better.

Sonatype, too, have put some full-time engineers into the Hudson project, making sure that "Hudson users can look forward to a long, bright future". See this or this post. Sonatype in the end of the day wants to earn money with Hudson (and Maven), so I expect to see outstanding features related to Maven 3 support, Eclipse integration and workflow extensions for Hudson. See here for some of their ideas.

Well, this really makes a thrilling game. I honestly appreciate what Kohsuke Kawaguchi and others have built up with Hudson from the ground up, and would like to see them win on the "evil company that pushed them out of the project". And by the way, Sonatype seems to be in good companion when talking about being evil – they removed the oldest commiter of m2eclipse from the project a year ago.

So, is this again the good vs. evil story? I don't know. In the end, both projects will have their users. And they will learn and benefit from each other. So let's wait and see.... Time will tell. It's going to be an interesting year, though!

Maven 3 and Plugin Mysteries

2010-10-15T12:15:00.005+02:00

You probably know that Maven 3 has landed. Before testing it with our projects, I was curious about the plugins that are defined in the Maven master POM's pluginManagement section and hence are locked-down with respect to their version. Since all projects inherit from this master POM, they will use the respective version of the plugins if not explicitely overwritten anywhere in the project's POM hierarchy.

Maven 3 is a bit more strict concerning automatic version resolution of invoked plugins. Other than Maven 2, it will always use the latest release (i.e. non-SNAPSHOT) version of a plugin if there was no explicit version specified in the POM or on the command line. Moreover, it will issue a warning when missing plugin versions are detected "to encourage the addition of plugin versions to the POM or one of its parent POMs". This is to increase reproducability of builds.

Thus, in Maven 3 the desired build stability is ensured by urging the POM author to give explicit plugin versions, and doesn't any more rely on a full list of plugins (with versions) defined in the master POM. That's why I expected to find a small or even empty pluginManagement section. Well, let's see.

To find out what's in the pluginManagement of master POM, you just have to create a minimal POM and show the effective POM (that results from the application of interpolation and inheritance, including master POM and active profiles) by calling help:effective-pom for this simple project.

So, what do we get? The following list shows the plugin versions that are defined in the Maven 2.2.1 master POM, the Maven 3 master POM, as well as the most recent version of those plugins.

Well, we can see some interesting details here:

The number of plugins defined in the master POMs pluginManagement section is drastically less for Maven 3 than for Maven 2.2.1 – that's what we expected. However, there are still a few.

Which pluings are listed and which are not? It seems like the plugins for the most basic lifecycle phases (like clean, install, deploy) are predefined, but others are not (like compile or jar). Is there any policy?

What is really odd: for some of the plugins that are predefined, there is a newer version available than is listed in the Maven 3 master POM (colored red). Why could that be? I have not checked, but Maven 3 is out for a few days now, so I suspect for most of those plugins the new version has been available before. Is that intentionally? Are the new versions not considered "good" or "stable" by the Maven guys? Or did they just forgot to upgrade? Or did not found it important in any kind?

Another thing I can't explain: when you look on the Maven 3 Project Plugin Management site, there are listed a lot more plugins, and some are even of other version than what we got by showing the effective POM for a minimal project POM. How could this be? I have no clue...

In a previous post, I have listed the plugins predefined by Maven 3.0-alpha5. Interestingly, there have been a lot more of them (like for Maven 2.2.1), but the "stale version" question was the same...

World of Versioning

2010-10-13T17:46:00.005+02:00

Today, we had a discussion on how to name a hotfix release of our framework product, built with Maven (you knew I'm a fan of Maven, didn't you?). It's a very basic question, but still an interesting one and it opens a whole universe of ideas, opinions and rules...

The previous versions of our product had been named like this:

1.3.0. 1.3.1 ... 1.4.0, 1.4.1, ... 1.5.0, 1.5.1, 1.5.2, ... 1.5.6

They all are based on a release plan and contain bugfixes as well as improvements and new features. For each of those versions, we have written release notes and built a site.

Now, what do we do when there is the need to release a bugfix version of a regular release we built a few days ago? There are some options:

1.5.7 – i.e. increment last number; however, this doesn't seem to fit well because the bugfix release is of another character than standard releases
1.5.6.1 – i.e. add an additional numerical identifier
1.5.6.a – i.e. add another non-numerical identifier
1.5.6-patch1 – i.e. add another qualifier describing it's actually a patch release

When searching the Net for version number rules in the Maven world, you'll stumble upon the DefaultArtifactVersion class in the core of Maven which expects that version numbers will follow a specific format:

<MajorVersion [> . <MinorVersion [> . <IncrementalVersion ] ] [> - <BuildNumber | Qualifier ]>

Where MajorVersion, MinorVersion, IncrementalVersion and BuildNumber are all numeric and Qualifier is a string. If your version number does not match this format, then the entire version number is treated as being the Qualifier (see Versions Maven Plugin).

This means, options 1 and 4 of above would be a viable alternative in the Maven world. However, note that there is some discussion about this Maven schema. It suffers from inconsistent/unintuitive parsing, lexically sorting of qualifiers and some other flaws. This would yield to unexpected comparison results especially when using Maven SNAPSHOT versions. The Proposal given on that page seems to be integrated with Maven 3.

Actually, we wouldn't have this discussion if the third level would not be named Incremental version in Maven world, but rather bugfix version or patch version. There is a Semantic Versioning Specification (SemVer) that recommends this version schema:

A normal version number MUST take the form X.Y.Z where X, Y, and Z are integers. X is the major version, Y is the minor version, and Z is the patch version. Each element MUST increase numerically. For instance: 1.9.0 < 1.10.0 < 1.11.0.

There are some rules describing when to increase which part. The main idea is to use the first numerical (major version) to indicate backwards incompatible changes to the public API, and in contrast the last numerical (patch version) suggests that only backwards compatible bug fixes have been introduced.

This SemVer schema is fully compatible with Maven (regardless of SNAPSHOT versions). If we had used this, we would probably have ended up in a "higher" version number like 5.4.0, but now the upcoming patch would have the version number 5.4.1 without any consideration.

By the way, a lot of public recommendations for software versioning follow this <major>.<minor>.<patch> schema. See this question and Wikipedia for more information on Software Versioning.

So. What do we do now? We'll release a version 1.5.6-patch1 for the patch, but think about changing our versioning according to SemVer, i.e. to upgrade the major number when introducing incompatible changes, and the minor number in most other cases.

Fix Foreign Code

2010-09-27T13:47:00.009+02:00

Well, finally, I'm back! I have been busy working on-site for a customer of my company, helping to fix their project and increase quality to successfully conduct the rollout. Additionally, I spent my evenings working as a release manager and keeper of the Maven based infrastructure for several projects developed in-house. So this was more than a fulltime job and unfortunately no time was left to read or write blog posts :-(

However, that project assignment is nearly over now and I intend to write more regularly now about my findings, trials and tribulations.

Foreign Code Dilemma

My main task when working for our customer was to fix bugs and improve quality of their application, which was nearly completely implemented with respect to use cases and business requirements. This is a situation that might be known to most developers: you are thrown into a project you don't know much of, lots of source code is already implemented, quality is, well, varying, and some important milestone or release date is right ahead. This is what I call the Foreign Code Dilemma.

What do you do to quickly get up to speed and rescue the project? Well, there are some things that I find quite useful in situations like this. In no particular order...

Introduce Continuous Integration

It should be common sense these days that Continuous Integration (CI) is able to improve software quality and reduce integration issues as well as overall risks. CI is a software development practice where changes are integrated frequently – usually at least daily – and the result is verified by an automated build and test to detect issues as quickly as possible. The distinguished article about Continuous Integration by Martin Fowler is a must-read.

Fortunately, the customer's project already provided automated Ant build scripts to checkout, build and test the software. Moreover, they were running on a Cruise Control server each night, so we were quite close.

The first thing I did was to move to Hudson, the best integration server available today (if you'd ask me). The transition was quite smooth and done within a few hours, including setting up a brand new build server. If you're still using Cruise Control, you really should consider to move over to Hudson... I think I should post about the cool distribution feature of Hudson soon.

One issue with the project was the build time: a full build takes 3-4 hours, mainly due to long-running unit and selenium test cases. Of course, this inhibits doing real CI. All we could do for now was to split up the build into the four main tasks, creating a Hudson job for each of them: (1) checkout & compile & package, (2) static code checks, (3) unit tests, (4) selenium tests. Since (1) and (2) are running rather quick (about 10 min) those jobs qualify for CI builds. This is not perfect but still better than doing no CI at all.

Introduce Test Cases

Test cases are an essential part of a software development project these days, and I always consider a tasks not being finished unless there are test cases ensuring that the functionality is implemented correctly. I'm sure you agree ;-)

The project I was working on had lots of JUnit test cases, as well as hundreds of Selenium tests checking the web application in the browser. That's not bad, really. Nevertheless, there were two issues:

The number of test cases does not tell anything about the test coverage. For example, the Selenium tests all did test a "happy day" scenario, moving through the wizard pages of the web application straight from the first to the end. But does it still work, for instance, if you step to the forth page, choose some options on that page, step back two pages, change an option, and go to the forth page again? Nobody tested.

Selenium web tests are slow, which is no surprise taken the fact that the tests are running in a browser and need to connect to the deployed web application. In my project, the full test suite took more than 3 hours to run... What's even worse is that some of the JUnit tests have not been designed as unit tests, i.e. they required a full service stack to run successfully, as such being more an integration than a unit test. As a consequence, those tests require to startup all services which takes a lot of time.

Thus, the task for this project actually was not to introduce, but to improve unit tests: increase code coverage and separate unit from integration tests. This way, unit tests can be run within CI builds, providing a quick result for the quality of committed code.

Introduce Code Metrics

When more than a few people are working on a project, establishing a coding standard is usually a rewarding idea. It helps you to be comfortable with the sources of anybody else from your team, and when doing code comparison you don't see differences all the time that are just caused by reformatting, hiding the significant changes.

If you have defined such coding standards, you need to check them. Checkstyle is the tool of choice. Here is what you should do:

Define a Checkstyle configuration to be used for your project. Discuss the rules with developers and stakeholders.
Run Checkstyle with your CI and/or nightly builds to create a report, including a list of violations for defined rules.
Establish Checkstyle within your IDE of choice to provide immediate feedback to the developers before they commit.
Define which exceptions to the rule are acceptable (should not be more than a dozen or so) and suppress them permanently, using Checkstyle suppression filters.
Get rid of all remaining violations, which might take a few days of effort. Still, this investment will pay off.
Once the number of Checkstyle violations is "small" (meaning less than 10, ideally zero), make sure it remains small.
Establish a team culture where committing code with Checkstyle violations is anything else but cool.

That works quite well in my experience. For the mentioned project, we already had common Eclipse formatting settings, but Checkstyle helped to further improve the code and people adopted it right from the start.

The Debugger is Your Best Friend

When you have to fix bugs in code you have never seen before, use the debugger as much as possible. To find the hot spot, you usually don't have to read or understand the whole class or even hierarchies of classes. Thus, it'll save you a lot of time when you don't start with code reviews but use the debugger to find the piece of code to blame.

BTW, the same applies to the look and feel of web applications. Instead of consulting lots of layout code and stylesheets, use browser tools like Firebug to debug pages, styles and JavaScript code (including Ajax requests) right in the displayed page.

Of course, this approach is not appropriate when fixing larger design issues...

Don't Be Shy!

When using this toolset, you shouldn't be shy. If you think some code needs refactoring, do so – maybe not a week before going live, but you get the point. The CI build should give you immediate feedback if the change could be integrated, and the tests will tell you if everything still works. Take your chance to improve the code. If your change anyhow is causing an issue, fix it, add another test and don't be discouraged!

HDD / SSD Battle

2010-04-19T15:37:00.003+02:00

The Problem

You know, the laptop I'm using for my daily work job is not the fastest one. In contrast, it's more than 5 years old and pretty slow. Yeah I know, hardware can never be fast enought, but it's really slow considering the things I have to work on.

For instance, we are using xtext modeling and hence usually have a couple of Eclipse instances running at the same time (outer & inner workbench), additionally to using m2eclipse to build the projects with Maven in Eclipse. Moreover, we have some quite big workspaces with tens of thousands of class files.

All of this is probably not unusual, but unfortunately too much for my poor old laptop. It takes minutes to start or end Eclipse, not to mention the times required for cleaning all projects. However, my company currently does not really like the idea to buy new laptops so we have to find ways to speed things up without spending too much money. I have blogged before about some ways to speeding up your system.

The Solution?

It's pretty clear that the bottleneck is the hard drive currently. We have proven this by some inspection tools, the drive is working hard all the time when executing some build, for instance. Now we managed to get a solid state drive (SSD) to test the performance improvements it would offer. Well, fasten your seatbelt...

We have measured some typical tasks with real data and projects on a developer's laptop – first with the built-in hard disk, then after installing the SSD and copying the harddrive content over. Note that we have tried to make a fair comparison, keeping the setup indentical in both scenarios. These are the results.

The Battle

Working With Eclipse:

Start Eclipse 3.5.1 with an empty workspace until Welcome screen is displayed: 52 s → 12 s (factor 4.3)
Start Eclipse with a medium-size workspace: 125 s → 30 s (factor 4.2)
Clean all projects in that workspace: 445 s → 115 s (factor 3.9)
Exit Eclipse and wait until workspace is saved: 28 s → 7 s (factor 4.0)

Working With Maven:

Maven "clean install" in mdium-size project: 668 s → 336 s (factor 2.0)

Booting Windows:

Turn on computer and wait for login screen: 62 s → 33 s (factor 1.9)
After login, until Windows is ready (autostart applications are loaded): 135 s → 44 s (factor 3.1)

The Bottom Line

As you can see, the SSD is speeding up boot time by factor 2-3, which already is impressive. Maven build usually gets executed 2 times faster. Eclipse speed-up is even more, namely around factor 4. That's pretty cool! You really feel the performance difference!

Additionally, after some more weeks of testing, what we like most is that the whole system feels much more reactive; that is, when executing some big job like rebuild a huge workspace, you can switch context and nicely work in another instance of Eclipse, for instance – a single tasks is not blocking the whole system any more.

All in all, that's an incredible speed-up considering the prices of SSD! Now, go and tell your boss ;-)

Having Fun with Encoding!

2010-03-26T16:20:00.004+01:00

Compiler Plugin

Recently, I edited our main company root POM to upgrade some plugins to new versions. Of course, we are following best practice to lock down the plugin version, so when a new version is available we only need to adjust the parent POM. Nearly all version updates were on the last build number digit, which is the z in x.y.z version string – so I didn't expect much difficulties.

However, for the compiler plugin, it was a jump from version 2.0.2 to 2.1, and indeed it turned out that some of the test cases failed compiling with strange encoding issues when using the new compiler plugin version.

Specify Encoding

We are following the suggestion to specify a POM property for source file encoding, for not being forced to configure encoding for all relevant plugins individually. Moreover, we were exactly using what's shown in the example:

<project>
  ...
  <properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    ...
  </properties>
  ...
</project>

That is, we assumed our source files were UTF-8 encoded, which is the most widely used encoding for unicode characters. But, for some of the projects, that's actually not the case since we are using Eclipse with the default setting for text file encoding which is Cp1252 (Western European) on our german Windows.

Why didn't we ever notice that? Well, it happens that both the UTF-8 as well as Cp1252 encodings are backwards compatible with ASCII. We are coding most of the stuff in english (concerning package, class, method, attribute and parameter names, and even Javadoc comments), so the resulting byte stream will never be different for both encodings. However, some of the files used german umlauts in line comments which are exactly the files that can't be compiled any more with new compiler plugin version.

When looking at the debug output of compiler plugin 2.0.2 mojo configuration, you can see that the encoding is not explicitely set, probably meaning that the platform default encoding is used (which is again Cp1252 on all build machines):

[DEBUG] Configuring mojo 'org.apache.maven.plugins:maven-compiler-plugin:2.0.2:compile' -->
[DEBUG]   (f) basedir = ...
[DEBUG]   (f) buildDirectory = ...
[DEBUG]   (f) classpathElements = [...]
[DEBUG]   (f) compileSourceRoots = [...]
[DEBUG]   (f) compilerId = javac
[DEBUG]   (f) debug = true
[DEBUG]   (f) failOnError = true
[DEBUG]   (f) fork = false
[DEBUG]   (f) optimize = true
[DEBUG]   (f) outputDirectory = ...
[DEBUG]   (f) outputFileName = xxx-0.2.0-SNAPSHOT
[DEBUG]   (f) projectArtifact = xxx:jar:0.2.0-SNAPSHOT
[DEBUG]   (f) showDeprecation = false
[DEBUG]   (f) showWarnings = false
[DEBUG]   (f) source = 1.6
[DEBUG]   (f) staleMillis = 0
[DEBUG]   (f) target = 1.6
[DEBUG]   (f) verbose = false
[DEBUG] -- end configuration --

The new version 2.1 of compiler plugin is now considering what has been configured in project.build.sourceEncoding property, and hence tries to compile the Cp1252 coded source file with UTF-8 encoding which doesn't work when umlauts are used.

Specify Correct Encoding

Of course, the solution is to specify the correct encoding in project.build.sourceEncoding property, matching the encoding that is used in the development environment when writing the source files.

Oh, yes, Cp1252 is quite similar to ISO 8859-1 encoding (only some special characters on positions 0x80–0x9F are different which we don't use), so in fact we are using ISO 8859-1 now to allow builds on non-Windows platforms as well.

Certainly, it would be nice if the plugins had a history on their site where you can find this type of changes for new versions, without having to search in the Jira...

Eclipse: Update Manager Needs Update!

2010-02-26T15:30:00.003+01:00

Eclipse Update Manager is really a special piece of software... I have blogged before about my battle, and here is another one.

We have a simple Eclipse plugin (created by xtext to provide an editor for our DSL, but actually this doesn't matter). I have a particular version (let's say 1.0.0) of that installed in my Eclipse 3.5.1. Now I want to upgrade to 1.1.0, but unfortunately the feature id has changed, so I need to uninstall my 1.0.0 version prior to installing the new one.

But... when I try to uninstall this plugin Eclipse tells me that it is "Calculating requirements and dependencies". To do so, Eclipse downloads a lot of stuff, including Eclipse features, mylyn, and much more. Seems to be half the internet which takes while. And then, about 15 min later, Eclipse tells me that it could not find a download site for some weird mozilla plugin.

Hello? What's that? I want to uninstall a plugin and Eclipse downloads tons of jars only to tell me that one is missing and it couldn't uninstall? Gosh!

After some googling, I found a trick to force Eclipse to just do what I want:

In Eclipse Preferences, on Install/Update > Available Software Sites page, export all sites to your filesystem.
Then remove all update sites and press OK.
Now uninstall the plugin -- for me, it just worked like a charm.
After restarting Eclipse, open Install/Update > Available Software Sites again and import the previously exported update sites.

That's it. Maybe just pulling the network cable would have worked, too... Oh boy.

Maven vs. Ant: Stop the Battle

2010-02-20T13:54:00.003+01:00

Maven? Ant?

Oh boy, how this bothers me. The endless debate and religious battle about which build tool is the better build tool, no, is the one and only right build tool...

There are many people out there who love Ant, who defend Ant with their blood and honour. That's fine, but some of them at the same time shoot at Maven. There is so much rant about Maven, so much unfair allegation and just plain wrong claims. This is just one example that has been discussed in the community lately.

Don't get me wrong. Maven has its flaws and issues, sure, and you don't have to like it. Use Ant, or Gradle, or Buildr, or Schmant, or batch files, or anything else if you like that more. But, Maven definitely can be used to build complex software projects, and lots of people are doing exactly that; and guess what -- some of them even like this tool... So, can everybody just please use what he or she likes the most for building their software, and stop throwing mud at each other? Let's get back to work. Let's put our effort in building good software.

We've Come a Long Way...

You may have guessed, I think Maven is the best build tool, at least for the type of projects I am dealing with in my company. We have started using a complex system of mutual calling batch files long time ago, and switched to Ant in 2000. That was a huge step ahead, but still it was a complex system with lots of Ant-Scripts on different levels. So we moved to Maven 1.0.2 in 2004 for another project. That brought nice configuration and reporting features, but still did not feel right, especially for multi-module projects that were not supported in the Maven core at that time.

When Maven 2 came out, we adopted that early and suffered from many teething troubles, but nevertheless we were sure to be on the right track. Today, Maven is a mature, stable, convenient build tool for all our projects, and the first time we are quite happy with how it works and what it provides. Moreover, it sound really great what the brave guys from Sonatype have in their pipeline: Maven 3, Tycho, and all those nice tools like Nexus and m2eclipse...

Hence, I am happy and honestly don't really care very much about what the blogosphere is telling about Maven. But the sad thing is, my colleagues (mostly used to Ant build systems) are complaining with the same weird theses about Maven. I'll give you one example.

The Inhouse Battle

In my current project, we create EJBs in some JARs and assemble an EAR file for the whole application. Now we have to create another RAR to be put in the EAR, so I setup a new project (following Maven's convention "one project, one artifact") for the RAR. This is what the "Ant guys" didn't like: "Why can't Maven create that RAR within the main project, you know Ant could do that, so maybe we should use Ant here again, why have so many small projects, this is polluting our Eclipse's Project View, so much complexity, Maven sucks, I knew that before, blah blah..."

Well, I tried to explain that Maven of course can be configured to create multiple artifacts per project, but that's not the recommended way because it violates Maven convention. It's all about modularity and standardization. That is how Maven works, and it's great this way. A small project is not much overhead at all, it is going to have a clean and simple POM, and by the way we discovered a dependency cycle in the code that had to be fixed in order to move the RAR code into a separate module.

So, what's wrong with Maven? Is it just that you want to do it your way and not to subordinate the Maven way? A matter of honor and ego? Is that enough to kick out Maven and go back to your Ant and script based build system (which BTW is so complex that only few guys really know how it works)? Come on.

The Bottom Line

IMHO, standardization of build systems is one of the main benefits that Maven brought to the world. If you know one Maven project, you can switch to any other project built with Maven and feel comfortable immediately. This increases productivity, both personally and for your company, which is one of the reasons more and more companies switch over from Ant to Maven.
We have clean conventions, a nice project structure, and a highly modular system. And, we have world class reporting with minimal effort.

You see, that's why we are using Maven. If you don't like it, go your own way but let us just do our job.

@Override Changes in Java 6

2010-02-08T11:39:00.002+01:00

Today I have ported a Java 6 project back to Java 5. This led to compiler failures in Eclipse, but not in Maven which seemed quite strange at first glance. Interestingly, they are caused by the @Override annotation.

The Java 5 API for @Override says:

: Indicates that a method declaration is intended to override a method declaration in a superclass. If a method is annotated with this annotation type but does not override a superclass method, compilers are required to generate an error message.

Note that it says "superclass", not "supertype". Hence, it's not allowed to add this annotation to methods that implement methods of an interface. Javac (which is called by Maven) does not report this as an error, but the Eclipse compiler does.

Well, if you take a look at Java 6, the API didn't change at all so I was surprised to see a different behavior: the @Override annotation is allowed for methods implementing interface methods in Javac, too. In the end, I had to remove those annotations to make the code compile with Java 5 in Eclipse.

After some googling, I found out that this has just been forgotten by Sun developers: the compiler's behavior is changed but the documentation does not reflect that (see here). And indeed, when you look at the API of @Override in upcoming Java 7 it looks like:

Indicates that a method declaration is intended to override a method declaration in a supertype. If a method is annotated with this annotation type compilers are required to generate an error message unless at least one of the following conditions hold:

The method does override or implement a method declared in a supertype.
The method has a signature that is override-equivalent to that of any public method declared in Object.

Here you got it: @Override may now bee used for interface methods, too.

Optimization: Don't do it... The compiler will!

2010-02-02T15:49:00.004+01:00

The Two Rules of Program Optimization

I've seen some bad code lately which was designed in an effort to improve performance. For instance, there was a long method (80 lines) that was not split into several methods for a single reason: to avoid the method call overhead (around 15 nanoseconds!). The result was code that was just hard to read.

This reminded me of the rules of program optimization (coined by Michael A. Jackson, a British computer scientist) we were teached back on university:
The First Rule of Program Optimization: Don't do it.
The Second Rule of Program Optimization (for experts only!): Don't do it yet.

Well, this is true for mainly two reasons:

Optimization can reduce readability and add code that is used only to improve the performance. This may complicate programs or systems, making them harder to maintain and debug.
Doing optimizations most of the time means we think to be smarter than the compiler, which is just plain wrong more often than not.

Cleaner Code

Donald Knuth said "Premature optimization is the root of all evil". Whereas "Premature optimization" means that a programmer lets performance considerations drive the design of his code. This can result in a design that is not as clean as it could have been, because the code is complicated by the optimization and the programmer is distracted by optimizing.

Therefore, if performance tests reveal that optimization or performance tuning really have to be done, they usually should be done at the end of the development stage.

Wrong Intuitions

This is what Sun Microsystem's Technology Evangelist Brian Goetz thinks: "Most performance problems these days are consequences of architecture, not coding – making too many database calls or serializing everything to XML back and forth a million times. These processes are usually going on outside the code you wrote and look at every day, but they are really the source of performance problems. So if you just go by what you're familiar with, you're on the wrong track. This is a mistake that developers have always been subject to, and the more complex the application, the more it depends on code you didn't write. Hence, the more likely it is that the problem is outside of your code." Right he is!

Smarter Compiler

Often, the best way to write fast code in Java applications is to write dumb code – code that is straightforward, clean, and follows the most obvious object-oriented principles in order to get the best compiler optimization. Compilers are big pattern-matching engines, written by humans who have schedules and time budgets, so they focus their efforts on the most common code patterns, in order to get the most leverage. Usually hacked-up, bit-banging code that looks really clever will get poorer results because the compiler can't optimize effectively.

A good example is string concatenation in Java (see this conversation with Java Champion Heinz Kabutz where he gives some measures)...

Back in the early days, we all used the String addition (+ operator) to concatenate Strings:
return s1 + s2 + s3;
However, since Strings are immutable, the compiled code will create many temporary String objects, which can strain the garbage collector.
That's why we were told to use StringBuffer instead:
return new StringBuffer().append(s1).append(s2).append(s3).toString();
That was around 3-5 times faster those days, but the code became less readable. Was it worth it? Is your code doing enough String concatenation to make you really feel a difference after you (for instance) made that execute three times faster?
Is that still the recommended way? A main downside of StringBuffer is its thread safety that is usually not required (since they are not shared between threads), but slows things down. Hence, the StringBuilder class was introduced in Java 5, which is almost the same as StringBuffer, except it's not thread-safe. So, using StringBuilder is expected to be significantly faster, and know what? When Strings are added using the + operator, the compiler in Java 5 and 6 will automatically use StringBuilder:
return s1 + s2 + s3;
Clean, easy to understand, and quick. Note that this optimization will not occur if StringBuffer is hard-coded!

That was just one example.... All in all, it's quite simple: today's Java JIT Compilers are highly optimized and clever in optimizing your code. Trust them. Don't try to be even more clever. You aren't!

Concurrent Builds with Hudson

2010-01-13T10:26:00.004+01:00

Multiple Build Executors

We are using the Hudson Continuous Integration Server for our integration builds and are quite happy with it. It is fast, stable, feature-rich, extensible, well integrated with Maven and has an appealing user interface.

One of the nice features that we are using regularly is the Build Executor setting that allows you to specify the number of simultaneous builds. This is useful to increase throughput of Hudson on multi-core processor systems, where the number of executors should (at least) match the number of available cores.

However, Maven isn't really designed for running multiple instances simultaneously since the local respository isn't multi-process safe. The chance for conflicts seems small (multiple processes must access the same dependency at the same time, at least one of them writing). However, in praxis, we encounter this type of concurrency issue at least once a day now, which is starting to hurt us! The build is failing with a message like this:

[INFO] ------------------------------------------------------------------------
[ERROR] BUILD ERROR
[INFO] ------------------------------------------------------------------------
[INFO] Failed to resolve artifact.

GET request of: some/group/some-artifact-1.2.3-SNAPSHOT.jar from my-repo failed
  some.group:some-artifact:jar:1.2.3-SNAPSHOT
...

Caused by I/O exception: ...some-artifact-1.2.3-SNAPSHOT.jar.tmp (The requested operation cannot be performed on a file with a user-mapped section open)

or this:

[INFO] ------------------------------------------------------------------------
[ERROR] BUILD ERROR
[INFO] ------------------------------------------------------------------------
[INFO] Failed to resolve artifact.

Error copying temporary file to the final destination: Failed to copy full contents from ...some-artifact-1.2.3-SNAPSHOT.jar.tmp to ...\some-artifact-1.2.3-SNAPSHOT.jar

The reason is, the JAR file is locked by another process that is executing some long-lasting test cases, for instance. At the same time, a second build tries to download a new version of this snapshot into the local repository, which is done with the help of the mentioned .tmp file.

Safe Maven Repository

The only way to avoid this type of issue is to use separate local Maven Repositories for each of the processes. You can tell Maven to use a custom local repository location by specifying the localRepository setting in your settings.xml file.

In Hudson, this is even more convenient. There is a checkbox Use private Maven repository in the advanced part of the Build section of Maven projects. Just click that to setup a private local Maven repo for that project. You should consider to do so when you run into the described issue now and then.

Obviously using private repos will increase the total amount of disk space due to caching the same dependencies in multiple places. Additionally, the first build will take significantly more time because everything has to be downloaded once. However, both consequences are well acceptable given the better stability and isolation of projects.

Instead of clicking the Hudson checkbox for all your projects, you should consider to setup the local Maven repo in your settings.xml instead. This has a number of advantages:

You don't have to setup the option for each and every project, but have it in a central place.
You can use a common root for all local Maven repos, like d:/maven-repo. This allows you to easily purge all your local repositories from time to time, in order to reduce disk space as well as validate the content (i.e. make sure the build is still running in a clean environment and all required artifacts are in your corporate Maven repository).

For instance, here is what works fine for us:

<localRepository>d:/builds/.m2/${env.JOB_NAME}/repository</localRepository>

This is using a Hudson environment variable (JOB_NAME) to create subfolders for the actual projects aka jobs. See here for a list of available variables.

Oh yes, what I suggest is also encouraged by Brian Fox in his Maven Continuous Integration Best Practices blog post, so you should consider twice to adopt this best practice :o)

Cargo Maven Plugin: Not Made for JBoss

2010-01-02T18:24:00.002+01:00

Again...

Well, actually this blog was supposed to be about Java in general and all the ups and downs I experience during my daily work. However, I've not been doing much other than Maven configuration and build management lately, so here is another Maven related post. Sorry folks.

As already shown in this post, I have been doing integration tests with JBoss by using the Cargo Maven plugin to start the JBoss locally and deploy the application to it. This all works quite as soon as you have figured out how to configure Cargo for JBoss.

But Remotely Now!

Now, the next step is to deploy our EAR file which is generated during nightly build to a running JBoss instance on a separate computer. This is different because no JBoss configuration has to be created locally and no JBoss has to be started. Instead, the EAR file must be transferred to a remote server where JBoss is already running, and JBoss must be persuaded to deploy this file.

That sounds feasible, and I've done exactly this before for other servers like Tomcat, so I did not expect any issue here. However, I was wrong.

Itch #1

First trouble was caused by my lack of knowledge regarding JBoss. With standard installation, you are not able to connect to the server remotely and all the services are bound to localhost only (see here or here). This is intentionally, to prevent unprotected installations appearing all over the net. You have to pass the option -b 0.0.0.0 when starting JBoss to allow remote connections to the services, but take care to secure your JBoss accordingly!

Itch #2

Okay, after this has been configured, I tried to use Cargo to deploy my EAR file to JBoss. This is the configuration I ended up with:

<!-- *** Cargo plugin: deploy the application to running JBoss *** -->
<plugin>
  <groupId>org.codehaus.cargo</groupId>
  <artifactId>cargo-maven2-plugin</artifactId>
  <version>1.0</version>
  <configuration>
    <wait>false</wait>
    <!-- Container configuration -->
    <container>
      <containerId>jboss5x</containerId>
      <type>remote</type>
    </container>
    <!-- Configuration to use with the Container -->
    <configuration>
      <type>runtime</type>
      <properties>
        <cargo.hostname>...</cargo.hostname>
        <cargo.servlet.port>8080</cargo.servlet.port>
      </properties>
    </configuration>
    <!-- Deployer configuration -->
    <deployer>
      <type>remote</type>
      <deployables>
        <deployable>
          <location>...</location>
        </deployable>
      </deployables>
    </deployer>
  </configuration>

  <executions>
    <execution>
      <id>deploy</id>
      <phase>deploy</phase>
      <goals>
        <goal>deployer-redeploy</goal>
      </goals>
    </execution>
  </executions>
</plugin>

However, I always got this error message:

[INFO] Failed to deploy to [http://...]
Server returned HTTP response code: 500 for URL: ...

The configuration seems to be correct, so what is the problem?

After asking Google, I realized that Cargo is not able to transfer a file to JBoss! Instead, it requires the deployable to be deployed to be present on the server filesystem (see here). This is obviously caused by the JBoss JMX deployer which is used by Cargo, but actually you don't care who is to blame – you just want it to work. The name "Cargo" implies the parcel is transferred to its destination, right? Also note that this issue is dated from Sep 2006, so there has been some time to fix it in either way.

What Can We Do?

Well, there are probably not many options. Since current version of Cargo is not able to transfer the file to the server, you'd have to do this on your own. The location given in our Cargo configuration above actually is the path on the JBoss server. So, when the file exists locally on the JBoss server, Cargo should be able to deploy it successfully.

For transferring the file to JBoss server, we could use the maven-dependency-plugin, a quite useful plugin for all kind of analyzing, copying or unpacking artifacts. We configure it to run in pre-integration-test phase and to copy the EAR file (produced by this POM) to some temp directory on the JBoss server:

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-dependency-plugin</artifactId>
  <executions>
    <execution>
      <id>copy</id>
      <phase>install</phase>
      <goals>
        <goal>copy</goal>
      </goals>
      <configuration>
        <artifactItems>
          <artifactItem>
            <groupId>${project.groupId}</groupId>
            <artifactId>${project.artifactId}</artifactId>
            <version>${project.version}</version>
            <type>${project.packaging}</type>
            <destFileName>test.ear</destFileName>
          </artifactItem>
        </artifactItems>
        <outputDirectory>${publish.tempdir}</outputDirectory>
        <overWrite>true</overWrite>
      </configuration>
    </execution>
  </executions>
</plugin>

The property ${publish.tempdir} can be anything on the JBoss server (which must be available in the network!) and is exactly what has to be used for the value of location element in Cargo configuration.

Another option would be to use the hot-deploy directory of JBoss as outputDirectory for the dependency plugin, and hence rely on hot deployment of JBoss instead of Cargo and JBoss JMX deployer. This way, we could get rid of Cargo configuration and cleanup the POM a bit, but in the end it seemed a bit less clean to me... your mileage may vary.

So, as always, in the end we got it to work, but not without unforeseen pain. When will Cargo be fixed to get the EAR file to JBoss server? Who knows.

Updates

2010/01/22: Note that the dependency plugin must be bound after the install phase so that the artifact has been copied at least to your local Maven repository. As a consequence, the Cargo plugin must be run in deploy phase, which is actually a good choice anyways. I have changed this in my code above.

Maven Plugins: Upgrade with Care!

2009-12-10T14:10:00.002+01:00

Upgrading Maven Plugins: Tips and Issues

After having shown the list of current Maven plugin versions in my previous post, now I'm going to share my experiences with upgrading. Just like expected, some of the new plugins did not work out of the box or required some changes in configuration:

maven-checkstyle-plugin

We have used version 2.3 previously which is based on Checkstyle 4.4. In contrast, plugin version 2.4 is eventually built on top of Checkstyle 5 which better supports Java 5 language features (I wrote a post on that issue). The configuration is not fully compatible, so you would have to upgrade it.

maven-javadoc-plugin

Starting with version 2.6, JavaDoc plugin can detect the Java API link for the current build. This did not work for us (probably due to missing proxy configuration), so we had to switch it off by setting detectJavaApiLink property to false. See plugin site.

taglist-maven-plugin

The configuration of tags has changed and is now a bit more extensive. Old format (using tags element) is still supported, but deprecated. See plugin documentation.

findbugs-maven-plugin

I have used version 2.2 which was the most recent one when changing the POM. However, this yielded some strange errors during site generation. This is the strack trace:

Generating "FindBugs Report" report.
  Plugin Artifacts to be added ->...
  AuxClasspath is ->D:\Profiles\Default User\.m2\repository\org\apache\maven\reporting\maven-reporting-impl\2.0\maven-reporting-impl-2.0.jar;...
[java] Exception in thread "main" java.io.FileNotFoundException: D:\builds\...\User\.m2\repository\...\maven-reporting-impl-2.0.jar;D:\Profiles\Default (The filename, directory name, or volume label syntax is incorrect)
[java]  at java.util.zip.ZipFile.open(Native Method)
[java]  at java.util.zip.ZipFile.<init>(ZipFile.java:114)
[java]  at java.util.zip.ZipFile.<init>(ZipFile.java:131)
[java]  at edu.umd.cs.findbugs.classfile.impl.ZipFileCodeBase.<init>(ZipFileCodeBase.java:53)
[java]  at edu.umd.cs.findbugs.classfile.impl.ZipCodeBaseFactory.countUsingZipFile(ZipCodeBaseFactory.java:92)
[java]  at edu.umd.cs.findbugs.classfile.impl.ZipCodeBaseFactory.makeZipCodeBase(ZipCodeBaseFactory.java:46)
[java]  at edu.umd.cs.findbugs.classfile.impl.ClassFactory.createFilesystemCodeBase(ClassFactory.java:97)
[java]  at edu.umd.cs.findbugs.classfile.impl.FilesystemCodeBaseLocator.openCodeBase(FilesystemCodeBaseLocator.java:75)
[java]  at edu.umd.cs.findbugs.classfile.impl.ClassPathBuilder.processWorkList(ClassPathBuilder.java:564)
[java]  at edu.umd.cs.findbugs.classfile.impl.ClassPathBuilder.build(ClassPathBuilder.java:195)
[java]  at edu.umd.cs.findbugs.FindBugs2.buildClassPath(FindBugs2.java:584)
[java]  at edu.umd.cs.findbugs.FindBugs2.execute(FindBugs2.java:181)
[java]  at edu.umd.cs.findbugs.FindBugs.runMain(FindBugs.java:348)
[java]  at edu.umd.cs.findbugs.FindBugs2.main(FindBugs2.java:1057)
[java] Java Result: 1

As you can see, we are using Windows default for local Maven repository (D:\Profiles\Default User\.m2) which obviously is causing a problem with the path later on. How sick is that?!?

Then, after having tried this and that, I discovered there is a brand new version 2.3 available so I tested that one and guess what – everything works fine again! Hence, don't use version 2.2, it seems to be broken...

docbkx-maven-plugin

The latest version of that plugin is 2.0.9, but that did not work correctly. It failed the build with this error:

ValidationException: null:30:723: Error(30/723): fo:table-body is missing child elements.
Required Content Model: marker* (table-row+|table-cell+)

The given line/position information did not match to anything suspicious, and I could not see anything wrong in our XSL files. So, no idea what this issue is telling me or what could I do about it. That's why I rolled back to 2.0.8 which just works fine.

The Bottom Line

With the given plugin versions, we are up to date again and managed to get rid of some issues hitting us since quite some time. Additionally, we are well prepared for upgrading to Maven 3. I hope I will be able to do so soon, to check if this is really a "drop-in replacement"... I will let you know!

Maven Plugins: Current Versions

2009-12-10T12:21:00.009+01:00

Upgrading Maven Plugins

In preparation for a later switch to Maven 3 (which is already knocking on the door) as well as to get rid of some plugin related issues we are suffering from, I decided to update the Maven plugins we use for build and site generation.

Of course, we are following best practice and are locking down the version of all plugins in project.build.pluginManagment.plugins section. This is done in the company's topmost POM, so that all company projects would use the same versions once they reference the latest parent POM.

As you might know, upgrading to new plugin versions is always an adventure and you have to test your builds seriously, which is of course hard when you are going to change the company settings...

Build Plugins

Well, here is the list of build plugins we now use with their current version, as well as (in brackets) the version that Maven 3.0-alpha5 defines in its internal POM. I have highlighted where both versions differ:

~~maven-archetype-plugin: 2.0-alpha-5~~ (see comment below)
maven-assembly-plugin: 2.2-beta-4 (2.2-beta-4)
maven-clean-plugin: 2.3 (2.3)
maven-compiler-plugin: 2.0.2 (2.0.2)
maven-dependency-plugin: 2.1 (2.0)
maven-deploy-plugin: 2.4 (2.4)
maven-ear-plugin: 2.4 (2.3.1)
maven-ejb-plugin: 2.2 (2.1)
maven-enforcer-plugin: 1.0-beta-1
maven-help-plugin: 2.1 (2.1)
maven-install-plugin: 2.3 (2.3)
maven-javadoc-plugin: 2.6.1 (2.5)
maven-jar-plugin: 2.3 (2.2)
maven-release-plugin: 2.0-beta-9 (2.0-beta-9)
maven-resources-plugin: 2.4.1 (2.4.1)
maven-site-plugin: 2.0.1 (2.0.1)
maven-source-plugin: 2.1.1 (2.0.4)
maven-surefire-plugin: 2.4.3 (2.4.3)
maven-war-plugin: 2.1-beta-1 (2.1-alpha-1)
build-helper-maven-plugin: 1.4
failsafe-maven-plugin: 2.4.3-alpha-1
cargo-maven2-plugin: 1.0
docbkx-maven-plugin: 2.0.8

It's a bit strange that Maven 3.0-alpha5 (which came out end of November) does not use the latest version of all those plugins, most of them having been released before that date. I don't know if this was intentional or not... Let's hope it's not because of unsure quality of latest plugin versions ;-) Anyways, I decided to upgrade to the latest available version for all plugins.

Reporting Plugins

Here's the list for plugins related to site reports:

cobertura-maven-plugin: 2.3
findbugs-maven-plugin: 2.3
jdepend-maven-plugin: 2.0-beta-2
maven-checkstyle-plugin: 2.4
maven-jxr-plugin: 2.1
maven-pmd-plugin: 2.4
maven-project-info-reports-plugin: 2.1.2
maven-surefire-report-plugin: 2.4.3
taglist-maven-plugin: 2.4

I'm going to show some tips and issues when upgrading to these versions in an upcoming post...

Unit and Integration Testing with Maven, Part 2

2009-12-02T08:51:00.020+01:00

Welome Back...

... to the second installment of this little series. After having seen the requirements and hassles when using Maven for testing in the last post, we are now going to list the possible solutions.

Remember, Maven does support different phases for unit and integration tests, but there is only one source directory (usually src/test/java), making it a bit difficult to setup and organize test environments. Hence, we have to use some way to separate both types of test.

Option 1: Separate Module for Integration Tests

In case you are using modules for your project anyways and want to have integration tests which are testing these modules in integration, this is the natural solution: just create another module that depends on the other ones and only contains the integration test sources and resources.

However, if you instead want to integration-test each module individually, this approach just doubles the number of modules, which can be difficult to manage. You could put all integration tests into one big module – but that has other disadvantages, of course.

Since we do not have any sources (only test sources) in that integration-test project, the usual build lifecycle would execute a lot of unneeded steps like copying resources, compiling, testing, creating JAR file etc. We could use the POM packaging type to suppress all of this, but then we have to configure the maven-compiler-plugin to force compilation of test sources.

Independantly of packaging type, we have to do some more configurations:

Execute Surefire plugin in integration-test phase.
Prepare integration tests (for instance, start the container and deploy the application) in pre-integration-test phase. This is quite easy with Cargo Maven plugin.
Shutdown integration tests (stop the container) in post-integration-test phase.

Here is the relevant part of such a POM file:

<build>
  <plugins>

    <!-- *** Compiler plugin: we must force test compile because we're using a 
             pom packaging that doesn't have this lifecycle mapping. -->
    <plugin>
      <artifactId>maven-compiler-plugin</artifactId>
      <executions>
        <execution>
          <goals>
            <goal>testCompile</goal>
          </goals>
        </execution>
      </executions>
    </plugin>

    <!-- *** Surefire plugin: run integration tests *** -->
    <plugin>
      <artifactId>maven-surefire-plugin</artifactId>
      <executions>
        <execution>
          <phase>integration-test</phase>
          <goals>
            <goal>test</goal>
          </goals>
        </execution>
      </executions>
    </plugin>

    <!-- *** Cargo plugin: start/stop application server and deploy the ear 
             file before/after integration tests *** -->
    <plugin>
      <groupId>org.codehaus.cargo</groupId>
      <artifactId>cargo-maven2-plugin</artifactId>
      <version>1.0</version>
      <configuration>
       ...
      </configuration>

      <executions>
        <!-- before integration tests are run: start server -->
        <execution>
          <id>start-container</id>
          <phase>pre-integration-test</phase>
          <goals>
            <goal>start</goal>
          </goals>
        </execution>
        <!-- after integration tests are run: stop server -->
        <execution>
          <id>stop-container</id>
          <phase>post-integration-test</phase>
          <goals>
            <goal>stop</goal>
          </goals>
        </execution>
      </executions>
    </plugin>

  </plugins>
</build>

Option 2: Different Source Directories

In this scenario, unit and integration test sources are placed in separate source directories, like src/test/java and src/integrationtest/java. I think this would definitely be the best solution, and it actually should be what Maven supports out of the box. Sadly, Maven does not, and as far as I know Maven 3 won't either :-(

Well, we should be able to configure things this way. For compiling and executing integration tests, you would have to configure Compiler plugin to compile the integration test sources in pre-integration-test phase (using the includes parameter), and then configure a second Surefire execution using testClassesDirectory parameter to point it to the integration test folder.

However, this option seems a bit fragile to me. Even if it works (I haven't checked), the integration test folder would probably not show up in Eclipse when using m2eclipse plugin and other plugins may have issues with this additional test source path as well. That's why I do not recommend this option.

Option 3: Different File Name Patterns

In this scenario, the package or file name is used to distinguish between unit and integration test source files. Let's say, for instance, that all integration test classes start with IT* and hence the filename pattern is **/IT*.java. We'd have to configure the Surefire plugin to execute twice: once in test phase for executing the unit tests only, and another time in integration-test phase to execute, well, the integration tests (and only those)

A common way to do so is this:

In configuration of Surefire plugin, set skip parameter to true to bypass all tests.
Add an execution element for unit tests, where skip is set to false and exclude parameter is used to exclude the integration tests.
Add another execution element for integration tests, where skip is set to false and include parameter is used to just include the integration tests and nothing else.
Prepare and shutdown of integration tests are like before.

Here is the POM section:

<build>
  <plugins>

    <!-- *** Surefire plugin: run unit and integration tests in 
             separate lifecycle phases, using file name pattern *** -->
    <plugin>
      <artifactId>maven-surefire-plugin</artifactId>
      <configuration>
        <skip>true</skip>
      </configuration>
      <executions>
        <execution>
          <id>unit-test</id>
          <phase>test</phase>
          <goals>
            <goal>test</goal>
          </goals>
          <configuration>
            <skip>false</skip>
            <excludes>
              <exclude>**/IT*.java</exclude>
            </excludes>
          </configuration>
        </execution>

        <execution>
          <id>integration-test</id>
          <phase>integration-test</phase>
          <goals>
            <goal>test</goal>
          </goals>
          <configuration>
            <skip>false</skip>
            <includes>
              <include>**/IT*.java</include>
            </includes>
          </configuration>
        </execution>
      </executions>
    </plugin>

    <!-- *** Cargo plugin: start/stop application server and deploy the ear 
             file before/after integration tests *** -->
    ...

  </plugins>
</build>

There is another approach to achieve the same result:

In configuration of Surefire plugin, use the exclude parameter to exclude the integration tests when executing unit tests in test phase.
Add an execution element for integration tests, where exclude is set to any dummy value (to override the default configuration) and include parameter is used to just include the integration tests and nothing else.

It's a bit shorter than the former configuration, but in the end it's a matter of taste. Again, here is the POM snippet:

<build>
  <plugins>

    <!-- *** Surefire plugin: run unit and integration tests in 
             separate lifecycle phases, using file name pattern *** -->
    <plugin>
      <artifactId>maven-surefire-plugin</artifactId>
      <configuration>
        <excludes>
          <exclude>**/IT*.java</exclude>
        </excludes>
      </configuration>
      <executions>
        <execution>
          <id>integration-test</id>
          <phase>integration-test</phase>
          <goals>
            <goal>test</goal>
          </goals>
          <configuration>
            <excludes>
              <exclude>none</exclude>
            </excludes>
            <includes>
              <include>**/IT*.java</include>
            </includes>
          </configuration>
        </execution>
      </executions>
    </plugin>

    <!-- *** Cargo plugin: start/stop application server and deploy the ear 
             file before/after integration tests *** -->
    ...

  </plugins>
</build>

Failsafe Plugin

Instead of using the Surefire plugin with custom filename pattern to distinguish between unit and integration test classes, you should rather use the Failsafe plugin for executing integration tests.

This plugin is a fork of the Surefire plugin designed to run integration tests.
It is used during the integration-test and verify phases of the build lifecycle to execute the integration tests of an application. Other than Surefire plugin, the Failsafe plugin will not fail the build when executing tests thus enabling the post-integration-test phase to execute.

Failsafe plugin has its own naming convention. By default, the Surefire plugin executes **/Test*.java, **/*Test.java, and **/*TestCase.java test classes. In contrast, the Failsafe plugin will look for **/IT*.java, **/*IT.java, and **/*ITCase.java. Did you note that this matches what we used before for our integration tests? ;-)

When using Failsafe, the last POM (of option 3) looks like this:

<build>
  <plugins>

    <!-- *** Surefire plugin: run unit and exclude integration tests *** -->
    <plugin>
      <artifactId>maven-surefire-plugin</artifactId>
      <configuration>
        <excludes>
          <exclude>**/IT*.java</exclude>
        </excludes>
      </configuration>
    </plugin>

    <!-- *** Failsafe plugin: run integration tests *** -->
    <plugin>
      <groupId>org.codehaus.mojo</groupId>
      <artifactId>failsafe-maven-plugin</artifactId>
      <version>2.4.3-alpha-1</version>
      <executions>
        <execution>
          <goals>
            <goal>integration-test</goal>
            <goal>verify</goal>
          </goals>
        </execution>
      </executions>
    </plugin>

    <!-- *** Cargo plugin: start/stop application server and deploy the ear 
             file before/after integration tests *** -->
    ...

  </plugins>
</build>

Of course, Failsafe would also work with first option (separate integration test module).

Conclusion

Well, after having shown all the ways that came into my mind, here are my personal "best practices":

Use the Failsafe plugin to execute integration tests.
If keeping integration tests in a separate module feels alright, do so (see option 1).
If you want to have unit and integration tests in the same module, choose a file or package name pattern to distinguish between both, and configure Surefire and Failsafe plugins accordingly.

Update (2010/02/18)

Note that there is a new version 2.5 of failsafe plugin available with changed group id: org.apache.maven.plugins:maven-failsafe-plugin:2.5. See the plugin site for details. Thanks to stug23 for pointing that out!

Unit and Integration Testing with Maven, Part 1

2009-11-28T17:39:00.007+01:00

Test Types

In my last post, I talked about integration testing with Maven's Cargo plugin and JBoss Application Server. Now let's see how integration testing fits into an overall testing strategy with Maven.

When thinking about tests and testing strategies, there is one important thing to keep in mind: integration tests are not unit tests (even though JUnit may be used to write integration tests). For the sake of completeness, let's pin down the main characteristics of both:

Unit Tests:

are testing a small piece of code in isolation
are independant from other tests
are usually written by software developers
have to be very fast because they are run quite often

In contrast, Integration Tests:

individual software modules are combined and tested as a group
are usually much slower than unit tests because a context has to be established (Spring, database, web server etc.)
normally are run after unit testing
may be created by QA team using tools, but also by developers using JUnit test cases (which still does not turn them into unit tests)

Testing with Maven

As you know, Maven Build Lifecycle provides phases both for testing and integration testing. Related phases are:

...
generate-test-sources
generate-test-resources
process-test-resources
test-compile
process-test-classes
test
...
pre-integration-test
integration-test
post-integration-test
...

Unfortunately, Maven does not support separate source directories for both test types. This is really complicating things (as we'll see in a minute). There have been some discussions on how to fix that, but I don't think it made it into Maven 3 (not quite sure, though).

Hence, what we need to have is this:

Both unit and integration tests have to be compiled by javac.
Nevertheless, they should be clearly separated from each other by a path or file name pattern.
Unit tests have to be run in test lifecycle phase, and the build should stop if they do not succeed.
Integration tests should be run in integration-test phase, and at least the post-integration-test phase has to be run independantly of test result (to be able to shutdown a running container, close database connections etc.).

The next part of this small series will show how these requirements can be met with Maven and what has to be configured, so please stand by...

Integration test with Maven, Cargo and JBoss

2009-11-27T17:22:00.017+01:00

It's Friday...

... and I thought it would be a good idea to setup an integration test of some EJBs we are creating in a new project. Actually, it was not, since it nearly ruined my evening. But eventually I got it to work, and here is how.

To test the EJBs, I need to create an EAR file, deploy that to the application server (JBoss 5.1.0 in our case) and run JUnit test cases against this server. For Maven, I have setup a separate integration-test module for executing the integration test for the following reasons:

to better separate unit tests (with JUnit) from integration tests (also with JUnit), which simplifies Maven configuration a bit.
to be able to run this module outside of the normal CI (continuous integration) build due to its lack of performance.

Cargo Maven Plugin

For automatically starting the container, deploying the EAR file and stopping the container when the tests are finished, I use the Cargo Maven plugin. I did this several times before (with Tomcat, though) – so I thought that'd be easy...

Well, when using JBoss, there are some tricks you have to know. Before going into the details, some more information on Cargo.

A Container is the base concept in Cargo. It represents an existing application server runtime, and Cargo hides all the details of the actual server implementation for you. There are two types of containers:

Local Container: this is executing on the machine where Cargo runs. This can either be an Installed Container which is, well, installed on the local machine and is run in a separate VM, or an Embedded Container that is executing in the same JVM where Cargo is running (currently only supported for Jetty).
Remote Container: a container that is already running anywhere (local or remote). It's not under Cargo's control and can't be started or stopped by Cargo.

You use a Configuration to specify how the container is configured (logging, data sources, location where to put the deployables, etc). The available configuration depends on the container type:

Local Configuration: for local containers. There are two local configuration types: Standalone Local Configuration which configures the container from scratch in a directory of your choice, and Existing Local Configuration that re-uses an existing container installation already residing on your hard drive.
Runtime Configuration: You use a runtime configuration when you want to access your container as a black box through a remote protocol (JMX, etc). This is perfect for remote containers.

In my case, I wanted to use a Local Installed Container with a Standalone Local Configuration, to eliminate dependencies from other deployments.

JBoss with Cargo

Well, and here are the pitfalls when using JBoss in this setting:

Experimental: JBoss 5.x is still an experimental container for Cargo. This is a bit strange given the fact that this version is now out for a while, but fortunately not really an issue.
Extensive Logging: When Cargo builds the JBoss configuration – remember, I use Standalone Local Configuration so Cargo creates one from scratch – it uses a logging setup (independantly from what is used with your JBoss installation!) that is way too chatty. The console scrolls forever, and things are slowing down in a way that you think everything is stuck in an infinite loop.
Thus, you have to tell Cargo to use another logging configuration file, which is a bit tricky and not documented very well (see this issue).
Shutdown Port: Now the container starts up, the tests are run, but after that JBoss AS is not shutting down. It's telling me javax.naming.CommunicationException: Could not obtain connection to any of these urls: localhost:1299, which means the wrong port is used for shutdown. Standard shutdown port is 1099, so we have to tell Cargo to use that port number.

All in all, the configuration now looks like this. Mentioned settings are highlighted. Perhaps this is useful for someone else...


<!-- *** Cargo plugin: start/stop JBoss application server and deploy the ear 
         file before/after integration tests *** -->
<plugin>
  <groupId>org.codehaus.cargo</groupId>
  <artifactId>cargo-maven2-plugin</artifactId>
  <version>1.0</version>
  <configuration>
    <wait>false</wait>
    <!-- Container configuration -->
    <container>
      <containerId>jboss5x</containerId>
      <type>installed</type>
      <home>${it.jboss5x.home}</home>
      <timeout>300000</timeout>
    </container>
    <!-- Configuration to use with the Container -->
    <configuration>
      <type>standalone</type>
      <home>${project.build.directory}/jboss5x</home>
      <properties>
        <cargo.jboss.configuration>default</cargo.jboss.configuration>
        <cargo.servlet.port>${it.jboss5x.port}</cargo.servlet.port>
        <cargo.rmi.port>1099</cargo.rmi.port>
        <cargo.jvmargs>-Xmx512m</cargo.jvmargs>
      </properties>
      <deployables>
        <deployable>
          <groupId>com.fja.ipl</groupId>
          <artifactId>ipl-lc-ear</artifactId>
          <type>ear</type>
        </deployable>
      </deployables>
      <!-- Override logging created by Cargo (which is way to chatty) with default 
           file from JBoss (see http://jira.codehaus.org/browse/CARGO-585) -->
      <configfiles>
        <configfile>
          <file>src/test/resources/jboss-log4j.xml</file>
          <todir>conf</todir>
        </configfile>
      </configfiles>
    </configuration>
  </configuration>

  <executions>
    <!-- before integration tests are run: start server -->
    <execution>
      <id>start-container</id>
      <phase>pre-integration-test</phase>
      <goals>
        <goal>start</goal>
      </goals>
    </execution>
    <!-- after integration tests are run: stop server -->
    <execution>
      <id>stop-container</id>
      <phase>post-integration-test</phase>
      <goals>
        <goal>stop</goal>
      </goals>
    </execution>
  </executions>
</plugin>
<properties>
  <it.jboss5x.home>${basedir}/../../../tools/bin/jboss-5.1.0.GA</it.jboss5x.home>
  <it.jboss5x.port>8080</it.jboss5x.port>
</properties>

JBoss and Maven

2009-11-19T09:47:00.012+01:00

The Challenge

During the last days I was busy trying to build and deploy a JEE application on JBoss – using Maven, of course. One interesting task was to run integration tests, i.e. JUnit tests that are testing a service bean deployed on JBoss application server. Integration tests with Maven are an interesting issue, worth its own blog entry probably. But this post is all about JBoss.

Well, this task doesn't sound hard for a Maven expert, but unexpectedly it was not that easy. Our test code (calling the EJB) was depending on JBoss artifacts, which might be questionable in itself but that's how it's currently done. Hence, during runtime the code needs the jbossall-client.jar from JBoss' client directory in the classpath.

The Eclipse Way

We are using JBoss 5.x, and in this version the jbossall-client.jar is rather small since it only references all required jar files in the Manifest's Class-Path element:

Manifest-Version: 1.0
Specification-Title: JBossAS
Specification-Version: 5.0.0.GA
...
Implementation-Vendor: JBoss.org
Implementation-Vendor-Id: http://www.jboss.org/
Class-Path: commons-logging.jar concurrent.jar ejb3-persistence.jar hi
 bernate-annotations.jar jboss-aop-client.jar jboss-appclient.jar jbos
 s-aspect-jdk50-client.jar jboss-client.jar jboss-common-core.jar jbos
 s-deployers-client-spi.jar jboss-deployers-client.jar jboss-deployers
 -core-spi.jar jboss-deployers-core.jar jboss-deployment.jar jboss-ejb
 3-common-client.jar jboss-ejb3-core-client.jar jboss-ejb3-ext-api.jar
  jboss-ejb3-proxy-clustered-client.jar jboss-ejb3-proxy-impl-client.j
 ar jboss-ejb3-proxy-spi-client.jar jboss-ejb3-security-client.jar jbo
 ss-ha-client.jar jboss-ha-legacy-client.jar jboss-iiop-client.jar jbo
 ss-integration.jar jboss-j2se.jar jboss-javaee.jar jboss-jsr77-client
 .jar jboss-logging-jdk.jar jboss-logging-log4j.jar jboss-logging-spi.
 jar jboss-main-client.jar jboss-mdr.jar jboss-messaging-client.jar jb
 oss-remoting.jar jboss-security-spi.jar jboss-serialization.jar jboss
 -srp-client.jar jboss-system-client.jar jboss-system-jmx-client.jar j
 bosscx-client.jar jbossjts-integration.jar jbossjts.jar jbosssx-as-cl
 ient.jar jbosssx-client.jar jmx-client.jar jmx-invoker-adaptor-client
 .jar jnp-client.jar slf4j-api.jar slf4j-jboss-logging.jar xmlsec.jar

This list is impressive... and the approach is working with current Eclipse. The equivalent for the Maven world would be a POM that references all other required libraries as normal dependencies (see this discussion).

The Maven Way

And yes, such a POM org.jboss.jbossas:jboss-as-client is available on JBoss Maven repository (note: it's not org.jboss.client:jbossall-client which is the reference-by-manifest's-class-path version!).

However, this approach involves two issues:

By following the dependencies defined in this org.jboss.client:jbossall-client transitively, Maven will download a vast number of JBoss and JEE libraries which you actually don't want to be used and packaged in your client. This includes things like org.jboss.jbossas:jboss-as-server:jar:5.1.0.GA and jacorb:jacorb:jar:2.3.0jboss.patch6-brew. Does not sound confidence-building, does it? Seems like JBoss should exclude transitive dependencies at the right places.
Moreover, some of the dependencies can't be found on any of the Maven repositories we configured to proxy in our Nexus. This includes Maven Central, JBoss of course, and a couple of others, so I do not know what else to add to provide the missing jars. Maybe it's just that the reference itself is wrong (version number?).

Instead of excluding everything we do not use currently, next idea is to just include those dependencies to the JBoss jar files that the client actually requires. But... it's quite hard to find out the corresponding Maven coordinates. This includes guessing the groupd and artifact id, but also the version – and unfortunately the version information given in the jar's manifest file is not useful since it notes the JBoss AS version instead of the version of the library which is required for Maven.

Lost in Space?

So, I ended up re-packaging a jar that holds the content of the required client jars and putting this on our Nexus...

I honestly wonder how everybody else is using JBoss 5.x with Maven on the client side???

Spring dm Server Considered Cool?

2009-11-13T08:02:00.004+01:00

w-jax is over...

I'm just returning from w-jax 2009, one of the biggest german Java conferences, covering every technology related to Java actually.

If there are any hypes this year, they are these: Scala, Cloud Computing and OSGi. And, of course, Spring – which is all around.

Spring Source employs a lot of brave guys, and hearing first-hand about their newest coolest technologies is always refreshing. Did you know that Tomcat is to a large extent pushed by guys paid by Spring Source? I forgot the exact number, but Eberhard Wolff said the majority of bug fixes and commits is done by Spring Source's Tomcat team.

Spring dm Server is Cool!

There have been some sessions about Spring dm Server, and when taking a closer look it seems to be clear why Spring Source invests in Tomcat that much. Spring dm server is based on Tomcat, and they add the ability to deploy web application modules as OSGi bundles. This way, you can partition your web apps into separate bundles to deploy them independantly and dynamically into the server. Yes, they are preserving OSGi's dynamic, meaning that you can stop, start or refresh a particular bundle without the need to stop and restart the server. As you would expect, other bundles continue to work. If a requested OSGi service is temporarily not available, dm Server will wait 5 minutes for it being redeployed.

Spring dm server is part of Spring Tool Suite, an Eclipse based development environment which is also able to auto-deploy a module whenever it changes.

All of this is quite gorgeous from a technical perspective, and the Spring guys had to cope with some really hard issues (for instance, realted to JPA where they need an application wide visiblity of bundles as well as load time weaving – you won't want to know the details...).

Really? What for?

But... is this really of any interest in the field?

Our customers are using WebSphere (or WebLogic or maybe JBoss) application server, and all of them are not capable of running such a modularized, OSGi based application. Moreover, in a production environment, there is no need and most often it is actually not even desired to be able to refresh application bundles dynamically.

So, what's left? Modularizing your application? Right, that is leading to a better structure and less coupling and blah blah, but the same can be achieved without OSGi (just let the business domain drive your application "slices"). It's just that you are forced to use modules (bundles) when you are using OSGi.

If at all, Spring dm server will pay off for developers since it may speed up development (especially the build-deploy-test cycle). But is this really, I mean really, hurting us that much? Hence, if you were asking me, a lot of technical overhead and server lock-in for no real benefit.

So, will Spring dm server be able to gain real attention, I mean beyond being technically cool? I'm sceptical. As always, time will tell...

Java Mystery: Directory Exists but Does Not?

2009-10-27T08:47:00.008+01:00

Well, yesterday for me was another one of those days where you think your computer must be fooling you... There is an issue with a DirectoryCleaner that works for a subproject but does not when called from the parent. But first things first.

The openArchitectureWare Part

We are using oAW (yep, the new version which is part of Eclipse Modeling Project of Eclipse Galileo). Before generating the artifacts from the model, we are executing a directory cleaner that just rubs out the folders, for instance src/generated/java. The workflow file generate.mwe looks like this:

<workflow>
   <property file="generateAll.properties"/>
   <component class='org.eclipse.emf.mwe.utils.DirectoryCleaner' directory='${srcGenDir}'/>
   <component file='.../generateAll.mwe' inheritAll="true">
      <modelFile value='...' />
   </component>
</workflow>

srcGenDir is a property that is defined in the included properties file, but that's irrelevant. The highlighted line configures the mentioned directory cleaner. The relevant code snippet is invokeInternal() method of org.eclipse.emf.mwe.utils.DirectoryCleaner which is part of Eclipse EMF frameworks:

protected void invokeInternal(final WorkflowContext model, 
                 final ProgressMonitor monitor, final Issues issues) {
   if (directory != null) {
      final StringTokenizer st = new StringTokenizer(directory, ",");
      while (st.hasMoreElements()) {
         final String dir = st.nextToken().trim();
         final File f = new File(dir);
         if (f.exists() && f.isDirectory()) {
            ... do the cleanup ...
         }
      }
   }
}

Pretty simple, right? The directory attribute contains one or more directories (comma separated), hence a tokenizer is used to get each of them and to create a file. If this file exists and it's a directory, that will be erased.

The Maven Part

Of course we are using Maven to build the stuff. The Fornax Maven plugin is configured to call the workflow during Maven build. Moreover, the mentioned workflow is part of a subproject B which belongs to an outer multi-module project A.

Now, this is where the mystery begins... When I build B (the submodule), everything is fine and works like expected. However, when I build A (the parent project), B will be built in turn and its workflow is executed, but the directory is not cleaned up! You wouldn't expect this, right?

The Strange Part

In an attempt to find out what's going on, we put some debugging code into DirectoryCleaner. It turns out that the file f returns the same value for getAbsolutePath() in both cases, but f.exists() reveals false when the build is started from the parent project – hence the condition is not met and nothing will be cleaned up.

Unfortunately, you can't look any deeper into the native code that is called when determining if a file exists. So, in a kind of trial and error approach, we found out that using the following code fixes the issue:

protected void invokeInternal(final WorkflowContext model, 
                 final ProgressMonitor monitor, final Issues issues) {
   if (directory != null) {
      final StringTokenizer st = new StringTokenizer(directory, ",");
      while (st.hasMoreElements()) {
         final String dir = st.nextToken().trim();
         final File f1 = new File(dir);
         final File f = new File(f1.getAbsolutePath());
         if (f.exists() && f.isDirectory()) {
            ... do the cleanup ...
         }
      }
   }
}

That is: by creating another file that is using the absolute path of the first one and using this further on, everything is fine in both scenarios – whether called from parent project or submodule.

To be honest: I have no explanation for these findings. Why is File behaving differently? When calculating the exist flag, the code should take into account the file's absolute path, right? So, why is creation of another file based on the absolute path is fixing the issue? Any insights are deeply appreciated...!

Speeding Up Your System

2009-10-16T13:37:00.012+02:00

We did a lot of interesting stuff lately, including upgrading Eclipse to the new Galileo release, and in turn upgrading oAW (openArchitectureware) to the new versions of Xtext, Xpand, Xcheck etc. which are now part of the Eclipse Modeling project. I will blog about all this later...

But, what really started to hurt us was the performance of our Windows XP based development laptops. They aren't really brand new ones, but not that old either. Nevertheless, they seem to have an issue with all those java, class and jar files involved when starting Eclipse, doing a "Clean Project", when using Maven to build the software etc.

IT department was not willing to provide us with new hardware (disk especially) at this time, and no, using Linux is no option either. Hence, we had to find out other areas of improvement by tuning our system.

Here is what we did to speed things up for Windows XP. Note that things might or might not be different for Windows Vista or Windows 7.

1. Disable Indexing Service

By default, there is a Microsoft "Indexing Service" running in your Windows system. According to msdn, this is "a base service for Microsoft Windows 2000 or later that extracts content from files and constructs an indexed catalog to facilitate efficient and rapid searching."

Well, to be honest, I have never heard of that service before (and rarely use the search function of Windows), but it turned out to cause lots of harddisk traffic. So we decided to disable this service, which is recommended by some people.

Actually, there are several ways to do so (wihtout using Microsoft Management Console (MMC) with an appropriate snap-in):

Disable the Indexing Service in your list of local services.
In the Properties window of your local disk, remove option "Allow Indexing Service to index this disk for fast file searching".
Remove the function via the Control Panel > Add or Remove Programs > Add/Remove Windows Components > Uncheck "Indexing Service".

2. Don't Virus-Scan Java Stuff

Our anti-virus tool was configured to scan literally everything, including java, class and jar files. This seems to be exaggerated from security point of view, but IT folks said they could not configure the anti-virus to ignore a particular set of file extensions (like *.jar).

Hence, what we did instead was to exclude two folders from being scanned on the fly: javatools (where everything Java related is put, including Eclipse releases) and projects (where all our projects reside). Instead, these folders are now scanned once a week, which is acceptable for IT guys.

Know what? That speeded up starting time of Eclipse by factor 5!

Surprisingly, excluding the Maven local repository (located in your personal settings folder) from being scanned on the fly did not make much difference, so we didn't handle this folder any special.

3. Optimize Subversion

Are you using Subversion? If so, are you using TortoiseSVN, the Windows Shell Extension for Subversion? Well, in that case you will know and probably like the little overlay icons that are used to indicate the state of files and folders. This feature is recursive, whereby overlay changes in lower level folders are propagated up through the folder hierarchy so that you don’t forget about changes you made deep in the tree.

Starting with release 1.2, a new TSVNCache program is used to maintain a cache of your working copy status, providing much faster access to this information. Not only does this prevent explorer from blocking while acquiring status, but it also makes recursive overlays workable.

This is all nice, but there is a major drawback: TSVNCache by default looks for changes on all drives and in all folders, killing disk performance with all the I/O it's doing.

You can enable a TSVNCacheWindow showing all the folders being crawled by TSVNCache. To do so, open the Registry Editor, and create a new DWORD at HKEY_CURRENT_USER\Software\TortoiseSVN\CacheTrayIcon with value of 1. After that you have to restart TSVNCache which is easiest done by just killing the process, it will be automatically restarted when you do any TortoiseSVN operation. Now there should be a small tortoise icon in Windows tray area which opens the TSVNCacheWindow. Watch how TortoiseSVN scans files and folders whenever you write to a file...

Time to fix that! That should be quite easy if you're keeping all of your working copies below one specific folder (or a small set of folders), like we do. All you have to do is to setup TortoiseSVN to only scan your sourcefolder paths:

Right-click in Explorer on any folder and select "TortoiseSVN > Settings...".
In the Settings window's tree, click on the "Icon Overlays" entry.
In the "Exclude Paths" input field, put C:\* to exclude the entire C drive. If you have more drives, exclude them all at the top level. Use newlines to separate the values.
In the "Include Paths" input field, list all of the locations where your working copies are stored, again separated by newlines.
Switch off "Network drives" option in "Drive Types" area.

All in all, your settings should now look like in the following screenshot. Thanks to Paraesthesia for this nice tip!

Epilogue

You won't believe what difference these three little tunings made to our system performance. The harddisk is not any more busy all the time, applications (like Eclipse) are starting much faster, and build time has decreased drastically. Not bad for not spending anything on new hardware! Well, next thing we will check is what effect a new (big, fast) hard disk will have... ;o)

Spring: Use Custom Namespaces!

2009-09-18T08:30:00.005+02:00

Have you ever heard of custom XML namespaces for Spring? I know you love Spring (like I do), so... probably yes. They are available since Spring 2.0 which came out October 2006, so I just can't believe we didn't use them up to now.

The Old-Fashioned Way

Well, we are building our web apps on JSF and Spring Web Flow, and additionally use a custom framework for adding a few more features like mapping between business objects and view objects based on XML mapping descriptions. Of course, this framework part is designed "IoC friendly", i.e. it's based on interfaces and standard implementations that need to be plugged by Spring configuration.

We did this by providing a basic Spring configuration that defines defaults for most elements and injects all required references into the main configuration objects. To allow the project to provide custom implementations, this code relies on particular bean names that have to be defined by the custom project. Here's a simple example for the framework's configuration:

<bean id="iplIntegrationInfo" class="...IntegrationInfo">
  <!-- myViewObjAccessor: to be defined in project specific Spring config file -->
  <property name="viewObjAccessor" ref="myViewObjAccessor" />
  <!-- myContextResolver: to be defined in project specific Spring config file -->
  <property name="contextResolver" ref="myContextResolver" />
  ...
</bean>

<!-- These can be used for aliases if standard implementation should be used -->
<bean id="defaultViewObjAccessor" class="..." />
<bean id="defaultContextResolver" class="..." />

As you can see, all references are injected into the main configuration (iplIntegrationInfo). However, the referenced beans are not actually defined but have to be provided by the project. So, here's an example for the project's configuration:

<!-- use default -->
<alias alias="myViewObjAccessor" name="defaultViewObjAccessor" />

<!-- use custom implementation -->
<bean id="myContextResolver" class="..." />

Here we use the default for the first bean (by just establishing an alias for the configuration done in the framework file), and a custom implementation for the second.

This approach works, but obviously has a few drawbacks:

Configuration is shared between framework and project specific configuration code.
Project needs to know the framework's Spring file to do its work.
Internal framework structures (bean names, classes) are exposed to the outside.
A lot of XML code is required, even if using all the defaults.

Using Custom Namespaces

What should I say, we have switched to custom namespace lately and now configuration looks like this:

<ui:view-config/>

That's as short as it can be, but still is a complete configuration in case the project is using default implementations for all the beans. If the project would like to override some settings, the configuration file may be:

<ui:view-config>
  <ui:viewobj-accessor ref="viewObjAccessor"/>
  <ui:context-resolver ref="contextResolver"/>
</ui:view-config>

<bean id="viewObjAccessor" class="..." />
<bean id="contextResolver" class="..." />

Hence, you have the option to override all default implementations, and use meaningful tags to do so. Can you see the gains?

How Does This Magic Work?

I'm not going to provide full description on how to author custom namespaces since that is available in the Spring reference documentation and elsewhere. So, just a few basics to give you the idea...

First of all, you have to provide an XML schema describing your namespace. Then you have to implement a Namespace Handler that links each tag of your namespace to an XML parser, the Bean Definition Parser. And then, of course, those parsers must be implemented.

Each bean defintion parser is supposed to parse the XML element (including all attributes and sub elements) and to build the appropriate Spring beans. Spring provides a small hierarchy of classes you can use, depending on your situation:

The most specific is AbstractSimpleBeanDefinitionParser that fits well when there is a strong correlation between the attributes on a tag and the properties on your bean. In this case, there is really few code on your side.
The AbstractSingleBeanDefinitionParser is a bit more general, it allows you to create any single bean definition for an arbitrary complex nested XML structure. This involves more coding, of course, but still does some work for you (like automatically registering the bean in the context). This is the one you'll probably use most of the time.
The AbstractBeanDefinitionParser is even more general and allows you to create multiple bean definitions and register them directly. It takes care of id generation and a few other things.
Instead of using the latter one, you could also implement the BeanDefinitionParser interface directly, giving you all the flexibility you need.

All in all, you'll need a few hours to get comfortable with all the classes that are used to build up the bean definitions (DomUtils, BeanDefinition, BeanDefinitionBuilder, RuntimeBeanReference and ManagedMap/ManagedList to name a few).

But after that, building the parser and creating/registering your beans is quite straightforward. Moreover, you always can sneak a peek on how Spring itself is doing it for the Spring provided custom namespaces (like for Spring Security, for instance).

Okay, that's all for now, so go ahead and make life easier for your customers, your team, or yourself by providing a well-designed custom Spring namespace!

Spring: Test Framework Never Heard of Web Apps?

2009-09-08T11:31:00.003+02:00

Do you know the Spring TestContext Framework? You really should. It drastically simplifies JUnit-testing of Spring powered applications. One particular cool thing is that it offers an easy, annotation based way to load your Spring context and auto-wire your beans. For example, this is how it could look like for JUnit 4:

@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration(locations = "classpath:spring-config/ui-test-config.xml")
public class MyParticularTest {
  @Autowired
  private Controller controller;
  ...
}

This is really nice, but... there is one thing we stumbled upon lately.

The test framework by default is using the GenericXmlContextLoader which loads the Spring application context as an instance of GenericApplicationContext. This class, obviously, is not aware of web applications that would require a WebApplicationContext instead. Hence, you can't access any web related resources, use associated scopes etc.

Don't get me wrong... Spring TestContext Framework is nice, but it's more than strange that it does not support the web scenario. There is a Jira issue for that, but it's still open for more than 10 months now, and is scheduled not until Spring 3.1?!? That's just annoying.

Well, there are some workarounds of course, based on using a custom context loader. See this blog post or this Spring community thread, for instance.

Highlight your Syntax

2009-09-08T09:53:00.006+02:00

Now, summer is nearly over and I'm back on this blog. During summer time, I use to spend more time with my family (and doing vacation ;-) but now my head is full with lots of snippets that could make it into this blog.

First of all, I have integrated some syntax highlighting to these pages. This might be well-known and boring for you, but I didn't really know how to do this until finding Alex Gorbatchev's SyntaxHighlighter. It's purely based on JavaScript and hence can be easily integrated with lots of products, tools and engines – like Blogger.com. Integration couldn't be easier, see here for a step-by-step explanation.

When adding code, you just have to give your <pre> element a class like this: <pre class="brush:java">. That's it!

What do you get by this? Well, the well-known, nicely formatted code blocks like this one

public void prepareMethodOverrides() throws BeanDefinitionValidationException {
   // Check that lookup methods exists.
   MethodOverrides methodOverrides = getMethodOverrides();
   if (!methodOverrides.isEmpty()) {
      for (Iterator it = methodOverrides.getOverrides().iterator(); it.hasNext(); ) {
         MethodOverride mo = (MethodOverride) it.next();
         prepareMethodOverride(mo);
      }
   }
}

This works for lots of languages (called "brushes"), including XML and JavaScript, and you can easily provide a custom brush. Standard layout can be customized, of course, but the default is pretty good IMO.

There are just two or three things (there is always room for improvements ;-)

Syntax Highlighter documentation is a bit sparse...
The server hosting the CSS and JavaScript files, http://alexgorbatchev.com, seems to be not the fastest one. To speed up things, the Syntax Highlighter files could be hosted somewhere else, and they most probably already are.
You can highlight particular lines using the <pre class="brush:java; highlight: [14, 15]"> syntax (see line 7 in example above). However, it seems you can't specify areas but have to list all individual lines which might be cumbersome.

All in all, pretty cool. Syntax Highlighter is open source under LGPL 3, but the author asks you to "buy him a beer" if you use his tool. Well, I honestly will think of some donation ;-)

Maven: How Relocated Artifacts Can Ruin Your Day

2009-06-23T15:25:00.011+02:00

Wow, this is one of those days I struggle with issues all the time without actually going forward one little bit on my original task... I guess you all know this feeling :o(

Now, one of the things that I wasted quite some time on is excluding a dependency. One of our modules is defining a dependency to Apache DBCP framework like this:

<dependency>
  <groupId>commons-dbcp</groupId>
  <artifactId>commons-dbcp</artifactId>
  <version>1.2.1</version>
</dependency>

This dependency is transitively depending on some XML artifacts, which you can see by using the maven-dependency-plugin's tree goal:

...
[INFO] +- commons-dbcp:commons-dbcp:jar:1.2.1:compile
[INFO]   +- commons-pool:commons-pool:jar:1.2:compile
[INFO]   +- xml-apis:xml-apis:jar:1.0.b2:compile
[INFO]   \- xerces:xercesImpl:jar:2.0.2:compile
...

Well, the xml-apis and xerces artifacts are a bit outdated and actually we didn't want them at all in our war files, so I decided to exclude them using the dependency exclusion feature:

<dependency>
  <groupId>commons-dbcp</groupId>
  <artifactId>commons-dbcp</artifactId>
  <version>1.2.1</version>
  <exclusions>
    <exclusion>
      <artifactId>xml-apis</artifactId>
      <groupId>xml-apis</groupId>
    </exclusion>
    <exclusion>
      <artifactId>xerces</artifactId>
      <groupId>xercesImpl</groupId>
    </exclusion>
  </exclusions>
</dependency>

What do you think will be the result? Surprisingly, xml-apis:xml-apis vanished, but xerces:xercesImpl didn't. The new dependency tree is the evidence:

...
[INFO] +- commons-dbcp:commons-dbcp:jar:1.2.1:compile
[INFO]   +- commons-pool:commons-pool:jar:1.2:compile
[INFO]   \- xerces:xercesImpl:jar:2.0.2:compile
...

That's strange, isn't it? It took me an hour or so playing around to find the reason for this behaviour: Xerces has been relocated with respect to its Maven coordinates... The only hint Maven is giving you is this message when executing in debug mode:

[DEBUG] While downloading xerces:xerces:2.0.2
  This artifact has been relocated to xerces:xercesImpl:2.0.2.

What does that mean? Well, you may relocate artifacts in the Maven repository when the group or artifact id should change (for more details, see the Guide to Relocation). Originally, the DBCP artifact specified a dependency to xerces:xerces in its dependencies – which you can see in the artifact's POM file (residing in your local Maven cache, for instance):

<dependency>
  <groupId>xerces</groupId>
  <artifactId>xerces</artifactId>
  <version>2.0.2</version>
</dependency>

However, xerces:xerces has been relocated to xerces:xercesImpl and these new coordinates are what is shown in the dependency tree. But, when excluding a dependency, you must specify the original transitive dependency (as defined by the artifact), not the new coordinates:

<dependency>
  <groupId>commons-dbcp</groupId>
  <artifactId>commons-dbcp</artifactId>
  <version>1.2.1</version>
  <exclusions>
    <exclusion>
      <artifactId>xml-apis</artifactId>
      <groupId>xml-apis</groupId>
    </exclusion>
    <exclusion>
      <artifactId>xerces</artifactId>
      <groupId>xerces</groupId>
    </exclusion>
  </exclusions>
</dependency>

How intuitive is that???

Maven should really do better, either by indicating the original coordinates when showing the dependency tree/list, or better yet by issuing a warning when trying to exclude a relocated dependency. I guess the latter one is not easy to implement, but current solution is just annoying!

Checkstyle: One and Only Configuration File?

2009-06-17T16:33:00.008+02:00

The Checkstyle Challenge

When you are using both Eclipse and Maven, you are probably facing the same challenge like we do: you would like to have a single configuration file for Checkstyle tool and reference that in Eclipse as well as in Maven.

Checkstyle ships with some default conventions (according to the Sun coding standards). But you usually want to customize this and create your own configuration set, based on your corporate coding styles.

Now, you want to setup Checkstyle within Eclipse to provide direct feedback to developers. You want to setup Checkstyle with Maven to be able to have a coprehensive report built as part of your project site. And you want both to reference the same configuration file, to avoid having to maintain two of them. Naturally, right?

Well, unfortunately, it seems impossible to achieve... if you really found a way, let me know!

However, there nonetheless are some tips to help you having a centralized configuration of Checkstyle with both tools. This post is about how to do that.

Checkstyle with Maven

The goal is to centralize the Checkstyle configuration to make its usage as simple as possible for all the projects, avoiding manual configuration as far as possible. Fortunately, this is quite easy using Maven's dependency mechanism.

Essentially, we build a custom artifact that contains nothing but our Checkstyle configuration file. This JAR is deployed to the company's Maven repository from where it can be downloaded as a normal dependency when needed. After that, the Checkstyle plugin is configured (in the reporting section) to reference the packaged configuration file. That's it!

Now, we'll look at the details...

Let's assume our checkstyle configuration is located in config/my-checkstyle.xml. Then the POM of the checkstyle artifact, which actually is a minimal Maven project, looks like this:

  ...
  <groupId>com.fja.ipl</groupId>
  <artifactId>checkstyle-config</artifactId>
  <version>1.0</version>
  <build>
    <resources>
      <resource>
        <directory>config</directory>
      </resource>
    </resources>
  </build>
  ...

(Of course, you additionally have to add the distributionManagement section to be able to deploy the artifact, but this is usually derived by the company's base POM.)

Deploy the artifact. You'll notice that the created checkstyle-config-1.0.jar indeed contains your my-checkstyle.xml file in its root.

Now let's switch to the project that should use the Checkstyle report. The configuration file is referenced in the Checkstyle plug-in configuration in the reporting section like this:

  <reporting>
    <plugins>
      <plugin>
        <artifactId>maven-checkstyle-plugin</artifactId>
        <version>2.2</version>
        <configuration>
          <configLocation>my-checkstyle.xml</configLocation>
        </configuration>
      </plugin>
      ...
    </plugins>
  </reporting>

That is not yet working, Maven can't find this file – our checkstyle-config artifact has not been referenced. We do this now by adding it as an extension dependency to the POM as follows:

  <build>
    <extensions>
      <extension>
        <groupId>com.fja.ipl</groupId>
        <artifactId>checkstyle-config</artifactId>
        <version>1.0</version>
      </extension>
    </extensions>
    ...
  </build>

The artifacts configured as extensions will be included in the running build's classpath. That means, checkstyle-config-1.0.jar now is in the classpath of the Checkstyle plugin, and the file referenced by the configLocation setting is located in the JAR and hence can be found.

Using this approach, the Checkstyle configuration can be maintained centrally, and projects can get new versions simply by updating the artifact's version in the extension section.

Checkstyle with Eclipse

Unfortunately, I don't really see a way to use the same Checkstyle configuration, packaged into the Maven artifact, with Eclipse as well...

So you'll have to duplicate the configuration file. However, to keep the number of duplicates small, I recommend to put it on a central network drive within your company and use the Remote Configuration option of the Checkstyle Eclispe plugin to reference the remote file. This does not prevent you from working offline because Eclipse downloads and caches this file.

In this scenario, changing the Checkstyle configuration is done centrally by just updating the remote file. This is a great advantage over using a configuration that is part of the project, because in the latter case you would have to upgrade every single project.

Google and the Crystal Ball

2009-06-04T10:18:00.006+02:00

Google brought us the Web Search. They brought us the Maps. They brought us their Mail, the News, the Images, the Videos... In other words, they revolutionized the web. They rule the web. You all know that.

They are so creative, cutting-edge and are always good for some surprise. Like this one: Google has announced the Wave project a few days ago, which indeed might have the power to turn into the "next generation of e-mail". Wave is a web based service platform designed to merge e-mail, instant messaging, wiki, and social networking. Gosh, this sounds exciting, really!

Yeah, Google is collecting data, probably more than we can imagine. And they will continue to do so. (A new study of UC Berkeley just revealed that Google is the dominant player in the tracking market; typically, tracking is done using "web bugs" that are embedded in the web page‘s HTML code, and are designed to enable monitoring of who is reading the page.)

But... aside from all discussions how evil or risky Google's data acquisitiveness might be, it opens a new world for something called "collective intelligence": discovering metadata based on the search engine data.

What does this mean? Well, Google records search phrases which can later be compared across specific regions, categories and time frames. Using Google's Insight for Search service, you can "see what the world is searching for". Pretty nice. For instance, search for "google wave" and see how the interest emerged end of May. Interestingly enough, there have already been a few searches in summer 2008!?!

Well, it even gets better. Google has found out that certain search terms used in their web search are good indicators of flu activity – when people have the flu, they search for symptoms, drugs etc. Google uses that to estimate flu activity on their Flu Trends page, and this indicator turns out to be quite good.

So, what is next? Possibilities seem endless, and Google will certainly surprise us again. Since they have all the data, and the (near) future is certainly hidden in that data – how about some kind of digital, web based crystal ball? I told you, so don't be surprised if they make it... ;o)

Eclipse Top Annoyances

2009-06-03T12:22:00.004+02:00

Eclipse is one of those love/hate things. I love it and hate it, both at the same time.

The Love

Firstly, it's the best Java IDE on the market, and we successfully use it for years now. In the meantime, there might be other IDEs with an even richer feature set, better integration, nicer UI etc. but we actually don't care very much. I love Eclipse just for what it did to the Java and Open Source community, and it definitely has been pushing the edge for a couple of years now.

BTW, of course I know Eclipse is more than just a Java IDE, but this is my personal short interpretation of "development platform comprised of extensible frameworks, tools and runtimes for building, deploying and managing software across the lifecycle".

The Hate

Well, having said all this... sometimes Eclipse really drives me crazy. There are a couple of annoyances, some of them rather small but insistent, others bothering us since quite a long time.

The strange thing is, all of my complaints are not related to the more sophisticated plugins like Mylyn, WTP, MDT and so forth. (Yes, we use them too, albeit in a sketchy way.) No, the issues annoying me most are shown by the Platform or JDT, and maybe that's why they are actually so annoying.

The List

Okay, here is my list for Eclipse 3.4 (Ganymede):

6. Formatting

We have setup the Eclipse Java code formatter to match the project requirements. There are plenty of options to adjust to your needs, but... sometimes the algorithm is just broken. Take this example:

private Accumulator getSpecificAccumAndSetRuleDefaultValue(
                                                            CoverageRule calcStepRule,
                                                            CalculationStepInfo calculationStepInfo,
                                                            ...)
{
   ...
}

What is this extra line break before the first method argument good for? Instead of aligning the method arguments horizontally, Eclipse messed it up vertically. You can find several other strange formattings where formatted code requires extra space, mainly in long method or class declarations. Here is another example of an extra line break:

public class PersistenceService<R extends PersistentRootObject, D extends HibernateDaoSupport>
      implements
         IPersistenceService {
  ...
}

5. Blocking Workspace

I have blogged about this blocking behaviour before: sometimes, Eclipse can't save my file while it is building "in the background" and tells me that my "User Operation is Waiting"...

That's just silly, Eclipse should be able to do my save action at any time, if compiling or not. Loosing editor's content (by unintended editing, Eclipse crash or hang-up, whatever) just due to the impossibility to save is just not acceptable.

4. Startup Time

Depending on size of the workspace, you got to wait minutes before you can start editing. That's just too slow. Period.

3. Dialog Window Dimensions

When you open a project's Properties window, for instance, it will show up quite small. I have a large screen and don't like to scroll unnecessarily, so I resize the window to be considerably larger. However, the dialog dimensions are not persisted – when I close and reopen the same dialog, it shows up as small as before. Eclipse just refuses to remember what I like.

For other dialog windows (like Open Type), this works perfectly; why not for all of them?

2. Update Manager

I used to think the new Eclipse 3.4 update manager (p2 Provisioning System) was a good thing, but I'm no longer sure. As end user, you just don't want to mess around with features, dependencies, build numbers... and get strange error messages (see this post of an issue with a custom built Eclipse plugin we struggled with recently).

Additionally, the list of available software is too technical for most end users. Subversive SVN Connectors plugin provides 6 optional connectors, but at least one is required. So, which one(s) do I choose? Instead of having the full list with just the name and version strings, I would rather like to see some typical sets of required features ("typical installation" similary to when installing software on Windows). Alternatively, at least provide some end user level meta information like "This is the default feature", "Choose this item if..." etc.

1. Vanished Project Contents

Sometimes the content of my workspace projects in Package Explorer view is "gone", meaning I can't expand the project nodes any more. Yes, of course there is some content, and fortunately it is still on the disk – but just not showing up in the view.

Project Explorer view still shows the projects correctly. Closing and re-opening them in Package Explorer usually helps.

The Hope

I used to install the latest milestones for some years, but have given up this habit because of stability/quality/compatibility issues with Eclipse core and/or the plugins I use. Hence, I can't tell which ones will be addressed by the Eclipse 3.5 (Galileo) which is going to happen on 24th of June.

So... let's keep our fingers crossed!

Maven Plugin Releases: Do it yourself!

2009-05-18T13:47:00.004+02:00

In my previous post, I have complained about Maven plugins that do not release new versions although there are blocking issues that are reported in Jira with patches attached to them. If you don't want to wait any longer for a new plugin release, just build one on your own. Here's how we usually do that.

Fix the Plugin

Checkout plugin sources from source repository and import as Eclipse project. The easiest way to do this is to use m2eclipse plugin's cool "Materialize Maven Projects" feature (see here).
Edit the POM and change the plugin's version number (see notes below).
Download the patch from the JIRA issue page.
Appy the patch using Eclipse: in project's context menu, select Team > Apply Patch...
Now build, unit-test and install the plugin using Maven.
Update your project's POM to use this new version, and test, test, test....
If everything is fine, upload your patched plugin to your repository manager to make it available to your team members.
Check in all changes you made, probably including the fixed plugin – into your repository, of course, not the plugin's one!

Give Some Thought to Version Numbers

A few rules on the fixed plugin's version string that have proven rational for us:

You should not just increase the version number to prevent mix-up with future official versions of the plugin.
Instead, we usually add a suffix made up of our company name and a consecutive number, like "-fja-1" for the first fix. This indicates the initiator and gives room for further fixes in case they should be required. Additionally, you always see the underlying official version. Example: <version>2.5-fja-1</version> for FJA's first fix of version 2.5.
Don't change the plugin's group or artifact id because this would bypass Maven's version resolution, i.e. in Maven's eyes your fix would not just be a new version, but another plugin. You possibly end up having both plugins (the original one and your fix) configured in your builds, leading to duplicate execution and unexpected results.
If you just change the version string, this can be done in the parent POM – you are locking down your plugins in your corporate/project root POM, do you? – making it easy to switch back to an official relase later... in case there is one available.

Maven Compromised by Plugins

2009-05-13T09:31:00.005+02:00

Every piece of software has its flaws... The important part is how the project is dealing with bugs.

Maven is fine

With Maven, the situation actually is good enough: there have been releases of 2.0.x branch 1-2 times per year, to fix bugs and provide new features or improvements. At the same time, the 2.1.0 branch appeared with a milestone and a final release six month later.

However, you would usually not want to switch your Maven build system to a new version too often. This is always risky since a new version of the "core" may cause unexpected results in any area of the build. A continuous integration engine helps to detect bugs, but you never know.

Fortunately, Maven is heavily based on plugins – which means if you encounter any issue with a specific Maven task, it's usually caused by the related plugin, not the Maven core. That's great, right? You just upgrade your POMs to use a new version of the particular plugin. Better isolation, fewer risks!

(Some) Plugins are not!

But... this assumes that there is a new version of the plugin available. Now imagine this: a plugin has a bug reported by the community, and there is a Jira issue for that. Some brave soul even attached a patch to fix the issue and others confirmed that it works for them. Thus, the new bugfix release could easily be built. But what happens? No new release. Nothing. Not even an alpha/beta version. Not even a comment explaining why. For months!

Here are just two examples we stumbled upon the last couple of weeks:

Javadoc plugin: The report is not generated at all for multi-module project if run from parent level and using the recommended aggregate goal. Jira issue reported 2008/08/18, patch added 2008/09/22. No new release ever since.

Checkstyle plugin: Current version does not work with Checkstyle 5.0 that started to appear on the stage last summer. Jira issue created 2008/10/28, along with a patch; another patch added 2009/03/27. No new release ever since.

Wow. How annoying is that? There is a major/blocking issue, and there is a patch available, but the plugin owners don't take the time or effort to create a new release?!? Why so? What else could the community do to help fixing the plugin?

This way, the plugins are compromising usability of Maven. IMHO, that's a serious issue...

So, what can you do?

If you don't want to continue waiting for a new plugin release, you have to build one on your own, for your "private use". That's quite easy, fortunately, and I'll probably post about that later.

However, this is actually not what a typical Maven user wants to do. The plugins should just do it right: release more often to fix their known bugs! You could increase the pressure a little bit by voting for the Jira issue or adding some comments, but my experiences are not too good ...

Update to Checkstyle 5.0

2009-05-11T16:12:00.003+02:00

Checkstyle 5 is available!

This is good news: a new version of Checkstyle is available that better supports Java 5 language features (like generics, annotations or package-info files). Additionally, some checks are cleaned up a bit, for instance concerning their parent in which they are contained. See the release notes for details.

Due to these changes, Checkstyle 5.0 is not fully compatible to previous versions 4.x – which is already indicated by the version number leap. Hence, don't expect the upgrade to be smoothly!

Nevertheless, I thought it would be time to upgrade, so here is what I did...

Step 1: Upgrade checkfile configuration

Due to the incompatabilities, I had to apply some changes to our checkstyle configuration to make it work with Checkstyle 5. To be sure I did it right, I have downloaded and used the Checkstyle binaries and checked my configuration for just a simple project. Hence, before upgrading Eclipse and Maven Checkstyle integration, I know my configuration is correct.

BTW, we are using a common checkstyle configuration file for all projects, and reference this "global" file with Maven and Eclipse in different ways:

With Maven, we use a custom artifact that contains nothing but our checkstyle.xml file. This in turn is included as build extension in our main base POM. See this post for more details.

For Eclipse, we use the "Eclipse Checkstyle Plugin" that provides a Remote Configuration option to reference the configuration file on an internal file server.

Step 2: Upgrade Eclipse Plugin

There is a new Beta-Release of Eclipse Checkstyle plugin (eclipse-cs) available on its update site http://eclipse-cs.sf.net/update. At the time of writing, there are three features available:

Eclipse Checkstyle Plug-in – version 5.0.0-beta4
Eclipse Checkstyle Plug-in 4.4.x -> 5.0.0 Migration (Optional) – version 5.0.0-beta4
m2eclipse Maven Synchronization Plugin (Optional/Experimental) – version 0.0.3

Oh, seems there would have been some help in migrating from Checkstyle 4.4 to 5.0... Doesn't matter, I always like to see what the changes are so it's good to do it manually.

The m2eclipse Synchronization plugin is a new feature providing mechanism to synchronize Checkstyle rules and configuration between the maven plugin and eclipse-cs. Sounds really interesting... but let's do one step after the other and test this later.

So. I just installed the "Eclipse Checkstyle Plug-in" feature. Eclipse didn't recognize that this actually is an update, so you have to uninstall the previous eclipse-cs installation manually.

Why that? Well, the "package" has changed from com.atlassw.tools.* to net.sf.eclipsecs.*, and this applies to the feature's ID, too. Moreover, this renaming also affects the buildCommand and nature in .project files, they have to be net.sf.eclipsecs.core.CheckstyleBuilder and net.sf.eclipsecs.core.CheckstyleNature now.

Additionally, the notation for file sets has been changed: a file set previously configured as src\\main\\java\\com\\mycompany\\.* does no more match to any file; instead, it has to be the slash now like in src/main/java/com/mycompany/.*.

Okay, maybe I should have tried the Migration plugin... Anyways, after these changes everything works fine for me in Eclipse.

Step 3: Upgrade Maven Plugin

Good. Last piece is Maven, which provides the Maven 2 Checkstyle Plugin. However, the current version is 2.2 which is based on Checkstyle 4.4 by defining these dependencies:

<dependency>
  <groupId>checkstyle</groupId>
  <artifactId>checkstyle</artifactId>
  <version>4.4</version>
</dependency>
<dependency>
  <groupId>checkstyle</groupId>
  <artifactId>checkstyle-optional</artifactId>
  <version>4.4</version>
</dependency>

There is an interesting way to override the plugin's dependencies pointed out by Brian Fox, but that's not going to work for us because Checkstyle versions 4.4 and 5.0 are not API compatible.

What can we do? Not much... we'll have to wait for a new version of the Checkstyle plugin that updates to Checkstyle 5. There is this Jira issue, and patches have already been provided some time ago. It's only that there seems to be no progress whatsoever... Checkstyle 5.0 is officially out since April 18th, so there is no reason to wait any longer! Create a new release (for my part, alpha/beta is fine as well) – please!!!

If you really need the plugin to be fixed now, you could checkout the plugin's sources and built your own version, applying the patch provided in the Jira issue. That works, but is nothing we want to do regularly!

Eclipse: Update Manager Fools Me

2009-05-05T13:46:00.000+02:00

Recently, we experienced an issue with a custom Eclipse plugin developed and hosted by my company. When trying to install the plugin into various versions of Eclipse Ganymede, the update manager always told me "Cannot find a solution satisfying the following requirements org.eclipse.ui [3.4.2.M20090204-0800]".

Finally, after some lost days of searching and trying, we discovered that this message is totally misleading: the issue was not caused by the feature org.eclipse.ui missing or being available only with wrong version. Instead, the reason was a misconfigured Manifest file of the custom plugin. It just defined wrong version for the required bundles (oAW, the MDA/MDD generator framework we use).

How nasty is that? Couldn't Eclipse update manager (p2) provide better support? More useful messages? Gosh!

Moreover, Eclipse update manager seems to cache the metadata of remote repositories. That might be nice to avoid network overhead of loading the repository again and again. But... when testing installation of different plugin versions with a local update site, it definitely gets into your way. The sad thing is, you can't get rid of the cached contents easily. You'd have to remove the site, restart Eclipse with -clean option two times, and after that recreate the location. Could be made easier, really.

Eclipse update manager will once more be refactored in upcoming Eclipse 3.5 Galileo. Folks, don't mess it up again!

Maven Enforcer Plugin: cool and annoying

2009-05-04T12:11:00.002+02:00

As a Maven expert, you probably have heard of maven-enforcer-plugin, "The Loving Iron Fist of Maven", as they say.

It's so cool!

The idea is pretty cool: the plugin provides goals to ensure that the environment used to run Maven is like you – the POM author – expected it to be. There are checks for constraints on the version of Maven or JDK, on the OS family, to enforce certain dependency requirements, and much more. This list on the web site gives you an impression on what kind of checks are provided.

Do you know what happens when you write your POMs and settings for Maven version 2.0.9 or greater, but somebody is still using an outdated 2.0.4? Well, this kind of issues is annoying and you better make sure it does not happen.

The current version of maven-enforcer-plugin (at the time we set it up) was 1.0-alpha-4 which sounded not very mature, but nevertheless we decided to use it. For our projects, we identified a couple of things we wanted to check, and we configured the plugin accordingly:

Java Version: should be 1.5.
Maven version: must at least be 2.1.0.
Enforce defined versions for all plugins, and disallow any use of "LATEST", "RELEASE" or "SNAPSHOT" as a version for any plugin.

Hey, SNAPSHOT plugins are just dangerous and you should never, never ever depend on them. The only exclusion could be plugins developed inhouse, but actually, no – not really. Bah, don't use SNAPSHOT plugins. Period.

So, we tried to setup our enforcer plugin configuration like this:

<plugin>
  <artifactId>maven-enforcer-plugin</artifactId>
  <version>1.0-alpha-4</version>
  <executions>
    <execution>
      <id>enforce-versions</id>
      <goals>
        <goal>enforce</goal>
      </goals>
      <configuration>
        <rules>
          <requireMavenVersion>
            <version>[2.1.0,)</version>
          </requireMavenVersion>
          <requireJavaVersion>
            <version>1.5</version>
          </requireJavaVersion>
          <requirePluginVersions>
            <message>Best Practice is to always define plugin versions!</message>
            <banLatest>true</banLatest>
            <banRelease>true</banRelease>
            <banSnapshots>true</banSnapshots>
          </requirePluginVersions>
        </rules>
      </configuration>
    </execution>
  </executions>
</plugin>

Of course, we wanted to apply the same checks on command line as within Eclipse when using m2eclipse plugin.

It's so annoying!

Well, and here is where the trouble started...

Shot one. Oh, wait, we are using Maven version 2.1.0-M1 currently. Guess what? This is not included in range [2.1.0,) which means every version x >= 2.1.0. Okay. Let's use [2.1.0-M1,) instead.

Shot two. What about Eclipse? The check fails within Eclipse, and it turns out that the version number reported by the Maven embedder used by m2eclipse (0.96 at that time) is 2.1-SNAPSHOT. Thus, enforcer fails and we'll have to use this for the Maven version specification:

<version>[2.1.0-M1,),[2.1-SNAPSHOT,)</version>

Ugly, but still manageable...

Shot three. My team reported the enforcer plugin to sometimes use some "thinking time" when checking the environment – it seems to hang for couple of seconds. We all see that betimes, and I have no idea what it is caused by. Actually, performing those version checks should be fast as light, right?

What makes this even worse is the fact that in multi-module builds, enforcer plugin is called for each subproject. It binds by default to the validate lifecycle phase which is executed for every subproject. However, the environment does not change very much during a single build, so it would be sufficient to check that only once per Maven call...

Shot four. When establishing the cool Sonar quality management platform, Maven started to complain that "some plugins are missing valid versions". Seems that Sonar defines dependencies to other artifacts used internally, but doesn't give a version for them. All you can do is specify a version for those plugins in the <pluginManagement> section of one of your parent POMs. That's actually not what you want: explicitely list versions for artifacts used internally by the current version of a build tool... But what else can you do? Completely disable the requirePluginVersions check?

Shot five. After upgrading m2eclipse to version 0.97, it does not correctly handle our enforcer configuration any more and instead yields an error:

org.apache.maven.lifecycle.LifecycleExecutionException: Invalid or missing parameters: [Mojo parameter [name: 'rules'; alias: 'null']] for mojo: org.apache.maven.plugins:maven-enforcer-plugin:1.0-alpha-4:enforce

This is a known issue, and you have to uncheck the "Skip Maven compiler plugin when processing resources" checkbox in the project's properties. That means, we have to change configuration for all our projects :-(

Moreover, it was the default to check this option and it seems to have negative performance impact when deselected.

Game over. This is the point where we eliminated enforcer plugin. Too much pain for a little helper tool. One of those things that seems to be quite cool at first glance, but starts to annoy you very soon.

Maven Documentation: The Missing List

2009-04-29T13:43:00.004+02:00

A rather weak talent of Maven is probably its documentation. This is my personal opinion, but it seem to match what other people think. Yeah, it has been gotten better, but there is still some room...

It's not that there is not enough documentation available, it's just that the existing documentation is poorly organized and you often can't find what you need. Moreover, many documents (especially for plugins) are only scratching the surface, not even diving into the important shallows. Or, even worse, they give you wrong or inconsistent information...

Here is just one example: The surefire plugin page for surefire:test goal gives this description for the excludes parameter: "List of patterns (separated by commas) used to specify the tests that should be excluded in testing. (...)". However, the example page for "Inclusions and Exclusions of Tests" shows an example that uses nested elements – wait, what about the commas? So, what do you do? Go ahead and try yourself? I bet this type of annoyance is hitting every Maven newbie sooner or later.

Well, maybe the Maven team should start a coordinated, collaborative effort to improve documentation just like Wikipedia is doing once in a while. Or, better yet, why not move all documentation into some open Wiki and let the community work on what it thinks is missing, broken, inconsistent, badly organized, ...

Well, until then, we have to use what is available. This post tries to list most important pieces of online documentation, indispensable when you are dealing with Maven in a professional way. Let me know if you think something is missing on this list...

Introductory & General

Main Site for Apache Maven project
Maven Getting Started Guide – intended as a reference for those working with Maven for the first time
The Maven 2 tutorial – a practical guide for Maven 2 users from Codehaus
offical FAQ and unofficial FAQ
Sonatype's Summary of Maven How-Tos
List of Web Tutorials (How-To's)

Books

Maven: The Definitive Guide – a free book by Sonatype (using a Creative Commons license). High quality, up-to-date, written by some of the famous Maven gurus: Tim O'Brien, John Casey, Brian Fox, Bruce Snyder and Jason Van Zyl.
Better Builds with Maven – another free book, originally maintained by Mergere, now managed by MaestroDev. This, too, is written by core members of the Apache Maven Project: Vincent Massol, Jason van Zyl, Brett Porter, et al.

Technical Details

Maven plugins

Maven Plugins – lists the core plugins (supported by the Maven projects) and others
Mojo Project Plugins – hosted at Codehaus repository
Cargo Maven Plugin – used to start/stop J2EE containers and deploy to them
Docbkx Maven Plugin – hard to find, but useful for generating HTML/PDF from DocBook

The POM

Other important references

Directory Structure – description of the standard directory layout
Build Lifecycle – lifecycle basiscs and bindings
Maven Properties – a list of properties available in Maven 2

Community

Maven User List, hosted at MarkMail
Maven Developer List, hosted at MarkMail
Maven Project Issue Tracking (Jira)

Updates

2009/05/11: added link to Sonatype's "Summary of Maven How-Tos" blog post – definitely worth to be listed here...
2009/05/28: added link to Maven Properties Guide wiki page

Spring Web Flow: watch out!

2009-04-27T11:23:00.000+02:00

Starting in November 2008, we found a concurrency issue in our web applications that are based on the following framework stack:

Spring 2.5.4
Spring Web Flow 2.0.5
JSF 1.2_09
Facelets 1.1.14a
Trinidad 1.2.9

The issue

For a single user, everything run fine. However, when multiple users hit the application, we got some strange exceptions occasionally – not always and not always at the same place. However, we managed to get this one regularly with our JMeter tests:

SCHWERWIEGEND: Error Rendering View[/plan/FindPlan.xhtml]
javax.el.PropertyNotFoundException: /file:/.../facelets/ipl-comboBox.xhtml @64,30 value="#{localField.jsfValue}": Target Unreachable, identifier 'localField' resolved to null
at com.sun.facelets.el.TagValueExpression.getType(Tag ValueExpression.java:62)
at org.apache.myfaces.trinidadinternal.renderkit.core.xhtml.ValueRenderer.getDefaultConverter(ValueRend erer.java:78)
at org.apache.myfaces.trinidadinternal.renderkit.core.xhtml.SimpleSelectOneRenderer.renderNonElementCon tent(SimpleSelectOneRenderer.java:323)
at org.apache.myfaces.trinidadinternal.renderkit.core.xhtml.FormElementRenderer.encodeAllAsNonElement(F ormElementRenderer.java:172)
at org.apache.myfaces.trinidadinternal.renderkit.core.xhtml.FormElementRenderer.encodeAll(FormElementRe nderer.java:105)
at org.apache.myfaces.trinidad.render.CoreRenderer.delegateRenderer(CoreRenderer.java:335)
....

This exception got rendered on the page instead of the actual component. At other times, we got java.lang.IllegalStateException and other funny behavior.

The strange thing is, the referenced variable is put into the flow scope, of course. Now that we took the time to debug into this issue, we can see that the scopes seem to be correctly filled and the property is actually there, it's just that the ELResolvers could not retrieve it...

Actually, we thought it would be caused by our Java or Spring configuration code, presumably using wrong bean scope or the like.

...is fixed!

Guess what... now I have been pointed to this bug of Spring Web Flow version 2.0.5: Class SimpleELContext is not thread-safe, but is stored in some application singletons :-( See this or this thread for more details.

The good news is, the bug is fixed in version 2.0.6; we have upgraded our projects and load tested them, and everything is just fine.

But...

IMO this is leaving a stale aftertaste. I'll tell you why.

That issue is a severe one: web applications using SWF 2.0.5 are just not working for concurrent users. Everybody using SWF in this version (maybe earlier versions as well) is affected. Given that, it seems strange that

the issue hasn't been reported and fixed earlier: Version 2.0.5 has been published on 14th of November, and it took more than 2 weeks for the Jira issue to show up.
this issue is hard to find in the web: Of course, we did some web research to find out if someone else might have a similar problem, but we did not find very much useful. The mentioned threads have started in February/March 2009 – did nobody suffer from this bug earlier than that? Why did nobody post this issue anywhere before?
the fact that this version is buggy hasn't been reported on Spring Source web pages: Since it's rather threatening, why not put an eye-catcher message on the download page (or somewhere else) telling that this version is buggy and should not be used?

I can see two reasons for this situation, and actually I don't like both: on one hand, there seems to be no load testing by Spring Web Flow team before doing a release; on the other hand, this version 2.0.5 is probably not used very much for real-world web applications by the community out there, and I honestly hope that this is not true to SWF as a whole...

Eclipse: User Operation is Waiting, and Waiting, ...

2009-04-24T09:18:00.000+02:00

I am using Eclipse since quite a long time, sometimes around 2002. That was version 2.0, if I remember correctly. Since then, I have always upgraded to the latest version, if not milestone build. Most of the time I was quite happy with the stability and really appreciated all the evolving features and UI improvements.

One major improvement (it was in 3.0, right?) was when they moved builds to a background process, enabling you to continue working when the class or project needed to be compiled.

But... what is this? In all the latest release versions (3.4.0 ... 3.4.2) I sometimes get this dialog when trying to save a file:

Eclipse can't save my file while it is building "in the background"? Come on! Nota bene, I am not talking about .project files or other stuff the whole build could depend upon, it's also true for simple resources or even documentation files!

That's really annoying. This is a modal dialog, which means I have to wait for minutes until build is complete or at least in a state where Eclipse thinks it can save the file. Of course, the user interface is blocked during that time... unless you cancel the save task.

I use the Eclipse Java EE package and additionally have installed a couple of regular plugins, like m2eclipse and oAW. So I can't tell for sure who is actually causing this mess, but on the other hand my installation is probably not so unusual and I'm sure others get this issue as well. There are some entries in various issue databases telling me that I'm right.

Eclipse is great and I (still) love it – but our relationship really gets poisoned by this kind of things... Maybe I try IntelliJ IDEA or NetBeans one day... I have warned you!

Maven Setting for Using a Single Repo Manager

2009-04-22T11:43:00.000+02:00

In a previous post I have tried to explain why it's a good idea to define your Maven repository in your settings.xml file instead of the POM.

Of course, there is some information available on how to do this, for instance in the Maven Settings Reference, the Mirrors Guide, or the great Nexus book from Sonatype.

Nevertheless, this setup of repositories, proxies and mirrors can be a bit tricky (and is quite confusing when just starting with Maven), so here is how we did that.

How should it look like?

We use the Nexus repository manager and especially do like the repository groups which combine repositories into single logical repo. (This feature is also implemented by other products like Archiva or Artifactory, see this Feature Matrix).

Now, here is the approach we want to implement:

Access the repository group "internal" for release and snapshot versions of all internal artifacts that have been produced by our company
Use the "public" repository group to provide release versions of all external artifacts (we don't want external snapshots!)
For downloading plugins, use both the "public" as well as the "internal" repository group to get release versions (we don't want to use snaphot version of plugins!)
Don't use any other repository than "public" or "internal", no matter what is configured in project's POM, inherited POM, transitive dependencies, dependencies of plugins, etc. -- never ever!

Defining the Repositories

This is the required repository definition:

We use a profile (that is activated by default) to group the repositories. This is actually not required, but gives a bit more flexibility.
For artifact resolution, there are two repositories with id "central" and "internal" to provide public (external) respectively internal artifacts.
The definition for "central" repository is overriding (by using the same id) the "central" repository already defined in the implicit super POM, which is using repo1.maven.org for downloads.
The given URLs are only bogus and are overridden by mirror settings – see below.
Additionally, we define the same two repositories "central" and "internal" as plugin repositories (required by Maven to find plugins). However, in contrast to artifacts, snapshots are not allowed at all.

And here is how the code looks like:

<profiles>
  <profile>
    <id>repo-config</id>
    <activation>
      <activeByDefault>true</activeByDefault>
    </activation>

    <repositories>
      <!-- repo "central": override Maven default,
           mirrored to Nexus group "public" -->
      <repository>
        <id>central</id>
        <url>http://central</url>
        <releases>
          <enabled>true</enabled>
          <updatePolicy>never</updatePolicy>
          <checksumPolicy>ignore</checksumPolicy>
        </releases>
        <snapshots>
          <enabled>false</enabled>
        </snapshots>
      </repository>
      <!-- repo "internal": provides internal release and snapshot artifacts,
           mirrored to Nexus group "internal" -->
      <repository>
        <id>internal</id>
        <url>http://internal</url>
        <releases>
          <enabled>true</enabled>
          <updatePolicy>never</updatePolicy>
          <checksumPolicy>ignore</checksumPolicy>
        </releases>
        <snapshots>
          <enabled>true</enabled>
          <updatePolicy>always</updatePolicy>
          <checksumPolicy>ignore</checksumPolicy>
        </snapshots>
      </repository>
    </repositories>

    <pluginRepositories>
      <!-- plugin-repo "central": override Maven default,
           mirrored to Nexus group "public" -->
      <pluginRepository>
        <id>central</id>
        <url>http://central</url>
        <releases>
          <enabled>true</enabled>
          <updatePolicy>never</updatePolicy>
          <checksumPolicy>ignore</checksumPolicy>
        </releases>
        <snapshots>
          <enabled>false</enabled>
        </snapshots>
      </pluginRepository>
      <!-- plugin-repo "internal": provides internal plugin releases,
           mirrored to Nexus group "internal" -->
      <pluginRepository>
        <id>internal</id>
        <url>http://internal</url>
        <releases>
          <enabled>true</enabled>
          <updatePolicy>never</updatePolicy>
          <checksumPolicy>ignore</checksumPolicy>
        </releases>
        <snapshots>
          <enabled>false</enabled>
        </snapshots>
      </pluginRepository>
    </pluginRepositories>

  </profile>
</profiles>

Defining the Mirrors

After defining the repositories, two issues are left: we have to specify the correct URLs, and to make sure nobody uses any other repository. Both of this is done with the <mirrors> definition.

Repository with id "internal" is mapped to the URL of the "internal" Nexus group; that is, Maven uses that URL instead of what is given with the repository definition (the bogus URL http://internal).
Every other repository (specified by using <mirrorOf>*</mirrorOf>) is mapped to the URL of the "public" Nexus group.
This last definition not only maps the "central" repository to the correct URL, but also any other repository and hence ensures every request is locked down to the internal repository manager.

Again, here is the section from settings.xml file:

<mirrors>
  <mirror>
    <id>nexus-internal</id>
    <name>Nexus internal repository group</name>
    <url>http://our-nexus-server:8081/nexus/content/groups/internal</url>
    <mirrorOf>internal</mirrorOf>
  </mirror>
  <mirror>
    <id>nexus-public</id>
    <name>Nexus public repository group</name>
    <url>http://our-nexus-server:8081/nexus/content/groups/public</url>
    <mirrorOf>*</mirrorOf>
  </mirror>
</mirrors>

What about the Proxy?

The <proxy> section in the settings.xml file can be used to define a network proxy that is used for some or all of your HTTP requests. In our configuration, since the Nexus repository manager is running on an internal server and Maven is configured to not connect to any other server, we just don't need this setting.

See Configuring a proxy for more details on Maven Proxies. BTW, it should now be very clear that mirrors in the Maven world are something completely different than proxies...

Maven Repositories: define in POM or settings?

2009-03-27T10:32:00.000+01:00

If you are using Maven for more than just playing, you certainly have a repository Manager installed to both proxy artifacts downloaded from public repositories, and host your own artifacts to make them available to other team members and teams. (If you really really don't have one yet, consider using Nexus, an open source Maven repository manager created by Sonatype.)

Well, so you use a repository manager. Now, you need to tell Maven to use it to download all the missing artifacts. Moreover, as an organization, you usually want to control where the artifacts are downloaded from. This means you need to make sure that all developers are using the identical set of repositories for all the projects.

There are two placed you could use to configure your repositories: in the project's POM, or in the <settings> element on the settings.xml file. This post will discuss both ways and tell you which one you should use ;-)

The POM

The "innocent way" is to add a definition like this to your POM:

<repositories>
  <repository>
    <id>internal</id>
    <name>Company internal repo</name>
    <url>http://your.company.org/nexus/content/groups/public</url>
    <releases>
      <enabled>true</enabled>
      <updatePolicy>always</updatePolicy>
      <checksumPolicy>warn</checksumPolicy>
    </releases>
    <snapshots>
      <enabled>false</enabled>
    </snapshots>
  </repository>
</repositories>

Maven uses all declared repositories to find missing artifacts. If it can't find what it's looking for, Maven will also fall back to the repository central which is defined in the built-in parent POM. However, this is usually not what you want; instead, all artifacts should be proxied by your repository manager.

This can be prevented by "overloading" the central repository with your own repository manager, i.e. you just add a definition with <id>central</id>.

When using a central repository manager, the definition of repositories should be the same for all your projects. This is usually done by putting this stuff to a company's base POM. But... if someone is starting in a clean environment, Maven will have to know where the repository is to find the project's parent POM, where it will find where the repository is... you're stuck.

Moreover, any POM that is included by transitive dependencies may specifiy additional repositories which are not redirected like the central repo. These external repositories are used by Maven to find all dependencies, even your internal ones that for sure are not hosted there; and you still do not really control where the artifacts are coming from.

Hence, using POM to define your repositories does not really solve any problem. Just don't do that!

The Settings

So, we end up with the alternative and recommended way of defining your repositories: the settings.

To really make sure that all developers and all projects are using the identical set of repositories, you should use mirrors to tell Maven to redirect all artifact requests to your internal repository manager.

Both the mirror and repository settings can be defined in the settings.xml file. I think I will do another post to explain what a good setup would look like...

There are two locations for settings.xml file:

the Maven installation at $M2_HOME/conf/settings.xml
the user's local settings: ${user.home}/.m2/settings.xml

Both can contain the same set of definitions. So, which one to use?

Actually, both locations require the Maven user (your team members) to manually do some configuration in their setting file. This is general not preferred for the known reasons: everybody needs to do this manually, no automatic update if settings change later on, unexpected behaviour when someone forgets to adopt his/her file etc.

The only way to avoid these drawbacks is to avoid all manual editing work, i.e. provide a central version of the settings file that is checked in to your sourcecode management system (SCM). This can't be done for the second option (user's settings) but it can be done for the first (installation settings) – if you put Maven installation under source control.

This sounds a bit strange at first (after all, it's an executable!) but is really clever IMO, for the following reasons:

You make sure everybody uses the same version of Maven; no more Maven version dependencies!
You can use a relative path from your projects to the Maven installation if they are part of the same source repository, for instance in batch files, Eclipse launch configurations etc.
The correct settings are applied automatically and may be updated in the repository without requiring any editing by the developers (except for updating).
Maven installation is only around 2 MB of size, which is not really an issue for any SCM.

The Bottom Line

To summarize, by putting your Maven installation in your SCM as part of your project environment, and using central settings to configure Maven to use your repository manager, you reduce dependency to local environment. And stabilizing your builds is always a good thing!

How big is BigDecimal?

2009-03-25T00:41:00.001+01:00

Lately, there was a debate in our company about rounding of numbers, more specific on how, when and where to do that.

One of the questions was if a calculation method should return a rounded value, or if the result should be precise and rounded by the caller. Another question was how to represent the values and which functionality to use to actually do the rounding.

There was a suggestion to use BigDecimal objects everywhere instead of simple double types because this class provides convenient methods for doing rounding.

Of course, when you need the higher precision, this might be a great choice. However, when you don't need that and are just using the class for being able to easily use it's rounding capabilities, the solution is probably over-engineered. Well, I voted against that mainly for two reasons: performance and object size.

1) Performance

It's obvious that calculations with primitive data types are faster than with BigDecimals (or BigInteger). But... how much?

A small Java code snippet helps to estimate the performance penalty:

final long iterations = 1000000;
long t = System.currentTimeMillis();
double d = 123.456;
for (int i = 0; i < iterations; i++) {
    final double b = d * (
                       (double)System.currentTimeMillis() 
                     + (double)System.currentTimeMillis());
}
System.out.println("double: "+(System.currentTimeMillis() - t));

t = System.currentTimeMillis();
BigDecimal bd = new BigDecimal("123.456");
for (int i = 0; i < iterations; i++) {
    final BigDecimal b = bd.multiply(
      BigDecimal.valueOf(System.currentTimeMillis()).add(
      BigDecimal.valueOf(System.currentTimeMillis())));
}
System.out.println("java.math.BigDecimal: "+(System.currentTimeMillis() - t));

We are not interested in absolute numbers here, but only in the comparison between double's and BigDecimal's. It turns out that one million operations (each is one multiplication and one addition of a double value) takes approximately 3-4 times longer with BigDecimal than with doubles (on my poor old laptop with Java 5).

Interestingly, when trying the same for BigInteger and long, the factor is approximately 5, i.e. the performance difference is even higher.

With Java 6, the method runs faster for all types, but calculation with primitives has a greater improvement so that the performance penalty for using Big* is even higher: 4-5 for BigDecimal, 6 for BigInteger.

2) Object Size

Everybody would expect that a BigDecimal would need more memory than a primitive double, right? But, how much is it? We are going to have big objects with up to hundreds of decimal values, so the bigger BigDecimal's might sum up to a critical value when thinking of transporting those objects between processes (web service calls) or holding them in the session (for web applications).

It happended that I have blogged about how to determine an object's size in my last post ;-) Hence, we can just move on to the actual figures:

double: 8 bytes

Double: 16 bytes (8 bytes overhead for the class, 8 bytes for the contained double)

BigDecimal: 32 bytes

long: 8 bytes

Long: 16 bytes (8 bytes overhead for the class, 8 bytes for the contained long)

BigInteger: 56 bytes

Wow. It seems that BigDecimal is 4 times as big than double and twice the size of Double – which is not that bad. As before, BigInteger has a bigger penalty with respect to object size as well.

3) Conclusion

All in all, when using BigDecimal instead of double, this means factor 4 for both memory footprint as well as performance penalty. A good reason to not use BigDecimal's just for using the rounding functionality...!

Size of Java Objects

2009-03-23T10:13:00.000+01:00

I'm sure you know that measuring the size of objects in Java is not so easy since there is no C style sizeof() functionality. Additionally, the actual size used to store an object on the Java heap depends on several variables: the JVM implementation, operation system (32/64 Bit) etc. Hence, a particular value for the amount of storage consumed by an object can be compared to the size of another object, but not between different runtime environments.

So... how can the size of an object (i.e. it's memory usage) be determined? There are actually two possibilities. Both are well-known and not invented by me, so I only provide some basic information and links.

1) Use Runtime.freeMemory()

The usual (old-fashioned) way to estimate the size of an object is like this: call garbage collector (GC) to ensure all unused memory is freed, then count current memory consumption (M1), construct the object, GC once again and count memory (M2). The difference M2-M1 indicates the amount of memory used for the created object.

There are a few things to note:

A single call to GC is more or less only a suggestion to the Java Virtual Machine to reclaim space from all discarded objects – it's no guarantee that GC has been finished (method is not blocking) and all old objects have been removed. To be a bit more aggressive, GC should be used a couple of times.

To make sure that supplementary memory (for static data etc.) is already allocated before starting memory count, you should construct an object and set the handle to null before starting the estimation cycle described above.

The precision might increase when creating not a single object, but a large amount of them.

JavaWorld's Java Tip 130 described this approach, and Heinz Kabutz published two JavaSpecialists newsletters (Issue 29, 78) about determining memory usage in Java.

Addtionally, there is an open source project java.sizeOf at SourceForge using this approach.

2) Use Instrumentation.getObjectSize()

Starting with Java 5 there is a new method to determine object size: the instrumentation interface. It's getObjectSize() method is still an estimate, but seems to provide more accurate results, albeit a bit slower than counting free memory.

In short, you have to implement an instrumentation agent that contains a premain(String, Instrumentation) method that will be called by the JVM on startup. The given instrumentation can be used to call methods on it later on. The agent has to be packaged into a JAR file that requires a Premain-Class specification in it's manifest file. To use the instrumentation agent, call java with the -javaagent option. For more information, see here and this blog post.

Guess what, Heinz Kabutz has published another JavaSpecialists newsletter 142 describing this approach (you see, it's really worth subscribing!). Refer to this java.net article for another example of how to use Java instrumentation.

That's it for today... One more remark: note that both described ways are not able to provide exact figures, but only estimates of memory consumption. This is not really an issue because these estimates are typically exact for small objects, and size of complex data structures can be calculated using the known size of basic types and data structures.

Sonar is SO cool!

2009-03-20T13:26:00.001+01:00

Do you know Maven? Then you know the project site that can be easily generated with Maven. For instance, look at the site for the Tomcat Maven plugin. It provides information on using the plugin, project related information and – probably most importantly for most "normal" (i.e. non-plugin) projects – the project reports.

You can easily configure Maven to execute a number of useful reports like JavaDoc, Checkstyle, PMD (coding rule verification), CPD (duplicate code detection), and JUnit test coverage. Additionally, you can install custom reports to also participate in project site.

This is great, but still lacks some features:

What if you would like to see overall code quality, without having to consult several detailed reports? Just a single, combined indicator?

With the reports, it's not always easy to drill down a particular issue up to source code level.

It would be nice to be able to access historic versions and compare quality between them to early recognize trends, wouldn't it?

All this (and more) is provided by Sonar (http://sonar.codehaus.org/), an open source tool that "enables to collect, analyze and report metrics on source code. Sonar not only offers consolidated reporting on and across projects throughout time, but it becomes the central place to manage code quality."

Sonar collects data provided by well-known Maven reports, stores them into a database, and provides a modern, fast and convenient user interface to browse the projects and quality metrics, and to drill down from project to Java code level.

Just look at the screenshot of an internal test project... how cool is that?

Installation is as simple as it can be. You can use Sonar with provided Jetty or install it in your existing container. For production, you can switch from embedded Derby database to a "real" database (like MySQL, Oracle, SqlServer, ...). Of course, you can adjust the rules to be checked or import your existing configuration (for Checkstyle or PMD).

To send data to Sonar, you just execute a Maven command to call a particular sonar-maven-plugin for your project:

mvn clean install org.codehaus.sonar:sonar-maven-plugin:1.6:sonar

That's it... now watch all the magic going on.

This Maven goal can be called manually, but is best integrated with nightly builds. To further simplify this, there is a Sonar plugin for Hudson, my favorite continuous integration engine. Using this nice plugin, configuration of a Job to connect to Sonar is as simple as clicking a checkbox in post-build section!

Sonar is so great I really wonder why I didn't find this tool earlier – current version is 1.6, so it must have been out for a while... You definitely should give it a try!

Maven Profiles: Activation... or not

2009-03-19T12:51:00.000+01:00

I love Maven. Really, I do. I should say that since this is my first post in my own blog (I know, I'm probably the last man on the planet... ;o) Having said this, here is another annoyance. There are lots of them, inside and outside of Maven, and I'm going to share some of my experiences here.

This time, it's about Profiles. I'm not going to explain the basics (see Introduction to Build Profiles or the chapter on profiles in Maven: The Definitive Guide, for instance).

I wanted to use profiles to specify some re-usable configuration (for code generation out of oAW models, but that doesn't matter) in a parent POM, without having to repeat all the stuff in child POMs. The profile definition in parent POM consists of a plugin configuration along with required dependencies and looks like this:

<profile>
  <id>adsl</id>
  <build>
    <plugins>
      <plugin>
        <groupId>org.fornax.toolsupport</groupId>
        <artifactId>fornax-oaw-m2-plugin</artifactId>
        ... all the execution and configuration details ...
      </plugin>      
    </plugins>
  </build>
  <dependencies>
    <dependency>
      <groupId>com.fja.ipl.adsl</groupId>
      <artifactId>adsl</artifactId>
      <version>${ipl.current.version}</version>
    </dependency>
    ... and some more ...
  </dependencies>
</profile>

Then I tried to activate the profile in those child POMs that should use it since they are based on an oAW model. Note that I wanted to activate the profile in the POM itself, not throug settingings or command line options. And this is where the pain begins...

First attempt was to just activate the profile in a child POM like this:

<profile>
  <id>adsl</id>
  <activation>
    <activeByDefault>true</activeByDefault>
  </activation>
</profile>

However, instead of activating the inherited profile with the given id, Maven just overrides the profile from parent POM with a new (and empty) one. Okay, this might be the desired behaviour.

So, next attempt was to use the activation on a present file: whenever there is a workflow file src/main/resources/generateAll.oaw, the profile should be active. This is obvious, since oAW generation uses this workflow file anyways. Parent POM now looks like this:

<profile>
  <id>adsl</id>
  <activation>
    <file>
      <exists>src/main/resources/generateAll.oaw</exists>
    </file>
  </activation>
  ...
</profile>

This works great... almost. When dealing with multi-module projects, it depends on from where you start the Maven build. For instance, let's assume project P aggregates modules M1 and M2. When you step down to M1 and start Maven, the file can be found in context of M1 and the profile is activated, so everything is just fine. Same is true for M2. However, when the build is started for project P, the file is searched in P's context where it can be found and hence the build of M1 and M2 fails.

Sounds not that complicated... let's just use the ${basedir} property:

<exists>${basedir}/src/main/resources/generateAll.oaw</exists>

Well, it appears that this property is not expanded at all (see MNG-1775). Thus, this doesn't work either.

So, we're stuck. I don't have any other idea (do you?)...

This might not be the typical use case for profiles, which are meant to modify the POM at build time to support environment/platform specifics. Agreed. However, as long as there is no other way of defining complex configuration in parent POM that should be reused in some (not all) child POMs, this would have been a useful approach, IMHO.

Now, we end up in defining the plugin configuration in <pluginManagement> section and to enable it and add required dependencies in child POMs. That is, we have to repeat the following section in every single child project that want to harness oAW generation:

<build>
  <plugins>
    <plugin>
      <groupId>org.fornax.toolsupport</groupId>
      <artifactId>fornax-oaw-m2-plugin</artifactId>
    </plugin>
  </plugins>
</build>
<dependencies>
  <dependency>
    <groupId>com.fja.ipl.adsl</groupId>
    <artifactId>adsl</artifactId>
    <version>${ipl.current.version}</version>
  </dependency>
  ... and more of them ...
</dependencies>

Cumbersome, redundant, hard to maintain. Maven should do better, really!