Having Fun with Encoding!

Compiler Plugin

Recently, I edited our main company root POM to upgrade some plugins to new versions. Of course, we are following best practice to lock down the plugin version, so when a new version is available we only need to adjust the parent POM. Nearly all version updates were on the last build number digit, which is the z in x.y.z version string – so I didn't expect much difficulties.

However, for the compiler plugin, it was a jump from version 2.0.2 to 2.1, and indeed it turned out that some of the test cases failed compiling with strange encoding issues when using the new compiler plugin version.

Specify Encoding

We are following the suggestion to specify a POM property for source file encoding, for not being forced to configure encoding for all relevant plugins individually. Moreover, we were exactly using what's shown in the example:

<project>
  ...
  <properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    ...
  </properties>
  ...
</project>

That is, we assumed our source files were UTF-8 encoded, which is the most widely used encoding for unicode characters. But, for some of the projects, that's actually not the case since we are using Eclipse with the default setting for text file encoding which is Cp1252 (Western European) on our german Windows.

Why didn't we ever notice that? Well, it happens that both the UTF-8 as well as Cp1252 encodings are backwards compatible with ASCII. We are coding most of the stuff in english (concerning package, class, method, attribute and parameter names, and even Javadoc comments), so the resulting byte stream will never be different for both encodings. However, some of the files used german umlauts in line comments which are exactly the files that can't be compiled any more with new compiler plugin version.

When looking at the debug output of compiler plugin 2.0.2 mojo configuration, you can see that the encoding is not explicitely set, probably meaning that the platform default encoding is used (which is again Cp1252 on all build machines):

[DEBUG] Configuring mojo 'org.apache.maven.plugins:maven-compiler-plugin:2.0.2:compile' -->
[DEBUG]   (f) basedir = ...
[DEBUG]   (f) buildDirectory = ...
[DEBUG]   (f) classpathElements = [...]
[DEBUG]   (f) compileSourceRoots = [...]
[DEBUG]   (f) compilerId = javac
[DEBUG]   (f) debug = true
[DEBUG]   (f) failOnError = true
[DEBUG]   (f) fork = false
[DEBUG]   (f) optimize = true
[DEBUG]   (f) outputDirectory = ...
[DEBUG]   (f) outputFileName = xxx-0.2.0-SNAPSHOT
[DEBUG]   (f) projectArtifact = xxx:jar:0.2.0-SNAPSHOT
[DEBUG]   (f) showDeprecation = false
[DEBUG]   (f) showWarnings = false
[DEBUG]   (f) source = 1.6
[DEBUG]   (f) staleMillis = 0
[DEBUG]   (f) target = 1.6
[DEBUG]   (f) verbose = false
[DEBUG] -- end configuration --

The new version 2.1 of compiler plugin is now considering what has been configured in project.build.sourceEncoding property, and hence tries to compile the Cp1252 coded source file with UTF-8 encoding which doesn't work when umlauts are used.

Specify Correct Encoding

Of course, the solution is to specify the correct encoding in project.build.sourceEncoding property, matching the encoding that is used in the development environment when writing the source files.

Oh, yes, Cp1252 is quite similar to ISO 8859-1 encoding (only some special characters on positions 0x80–0x9F are different which we don't use), so in fact we are using ISO 8859-1 now to allow builds on non-Windows platforms as well.

Certainly, it would be nice if the plugins had a history on their site where you can find this type of changes for new versions, without having to search in the Jira...

Java Mystery: Directory Exists but Does Not?

Well, yesterday for me was another one of those days where you think your computer must be fooling you... There is an issue with a DirectoryCleaner that works for a subproject but does not when called from the parent. But first things first.

The openArchitectureWare Part

We are using oAW (yep, the new version which is part of Eclipse Modeling Project of Eclipse Galileo). Before generating the artifacts from the model, we are executing a directory cleaner that just rubs out the folders, for instance src/generated/java. The workflow file generate.mwe looks like this:

<workflow>
   <property file="generateAll.properties"/>
   <component class='org.eclipse.emf.mwe.utils.DirectoryCleaner' directory='${srcGenDir}'/>
   <component file='.../generateAll.mwe' inheritAll="true">
      <modelFile value='...' />
   </component>
</workflow>

srcGenDir is a property that is defined in the included properties file, but that's irrelevant. The highlighted line configures the mentioned directory cleaner. The relevant code snippet is invokeInternal() method of org.eclipse.emf.mwe.utils.DirectoryCleaner which is part of Eclipse EMF frameworks:

protected void invokeInternal(final WorkflowContext model, 
                 final ProgressMonitor monitor, final Issues issues) {
   if (directory != null) {
      final StringTokenizer st = new StringTokenizer(directory, ",");
      while (st.hasMoreElements()) {
         final String dir = st.nextToken().trim();
         final File f = new File(dir);
         if (f.exists() && f.isDirectory()) {
            ... do the cleanup ...
         }
      }
   }
}

Pretty simple, right? The directory attribute contains one or more directories (comma separated), hence a tokenizer is used to get each of them and to create a file. If this file exists and it's a directory, that will be erased.

The Maven Part

Of course we are using Maven to build the stuff. The Fornax Maven plugin is configured to call the workflow during Maven build. Moreover, the mentioned workflow is part of a subproject B which belongs to an outer multi-module project A.

Now, this is where the mystery begins... When I build B (the submodule), everything is fine and works like expected. However, when I build A (the parent project), B will be built in turn and its workflow is executed, but the directory is not cleaned up! You wouldn't expect this, right?

The Strange Part

In an attempt to find out what's going on, we put some debugging code into DirectoryCleaner. It turns out that the file f returns the same value for getAbsolutePath() in both cases, but f.exists() reveals false when the build is started from the parent project – hence the condition is not met and nothing will be cleaned up.

Unfortunately, you can't look any deeper into the native code that is called when determining if a file exists. So, in a kind of trial and error approach, we found out that using the following code fixes the issue:

protected void invokeInternal(final WorkflowContext model, 
                 final ProgressMonitor monitor, final Issues issues) {
   if (directory != null) {
      final StringTokenizer st = new StringTokenizer(directory, ",");
      while (st.hasMoreElements()) {
         final String dir = st.nextToken().trim();
         final File f1 = new File(dir);
         final File f = new File(f1.getAbsolutePath());
         if (f.exists() && f.isDirectory()) {
            ... do the cleanup ...
         }
      }
   }
}

That is: by creating another file that is using the absolute path of the first one and using this further on, everything is fine in both scenarios – whether called from parent project or submodule.

To be honest: I have no explanation for these findings. Why is File behaving differently? When calculating the exist flag, the code should take into account the file's absolute path, right? So, why is creation of another file based on the absolute path is fixing the issue? Any insights are deeply appreciated...!

Checkstyle: One and Only Configuration File?

The Checkstyle Challenge

When you are using both Eclipse and Maven, you are probably facing the same challenge like we do: you would like to have a single configuration file for Checkstyle tool and reference that in Eclipse as well as in Maven.

Checkstyle ships with some default conventions (according to the Sun coding standards). But you usually want to customize this and create your own configuration set, based on your corporate coding styles.

Now, you want to setup Checkstyle within Eclipse to provide direct feedback to developers. You want to setup Checkstyle with Maven to be able to have a coprehensive report built as part of your project site. And you want both to reference the same configuration file, to avoid having to maintain two of them. Naturally, right?

Well, unfortunately, it seems impossible to achieve... if you really found a way, let me know!

However, there nonetheless are some tips to help you having a centralized configuration of Checkstyle with both tools. This post is about how to do that.

Checkstyle with Maven

The goal is to centralize the Checkstyle configuration to make its usage as simple as possible for all the projects, avoiding manual configuration as far as possible. Fortunately, this is quite easy using Maven's dependency mechanism.

Essentially, we build a custom artifact that contains nothing but our Checkstyle configuration file. This JAR is deployed to the company's Maven repository from where it can be downloaded as a normal dependency when needed. After that, the Checkstyle plugin is configured (in the reporting section) to reference the packaged configuration file. That's it!

Now, we'll look at the details...

Let's assume our checkstyle configuration is located in config/my-checkstyle.xml. Then the POM of the checkstyle artifact, which actually is a minimal Maven project, looks like this:

  ...
  <groupId>com.fja.ipl</groupId>
  <artifactId>checkstyle-config</artifactId>
  <version>1.0</version>
  <build>
    <resources>
      <resource>
        <directory>config</directory>
      </resource>
    </resources>
  </build>
  ...

(Of course, you additionally have to add the distributionManagement section to be able to deploy the artifact, but this is usually derived by the company's base POM.)

Deploy the artifact. You'll notice that the created checkstyle-config-1.0.jar indeed contains your my-checkstyle.xml file in its root.

Now let's switch to the project that should use the Checkstyle report. The configuration file is referenced in the Checkstyle plug-in configuration in the reporting section like this:

  <reporting>
    <plugins>
      <plugin>
        <artifactId>maven-checkstyle-plugin</artifactId>
        <version>2.2</version>
        <configuration>
          <configLocation>my-checkstyle.xml</configLocation>
        </configuration>
      </plugin>
      ...
    </plugins>
  </reporting>

That is not yet working, Maven can't find this file – our checkstyle-config artifact has not been referenced. We do this now by adding it as an extension dependency to the POM as follows:

  <build>
    <extensions>
      <extension>
        <groupId>com.fja.ipl</groupId>
        <artifactId>checkstyle-config</artifactId>
        <version>1.0</version>
      </extension>
    </extensions>
    ...
  </build>

The artifacts configured as extensions will be included in the running build's classpath. That means, checkstyle-config-1.0.jar now is in the classpath of the Checkstyle plugin, and the file referenced by the configLocation setting is located in the JAR and hence can be found.

Using this approach, the Checkstyle configuration can be maintained centrally, and projects can get new versions simply by updating the artifact's version in the extension section.

Checkstyle with Eclipse

Unfortunately, I don't really see a way to use the same Checkstyle configuration, packaged into the Maven artifact, with Eclipse as well...

So you'll have to duplicate the configuration file. However, to keep the number of duplicates small, I recommend to put it on a central network drive within your company and use the Remote Configuration option of the Checkstyle Eclispe plugin to reference the remote file. This does not prevent you from working offline because Eclipse downloads and caches this file.

In this scenario, changing the Checkstyle configuration is done centrally by just updating the remote file. This is a great advantage over using a configuration that is part of the project, because in the latter case you would have to upgrade every single project.

Eclipse Top Annoyances

Eclipse is one of those love/hate things. I love it and hate it, both at the same time.

The Love

Firstly, it's the best Java IDE on the market, and we successfully use it for years now. In the meantime, there might be other IDEs with an even richer feature set, better integration, nicer UI etc. but we actually don't care very much. I love Eclipse just for what it did to the Java and Open Source community, and it definitely has been pushing the edge for a couple of years now.

BTW, of course I know Eclipse is more than just a Java IDE, but this is my personal short interpretation of "development platform comprised of extensible frameworks, tools and runtimes for building, deploying and managing software across the lifecycle".

The Hate

Well, having said all this... sometimes Eclipse really drives me crazy. There are a couple of annoyances, some of them rather small but insistent, others bothering us since quite a long time.

The strange thing is, all of my complaints are not related to the more sophisticated plugins like Mylyn, WTP, MDT and so forth. (Yes, we use them too, albeit in a sketchy way.) No, the issues annoying me most are shown by the Platform or JDT, and maybe that's why they are actually so annoying.

The List

Okay, here is my list for Eclipse 3.4 (Ganymede):

6. Formatting

We have setup the Eclipse Java code formatter to match the project requirements. There are plenty of options to adjust to your needs, but... sometimes the algorithm is just broken. Take this example:

private Accumulator getSpecificAccumAndSetRuleDefaultValue(
                                                            CoverageRule calcStepRule,
                                                            CalculationStepInfo calculationStepInfo,
                                                            ...)
{
   ...
}

What is this extra line break before the first method argument good for? Instead of aligning the method arguments horizontally, Eclipse messed it up vertically. You can find several other strange formattings where formatted code requires extra space, mainly in long method or class declarations. Here is another example of an extra line break:

public class PersistenceService<R extends PersistentRootObject, D extends HibernateDaoSupport>
      implements
         IPersistenceService {
  ...
}

5. Blocking Workspace

I have blogged about this blocking behaviour before: sometimes, Eclipse can't save my file while it is building "in the background" and tells me that my "User Operation is Waiting"...

That's just silly, Eclipse should be able to do my save action at any time, if compiling or not. Loosing editor's content (by unintended editing, Eclipse crash or hang-up, whatever) just due to the impossibility to save is just not acceptable.

4. Startup Time

Depending on size of the workspace, you got to wait minutes before you can start editing. That's just too slow. Period.

3. Dialog Window Dimensions

When you open a project's Properties window, for instance, it will show up quite small. I have a large screen and don't like to scroll unnecessarily, so I resize the window to be considerably larger. However, the dialog dimensions are not persisted – when I close and reopen the same dialog, it shows up as small as before. Eclipse just refuses to remember what I like.

For other dialog windows (like Open Type), this works perfectly; why not for all of them?

2. Update Manager

I used to think the new Eclipse 3.4 update manager (p2 Provisioning System) was a good thing, but I'm no longer sure. As end user, you just don't want to mess around with features, dependencies, build numbers... and get strange error messages (see this post of an issue with a custom built Eclipse plugin we struggled with recently).

Additionally, the list of available software is too technical for most end users. Subversive SVN Connectors plugin provides 6 optional connectors, but at least one is required. So, which one(s) do I choose? Instead of having the full list with just the name and version strings, I would rather like to see some typical sets of required features ("typical installation" similary to when installing software on Windows). Alternatively, at least provide some end user level meta information like "This is the default feature", "Choose this item if..." etc.

1. Vanished Project Contents

Sometimes the content of my workspace projects in Package Explorer view is "gone", meaning I can't expand the project nodes any more. Yes, of course there is some content, and fortunately it is still on the disk – but just not showing up in the view.

Project Explorer view still shows the projects correctly. Closing and re-opening them in Package Explorer usually helps.

The Hope

I used to install the latest milestones for some years, but have given up this habit because of stability/quality/compatibility issues with Eclipse core and/or the plugins I use. Hence, I can't tell which ones will be addressed by the Eclipse 3.5 (Galileo) which is going to happen on 24th of June.

So... let's keep our fingers crossed!

Update to Checkstyle 5.0

Checkstyle 5 is available!

This is good news: a new version of Checkstyle is available that better supports Java 5 language features (like generics, annotations or package-info files). Additionally, some checks are cleaned up a bit, for instance concerning their parent in which they are contained. See the release notes for details.

Due to these changes, Checkstyle 5.0 is not fully compatible to previous versions 4.x – which is already indicated by the version number leap. Hence, don't expect the upgrade to be smoothly!

Nevertheless, I thought it would be time to upgrade, so here is what I did...

Step 1: Upgrade checkfile configuration

Due to the incompatabilities, I had to apply some changes to our checkstyle configuration to make it work with Checkstyle 5. To be sure I did it right, I have downloaded and used the Checkstyle binaries and checked my configuration for just a simple project. Hence, before upgrading Eclipse and Maven Checkstyle integration, I know my configuration is correct.

BTW, we are using a common checkstyle configuration file for all projects, and reference this "global" file with Maven and Eclipse in different ways:

• With Maven, we use a custom artifact that contains nothing but our checkstyle.xml file. This in turn is included as build extension in our main base POM. See this post for more details.


• For Eclipse, we use the "Eclipse Checkstyle Plugin" that provides a Remote Configuration option to reference the configuration file on an internal file server.

Step 2: Upgrade Eclipse Plugin

There is a new Beta-Release of Eclipse Checkstyle plugin (eclipse-cs) available on its update site http://eclipse-cs.sf.net/update. At the time of writing, there are three features available:

• Eclipse Checkstyle Plug-in – version 5.0.0-beta4
• Eclipse Checkstyle Plug-in 4.4.x -> 5.0.0 Migration (Optional) – version 5.0.0-beta4
• m2eclipse Maven Synchronization Plugin (Optional/Experimental) – version 0.0.3

Oh, seems there would have been some help in migrating from Checkstyle 4.4 to 5.0... Doesn't matter, I always like to see what the changes are so it's good to do it manually.

The m2eclipse Synchronization plugin is a new feature providing mechanism to synchronize Checkstyle rules and configuration between the maven plugin and eclipse-cs. Sounds really interesting... but let's do one step after the other and test this later.

So. I just installed the "Eclipse Checkstyle Plug-in" feature. Eclipse didn't recognize that this actually is an update, so you have to uninstall the previous eclipse-cs installation manually.

Why that? Well, the "package" has changed from com.atlassw.tools.* to net.sf.eclipsecs.*, and this applies to the feature's ID, too. Moreover, this renaming also affects the buildCommand and nature in .project files, they have to be net.sf.eclipsecs.core.CheckstyleBuilder and
net.sf.eclipsecs.core.CheckstyleNature now.

Additionally, the notation for file sets has been changed: a file set previously configured as src\\main\\java\\com\\mycompany\\.* does no more match to any file; instead, it has to be the slash now like in src/main/java/com/mycompany/.*.

Okay, maybe I should have tried the Migration plugin... Anyways, after these changes everything works fine for me in Eclipse.

Step 3: Upgrade Maven Plugin

Good. Last piece is Maven, which provides the Maven 2 Checkstyle Plugin. However, the current version is 2.2 which is based on Checkstyle 4.4 by defining these dependencies:

<dependency>
  <groupId>checkstyle</groupId>
  <artifactId>checkstyle</artifactId>
  <version>4.4</version>
</dependency>
<dependency>
  <groupId>checkstyle</groupId>
  <artifactId>checkstyle-optional</artifactId>
  <version>4.4</version>
</dependency>

There is an interesting way to override the plugin's dependencies pointed out by Brian Fox, but that's not going to work for us because Checkstyle versions 4.4 and 5.0 are not API compatible.

What can we do? Not much... we'll have to wait for a new version of the Checkstyle plugin that updates to Checkstyle 5. There is this Jira issue, and patches have already been provided some time ago. It's only that there seems to be no progress whatsoever... Checkstyle 5.0 is officially out since April 18th, so there is no reason to wait any longer! Create a new release (for my part, alpha/beta is fine as well) – please!!!

If you really need the plugin to be fixed now, you could checkout the plugin's sources and built your own version, applying the patch provided in the Jira issue. That works, but is nothing we want to do regularly!

Eclipse: Update Manager Fools Me

Recently, we experienced an issue with a custom Eclipse plugin developed and hosted by my company. When trying to install the plugin into various versions of Eclipse Ganymede, the update manager always told me "Cannot find a solution satisfying the following requirements org.eclipse.ui [3.4.2.M20090204-0800]".

Finally, after some lost days of searching and trying, we discovered that this message is totally misleading: the issue was not caused by the feature org.eclipse.ui missing or being available only with wrong version. Instead, the reason was a misconfigured Manifest file of the custom plugin. It just defined wrong version for the required bundles (oAW, the MDA/MDD generator framework we use).

How nasty is that? Couldn't Eclipse update manager (p2) provide better support? More useful messages? Gosh!

Moreover, Eclipse update manager seems to cache the metadata of remote repositories. That might be nice to avoid network overhead of loading the repository again and again. But... when testing installation of different plugin versions with a local update site, it definitely gets into your way. The sad thing is, you can't get rid of the cached contents easily. You'd have to remove the site, restart Eclipse with -clean option two times, and after that recreate the location. Could be made easier, really.

Eclipse update manager will once more be refactored in upcoming Eclipse 3.5 Galileo. Folks, don't mess it up again!

Eclipse: User Operation is Waiting, and Waiting, ...

I am using Eclipse since quite a long time, sometimes around 2002. That was version 2.0, if I remember correctly. Since then, I have always upgraded to the latest version, if not milestone build. Most of the time I was quite happy with the stability and really appreciated all the evolving features and UI improvements.

One major improvement (it was in 3.0, right?) was when they moved builds to a background process, enabling you to continue working when the class or project needed to be compiled.

But... what is this? In all the latest release versions (3.4.0 ... 3.4.2) I sometimes get this dialog when trying to save a file:

Eclipse can't save my file while it is building "in the background"? Come on! Nota bene, I am not talking about .project files or other stuff the whole build could depend upon, it's also true for simple resources or even documentation files!

That's really annoying. This is a modal dialog, which means I have to wait for minutes until build is complete or at least in a state where Eclipse thinks it can save the file. Of course, the user interface is blocked during that time... unless you cancel the save task.

I use the Eclipse Java EE package and additionally have installed a couple of regular plugins, like m2eclipse and oAW. So I can't tell for sure who is actually causing this mess, but on the other hand my installation is probably not so unusual and I'm sure others get this issue as well. There are some entries in various issue databases telling me that I'm right.

Eclipse is great and I (still) love it – but our relationship really gets poisoned by this kind of things... Maybe I try IntelliJ IDEA or NetBeans one day... I have warned you!