JRuby developments

I've spent another saturday at work, mostly working with KIMKAT, but taking some time to look at different JRuby-issues before the 0.9 release. The first - and only major - one, was that Zlib didn't seem to work for Charlie and Tom. I couldn't reproduce this error locally, which was very strange. But finally I found the error in my code, and also the reason I couldn't reproduce it in my own JRuby.

I'd managed to not remove a zlib.rb from one of the directories, and my test cases worked (albeit slowly), because it was using the former Zlib-code, and not my newritten one. After I found this problem, it went fairly fast to isolate the real error, in my GzipWriter. The problem was that I didn't call finish on the GZIPOutputStream which means the gzip-stream doesn't end with a gz-trailer. Once this was added, everything seems to work.

Problem number two seemed harder, initially, and the reason for this was that I needed to muck with the innards of JRuby threads to get it working. The problem concerns Signal handling. In Ruby you can trap operating system signals and execute blocks of code when they happen. This is obviously very powerful and useful for lots of things. My first solution for this, was to just save the block, and then call it when the signal occured. This didn't play well with the rest of JRuby, and Tom got some other strange errors from this. So, after some thinking I came to the realization that I had to create a new RubyThread, and start this when the signal occured, and then JRuby's internal thread scheduling would acertain that nothing untoward happens. This works really well, but I just realized one drawback with it, so I'm going back in to fix it now. The current approach can only execute the block once. Ouch.

So, now I've fixed a new approach. This is much more involved though. I had to get really into the internals of the thread architecture so I could actually create a new JRuby-thread with an existing RubyProc, Frame and Block object. So now I create a new RubyThread each time the signal is called, and sets the block-information manually. This is needed so we can run it multiple times.

Anyway, the real point of this blog is the last problem. The speed of RDoc. RDoc does some fairly heavy lifting, and it itsn't especially fast on C Ruby either:

$ time ruby d:/programming/ruby/bin/gem install rake
Attempting local installation of 'rake'
Successfully installed rake, version 0.7.1
Installing RDoc documentation for rake-0.7.1...

real 0m13.915s
user 0m0.000s
sys 0m0.000s

But now, JRuby isn't near this speed in any way at all:

$ time bin/jruby.bat bin/gem install rake
Attempting local installation of 'rake'
Successfully installed rake, version 0.7.1
Installing RDoc documentation for rake-0.7.1...

real 1m42.985s
user 0m0.000s
sys 0m0.010s

Yes, it's about one magnitude slower. Yesterday I found a big reason for this. JRuby does a lot of hashing and spends very much time in HashMap#get. So I was curious what hashCode-algorithm was used internally. What I found was this:

    public final int hashCode() {
        return RubyNumeric.fix2int(callMethod("hash"));
    }

Which actually calls the Ruby hash-method every time someone wants a plain hashCode. This explained some of the bad performance. I added some better hashCodes to different common Ruby-objects, like RubyString, RubyHash, RubyFixnum and others. After this small change I got this:

$ time bin/jruby.bat bin/gem install rake
Attempting local installation of 'rake'
Successfully installed rake, version 0.7.1
Installing RDoc documentation for rake-0.7.1...

real 0m59.946s
user 0m0.000s
sys 0m0.010s

... That is almost double the speed for general JRuby performance. By just adding a few simple hashCode-methods at different places. So this is the lesson from all this: if you have performance problems; look at your hashCode(), it may be easy to fix!