Tomorrow is today - or from London to Stockholm to Krakow

Part I: London

I got up at 5am, got dressed and packed and headed of to the airport. Finally got on the plane to London (thank god for online check-in) and tried to get some work done. That didn't go very well, but I got to read some instead. I'm rereading Ubik. Classic. Landed at Heathrow about 10 minutes behind schedule. Ran to the Piccadilly line, tried to talk with people in Sweden, but the connection was kinda bad. Arrived at Covent Garden, walked in brisk pace to the place of my meeting. Finally arrived, 5 minutes late, to find out there was no hurry. Sat down and had a pleasant chat with Dan North and Simon Stewart. Very nice. (Hey Dan, if you read this, get in touch when you're in Stockholm for ExpertZone. I know places with good beer.).

Then, Lunch. Haha. What a joke. I found a place that seemed to offer great vegetarian burgers, so I settled down for a quorn burger and some British chips. After 15 minutes waiting I got told the grill wouldn't light up. Lucky me. Instead, I had to make do with a Pret sandwich and a spicy vegetable Cornish pasty (someone told me it was the day of the Cornish patron saint yesterday, so I guess that fits). A few minutes walk on Oxford street was nice. I got photographed by some guys from an Austrian fashion magazine. Then back for the real meeting.

Cyndi Mitchell is a very formidable person. I was impressed. We had a 2 hour talk; very interesting. I will tell you all about it as soon as I can. But suffice to say, I was very happy when I left. Until I realized that I had exactly 2 hours until my flight taxed out from Heathrow. So, back to Covent Garden. Back on the underground. Which through some bad communication on the drivers part, almost ended badly when the train reached the end stop before Heathrow. Thanks to the people who told me to get of the train and switch to the other side. =) I finally got to Terminal 3, realized I already had a boarding card, found my way to the security check, got my boots scanned two times, and found the gate about 20 times before takeoff. Of course, as soon as the aircraft had taxed out, the flight management decided that we should stay 30 minutes on ground before finally taking off towards Copenhagen. It's a wonder I actually got there in time. So, in Copenhagen I walked 2 klicks to the transfer center, got a new boarding card and walked back to almost the exact place I started from, got on my flight to Stockholm and finally landed. Wow.

Airport coach to Stockholm city, and night bus home to my dear old mental hospital. When I finally got to bed I ached all over, and had been awake and active for 22 hours solid. Sleep was nice at that point. Especially since the day had been a success on all accounts.

Part II: Stockholm

I finally woke up, took care of mail and told everyone about London. Thought I'd take it easy, maybe practice on my presentation for Saturday, possibly start writing on the presentations for the CS course I'm going to speak for this April. But alas, after some mailing with Poland, I realized I've made a big mistake. I thought I was going to Kraków this Friday, and staying until Sunday. That's not entirely right. I'm actually going tomorrow. Surprise!

So.

Part III: Kraków

As mentioned above, I'll land in Kraków tomorrow (Wednesday) evening. This is good in several ways. I will have a chance to see the city, and I will have the opportunity to see some more presentations at the conference I'm at. There are many interesting things to be had; I'm sad half of them are in Polish. I guess I can look at the slides and have some fun trying the guess the meaning. I am going to see the talks about IronPython, Google Ads and Seaside. And, of course, if anyone is nearby and would like to take a beer and talk computers, programming languages, AI, or whatnot, please hit me a mail, or a comment on this post. My evenings will be mostly free it seems.

That is, unless someone knows any good clubs with nice music; especially electronic stuff and/or postpunk, new wave, darkwave or gothy things.

Anyway. Time to sleep.

JRuby 0.9.8 is here

The JRuby team is pleased to announce the release of JRuby 0.9.8.

Download at: http://dist.codehaus.org/jruby/

This release has some great improvements:

• Ruby on Rails support. We have been working hard on getting Rails own unit tests running and over 98% of them now run successfully. We feel things are running well enough to invite Ruby users to kick the tires and help root out any final issues.
• Ruby classes can extend concrete/abstract Java classes and override methods
• New Java primitive array syntax
• Reimplementation of String, Numeric classes, and Array to be more correct and performant
• Significant bottlenecks have been identified. In some cases IO is 6.5x faster than previous releases. Java included classes are significantly faster than in the past.
• 220 Jira issues resolved since last release

Special thanks to Marcin Mielżyński for his tireless work in rewriting a number of core classes to be much for correct and quick. His attention to detail has rooted out many corner cases.

The amount of IRC conversations, mailing list threads, bug reports, patches, and blog entries in the community has been a great help and our community is really making a huge difference in how fast JRuby is improving. The amount of progress is really staggering!

If you have ever thought that JRuby wasn't mature enough, I would like to contradict that now. With this release we are better than ever.

More information can be found at http://www.jruby.org.

The world is spinning

If you didn't know that, the title tells it all. Of course, that's old news.

I haven't really been able to blog as much as I wish I could have, lately. There are reasons for this, of course. Two very exciting reasons, in fact. If everything pans out, I will be able to write about it in 2 to 3 weeks time.

In other news, we are gearing up for another JRuby release. This one will be a biggie. Many nice things will be in place, and it will set the record for both new features and bug fixes. I think no one will be disappointed by it, actually.

I have a few presentations lined up too. The closest to now will be in exactly two weeks. I will speak at the Academic Computer Science Festival in Craków, Poland. If you're somewhere close, by all means offer to show me the city. =) I will land March 9:th and fly out again March 11:th. My presentation will be at 17:00 March 10:th, CET. If you would like more information, it can be found at the festivals homepage, here. The presentation will be in English (since I don't speak Polish, obviously), and it will be slightly more technical than the usual JRuby presentations. There will probably be some detail about our runtime, interpreter, parser and lexer, and hopefully I'll get some info in about our YARV and Java bytecode compiler efforts. This will be very exciting to talk about, I'm kinda salivating just thinking about it. =)

Another, more long term presentation, has just been decided. I will attend TheServerSide Java Symposium Europe, in Barcelona, from June 27:th to June 29:th, and talk about JRuby from the perspective of a Java developer, and what it can do for you. Hopefully I will have time to see the city too.

I will update with more information when possible.

The new base

I have for a time argued that the JVM should become more like an Operating System, and Java the language for OS development. Other languages should run on top of the JVM for running applications. It seems I'm not alone in this line of thought. Robert Varttinen wrote some about it here. To me it seems like a compelling future. But it seems the next logical step for enterprise applications would be further virtualization. I would for example like my beans implemented in Ruby. But not only that, I would want all the business logic to be OS agnostic. I would like my Ruby logic to live on top of a JVM J2EE server, but that logic should be able to move, transparently, to a .NET-server and provide the same business logic at that place. What would be even better is if I didn't have to deploy it manually to all places, but the logic would just move to the places where it's needed. Will we see that anytime soon?

RailsConf galore

So, it's decided. Me and two colleagues from Karolinska Institutet will attend RailsConf in Portland, OR. I'm looking very much forward to it, especially since we are also going to JavaOne, and we plan to drive from San Francisco to Portland.

Hopefully I'll get to meet many of the people on the Ruby scene that I've only had mail contact with until now. See you there!

Ragel performance

I did some performance testing on the old and new Resolver implementation. The testing have some stupid tests that exercise bad parts of both implementations (like longest match, where it can't be decided what type something is until we have to backtrack about 20 characters). I placed these 24 strings in an array, and pounded on it with an instance of the ResolverImpl that is used in exactly the same way on all scalar values in an YAML document. The objective is to find out if the value is an implicit type or not. So basically, we give it a String, and get back a tag URI. So it's not like I'm parsing a language or anything. I'm just doing some recognizing here.

The old implementation was based on a Map> where the first letter of the string to resolve was used as an index to find a list of patterns to try sequentially. This worked fine, and made it extensible. But not very fast. This is the baseline. For 24 different strings, iterated 100 000 times for 2 400 000 resolves it takes 7879ms. That's OK, but not great.

Now, the new Ragel implementation is dead simple. It's just a translation of the regexps in the aforementioned Pattern's into a state machine. At EOF out actions (%/ for people in the Ragel knowhow), I execute an action that sets a local variable to a tag, and at the end of the resolve method returns that tag. Dead simple, and not exercising the full strength of Ragel, of course.
So, for the same number of resolves, this ResolverImpl takes 1288ms. That's 611% improvement in speed. Ain't it nice to have a friend such as Ragel? And the best part is, for harder tasks, these improvements would be even larger.
Finite State Machines are your friends. All your base are belongs to us.

Results of jvYAMLb

Well, the YAML-based loading is in JRuby trunk. On the way, some parts of the codebase got seriously simplified. Very nice. The final result, with regard to performance, is about 20-30% on speed. But the important gain is in memory usage. The new implementation takes only about one fourth of the memory the original used. So that's great.

Regarding the Resolver, as I mentioned in the last post, it required a different approach, since regular JvYAML uses regular expressions to recognize implicit tags. Since that approach isn't good with byte arrays, I decided to use Ragel to generate a recognizer. That approach was very successful. As soon as I got that working it was the obvious approach. Ragel is good. Ragel is great. Ragel is wonderful. I will use the same approach for regular JvYAML to get away from all those Java regexps.

So, next step will be to do the same conversion of the emitter. Of course, at that point performance isn't that important. It's more about memory usage and the need to get away from another external dependency in JRuby.

Faster YAML with byte processing

As noted in my last post, I have started work on converting JvYAML into JvYAMLb. Right now I have finished the work on the Scanner and the Parser, and it's looking quite good. The numbers I reported in the last post for regular JvYAML performance was wrong though. We're looking at about 7.8s to 10.0s for scanning that 3.5MB gemspec file. (And that's only the scanning, not file IO). But with the Scanner converted to use bytes and ByteList, the same processing takes 2.8s. That's a substantial difference. But it doesn't end with that.

As I said I also converted the Parser. It doesn't do any String processing at all, so I didn't expect either a speedup or slowdown except for that from the Scanner. But... Before, parsing the gemspec took 18.515s, but after, it runs in 4s. That's a dramatic speedup, and I don't really know where it comes from. Unless the earlier implementation generated so much more garbage, and used more memory, that it was noticeable in speed. Anyway, this looks good for JRuby YAML processing, since I expect big reductions in complexity in the callpath and generation of objects after the YAML processor is byted all the way through.

But tomorrow it's time to work on the Resolver, and that's going to be hard. Optimally, it would be nice to have a byte-based Regexp engine. And maybe that would be something for JRuby too, know? Our Regular Expressions must be dead slow now that they have to convert to strings all the time.

Announcing JvYAMLb, a fork

The conversion to using byte-arrays as the basis of our String work in JRuby has led me to realize that JvYAML just doesn't cut it anymore. The performance wasn't good to begin with, and it's even worse having to convert EVERY SINGLE STRING read into bytes. That's no good. As an example why something needs to be done I'm going to describe the transformations that happen to data in JRuby if executing this code:

YAML.load_file "gems.yml"

First, the file is opened, and wrapped inside a RandomAccessFile. Then data is read from it by YAML. Reading will proceed like this:
1. Bytes are read through the RAF, hopefully in chunks.
2. Those bytes are wrapped in a RubyString so they can be returned from the IO#read method.
3. An IOReader wraps that RubyIO object, gets the RubyString and converts it from bytes into a String, and this String gets converted into a char array.
4. That char array is returned to the YAML Scanner.
5. The chars from the char array is collected in a StringBuffer, and saved in various Strings as token values.
6. The parser, resolver and constructor work on these Strings in various ways.
7. The JRubyConstructor takes these Strings and creates RubyString objects from them and in the process converting the String back to a byte array.

Is there any doubt that this process is slow? Well, it hasn't been that big of a problem until now, since we are doing so well on performance in other parts of the system.

So, the radical decision is to rewrite JvYAML, making it more SYCK-compliant, working with InputStreams and byte-arrays, and in the process get away from several of the steps above. So that's what I'm going to do. I hereby create JvYAMLb. It will only be a part of the JRuby codebase, but it will be reasonably separate, so it can be extracted for other purposes. I will not stop work on regular JvYAML, but will maintain both projects.

Since the objective of this new project is blazing speed, I will post some numbers on this now and again. But first I will show you the speed of the regular system. JvYAML's Scanner can scan an old gem source index (about 3.5MB) of 435654 tokens in about 1654ms. This is the baseline I'm going to use to test performance, and I'll post more on this as soon as the byte-based Scanner is ready to try out.

Bytes bites. Or maybe not.

Well, the byte arrays are in, for good and evil. We had to wrap them in a counterpart to StringBuffer, but backed by byte[] instead, since all that explicit allocation and deallocation was way unperformant.

Of course, we aren't seeing any performance benefits from this right now. The problem is that there is still many places that use IRubyObject#toString to get at the contents. That operation is very expensive right now, so gem installs are slower, for example. But we have good hopes on improving the situation, and many parts of the codebase have become much clearer without the need to do String-to-byte[] and byte[]-to-String all over the place.

Fractured blogging

My blogging in the future will be a little bit fractured (or more fractured, some might say), since I have been invited to write at Inside Java for APress. The address is http://java.apress.com and I do recommend that you subscribe to the feed if you're interested in Java or associated technologies. My first posting was about the two closure proposals for Java 7, and I will try to focus my posting there to be more Java specific, mostly in article format. But if I write anything I deem to be exceptionally good, I promise to link from here to there.

Go subscribe now!

Serial JRuby

Things are really moving along faster than ever in JRuby land. It's so fun! As my last entry told you, Hpricot is now available for JRuby (and Java) people. I need to share a few lines from the logs of yesterday evenings conversation at #jruby:

<headius> seeya ola!
* shellac does some xsl-ing, plays on the wii, 
  then finds ola got HPRICOT working in that time
<shellac> I'm wasting my life

Some would say that what I do with JRuby is a waste of life... Well, we'll see about that.

Anyway, what's happened in JRuby world since last week? First, and most important, Charles has changed our RubyString implementation. It used to be backed by either a Java String or a StringBuffer. The problem with both of these is that Ruby has a tendency to use Strings as byte buckets. And our code was riddled with encoding and decoding into and out of byte arrays. So Charles took the big step, converted RubyString to use a byte-array instead, and fixed all the bugs that he found by doing that. The result is a happier codebase, less encoding and possibly faster Zlib and IO operations. That's big.

Tom is working on removing visibility and refactoring scopes. That could have huge impact too.

This Sunday I merged and fixed some code that allow Ruby code to inherit from Java classes and override methods there, and this overriding will be seen if an instance is sent back to Java. I'm planning on using this for some interesting tricks with Java ContentHandler's, and this functionality is really, really, really important. But it's also complex, since it requires generating bytecode at runtime. Fun, but hard. But now it's in trunk, and it's time to find the bugs in it and fix them.

I also need you to go read what Jonas Bonér has done with JRuby and OpenTerracotta. I could describe it here, but Jonas does a good job of it himself. So go there: http://jonasboner.com/2007/02/05/clustering-jruby-with-open-terracotta/. Very cool stuff, indeed!

So, the future is coming faster each day. JRuby will still conquer the world!

Hpricot goodness

This is just so cool, I cannot contain it. For those of you who haven't heard about Hpricot, it is one of why the lucky stiff's incredibly cool tools (which he probably will use to take over the world any day now...). It's HTML parsing goodness, very flexible, with the goal of being able to parse (and fix) everything that Firefox handles.

"So what?" you're probably asking... Well, Hpricot uses Ragel and some C code to achieve blinding speed. This means JRuby can't run it. Or I should say couldn't run it:

orpheus:~/workspace/jruby> jruby bin/gem install hpricot --source http://code.whytheluckystiff.net
Bulk updating Gem source index for: http://code.whytheluckystiff.net
Select which gem to install for your platform (java)
1. hpricot 0.5.110 (jruby)
2. hpricot 0.5.110 (mswin32)
3. hpricot 0.5.110 (ruby)
4. hpricot 0.5 (ruby)
5. hpricot 0.5 (mswin32)
6. hpricot 0.5.0 (ruby)
7. hpricot 0.5.0 (mswin32)
8. hpricot 0.4.99 (ruby)
9. hpricot 0.4.99 (mswin32)
10. hpricot 0.4.92 (ruby)
11. hpricot 0.4.92 (mswin32)
12. Skip this gem
13. Cancel installation
> 1
Successfully installed hpricot-0.5.110-jruby
Installing ri documentation for hpricot-0.5.110-jruby...
Installing RDoc documentation for hpricot-0.5.110-jruby...

That's right, Hpricot is now more promiscuous than any other gem with native parts.
What can you do with it? Well, I'm just going to point you to _why's own description of it. All he says at http://code.whytheluckystiff.net/hpricot/ will work fine in JRuby!

How did this come to be? Well, me and _why did some joint hacking, which was helped along by the fact that Adrian Thurston (the genius behind Ragel) recently added Java support to it. So, basically, most of the Ragel definition is exactly the same for both the C and the Java versions. The native code has been factored out, and both versions are buildable with rake from _why's code repository.

This is important. Don't think anything else. This strategy will, and can, be used for other gems with native parts. It's just a question of time.

JFokus post-mortem

Yesterday marked the first one day conference in Sweden focused on Java. All in all, it was a resounding success. I am personally very happy about how well everything ran. The only part that wasn't really up to par was the panel discussion.

So, a quick rehash of the day. First Simon Phipps did his thing in the keynote, speaking much about where Sun's open source initiative comes from. I would have liked some more specific info on the future though, but I guess that's hard for him to say...

After that, Thorbiörn Fritzon from Sun in Sweden started of the Language track with talking about some new features in Mustang, specifically some Swing topics and the JSR 223 support (scripting). After that he showcased a really cool stack-based calculator he'd hacked during the weekend, where you could attach the functioning of the buttons to different scripts, and the scripts could be implemented in any language on the classpath that supported JSR 223. The final showdown of calculator buttons was first a script in Haskell (using Jaskell) that pushed a lambda on the stack. This lambda curried addition, with the first object on the stack. Then another script written in Python popped the lambda from the stack, popped the next value and called the lambda with the value from the stack, in effect doing an addition the longer way...

After that it was time for my presentation. It felt really good and I believe most in the audience was satisfied.

After that I sat in on Dan Bergh Johnsson from Omegapoint talking about domain driven design, and how to refactor to find your domain. I found it quite elementary, and nothing really new.

Last, Jonas Bonér talked about OpenTerracotta, which is an incredibly cool tool for clustering your JVM applications, and by doing that achieving many different cool things. Look it up! I wish there had been more time for Jonas to get in depth about the implementation of Terracotta, though. Most interesting thing he told us was about a situation where they wanted about 50-80 clients pounding in a cluster at the same time, and they used Terracotta for this too, using a CyclicBarrier that was synchronized over all the JVM's. Very nice.

The panel debate wasn't that interesting. Most of the questions asked by the moderator didn't really achieve any discussion. Open sourcing Java doesn't seem like such a controversial topic anymore.

After the debate, the conference was over, but then BEA provided a pub evening with entertainment, so we stayed and networked a little, talked with interesting people and generally had fun. After that, six of us continued to a pub in Stockholm, called Monk's Cafe, where we drank good Belgium beer and talked until our throats were sore. A very interesting day, no doubts.

Current JRuby status - AKA The what's-cool-and-happening

This is just a small update on what's going on in JRuby development at the moment. The first and most important item is that Tom has merged his block-work to trunk. He has done a monumental achievement with a patch on over 17k lines. The most amazing thing is that my main test case for block problems (Camping) runs perfectly. Tom also reports some performance improvements, but that isn't the important thing. What this block patch is all about is to remove the block stacks from our ThreadContext, and instead just pass the current block along through the Java stack (in other words, as a parameter to the method call in question). This simplifies many things and also make it possible for our compiler to finally compile closures and blocks. Charles will soon continue the work on making this happening, but being able to do that means the JIT will work for vastly more of the code than it does at the moment.

The second work we're hard at right now is Rails compatibility. Tom is looking at ActiveRecord, Charles have improved Marshalling (with the side effect that the gem local source cache finally works) and also did some work on Multibyte::Chars. This work is going fast forward. My part in this is ActionPack, where we right now are down to 9 failures and 0 errors, from 23 failures and 8 errors. Those numbers are for 1150 tests and 5202 assertions, so there isn't much left to get this working.

My work on the YARV compiler continues, and I will soon post an update about this too.

Focus on Java, this Tuesday

This Tuesday is JFokus in Stockholm, where I will talk about Java, JRuby and RSpec and how to combine these in exciting ways. Let me tell you, I look forward to speaking about this. I believe there is something incredibly useful in combining technologies like this.

Of course, it would also have been great fun to take about where JRuby is at right now, since there is so much cool stuff going on at the moment. When the 0.9.3 release comes along, I bet many people will be surprised and as excited as I am about this. I will get back with more information what has happened in the codebase lately quite soon.

If you're going to JFokus, please say hi if you see me!

Advice

I usually don't like to use this blog for strictly personal reasons, but I will do an exception for once, to get advice from y'all.

Hypothetically, I would like to know what kind of work opportunities I should look for, if I was in a situation where my current work situation was alright, but not satisfactory (since I've been at the same place for 6 years.) Further, I'm interested in getting away from Sweden. The places I would most likely want to be situated in would be SF or London.

The picture is complicated by my lack of academic degrees. The kind of experience I have is strictly professional and recreational. (In which category should I put my work on JRuby?). Apart from that, my interests are in the region of work where I can use different languages, and will not be constrained by Java. I haven't done consulting, and maybe I should do that. On the other hand, I'm not sure that is the right thing for me either.

So, please. I need advice. What should I do with my professional life at this moment?

YARV tail call optimization in JRuby

After some further developments on the YARVMachine, I can report that JRuby trunk supports most of the optimized instructions that YARV makes use of. The big thing missing is the local cache optimization. But that is just an aside from the main course.

Because right now there is support in JRuby's YARVMachine to handle a limited form of tail call optimization. Specifically, if the last call in a method is to the method we're in and the receiver of that call is the same as the current self, tail call optimization will happen. That allows JRuby to run this script:

def fact(n, acc)
if n == 0
acc
else
fact n-1, n*acc
end
end

p fact(25000, 1)

That will explode in both YARV and MRI, long before getting anywhere, since the stack goes out. But this will work correctly in JRuby with -y, provided that the tail call optimization is actually turned on. To turn it on, use $JAVA_OPTS and set the variable jruby.tailcall.enabled to true, like this:

JAVA_OPTS="-Djruby.tailcall.enabled=true"
jruby -y testRecFact.rbc

Of course, you need latest trunk for this to work, and you need to compile the YARV file as described in my previous post. But provided you have done that, the above will actually work perfectly, without deepening the stack at all. The reason this is not turned on by default in the YARVMachine is that it isn't completely safe yet. There is one small kink, which is that if you redefine the current method within the current method, and then call self, this will not work as expected. It should be easy to add checks for this behavior, though, so that will come soon.

Of course, the big problem with something like this is the stack trace. How to handle it? The implementation behavior would make it look like those 25000 calls to fact was just a single one. In the same manner, trace methods won't get called.

But trunk will actually report when these tail calls have been done. Say you have a script like this:

def a(n)
if n > 20000
  raise "hell"
end
a(n+1)
end

a(0)

(it is designed to explode at a certain depth, of course.) In current JRuby trunk, executed with -y, this will generate an output that looks like this:

testRaise.rb:-1:in `a' TAIL CALL(20001): hell (RuntimeError)
       from testRaise.rb:-1

So it is quite obvious that the exception was raised in the 20001:th invocation of 'a'.

This work goes forward very fast, and it's funny to see progress happen this quickly. Yesterday SASADA Koichi visited the #jruby and #rubinius IRC channels, and we had some very interesting discussions on further collaboration. The future looks bright indeed, and this weekend I'll probably have time enough to take a first crack at a YARV compiler for JRuby.

Executing YARV (in JRuby)

I guess the title says it all. Oh, it doesn't? You want what? A tutorial on how to play with this? Well, alright then.

What you will need to get started is first and foremost JRuby trunk. Download and build that. You will also need either a very recent version of YARV or Ruby trunk. Download either and build that too. I have the source in $YARV_SRC and $RUBY_TRUNK_SRC. I've also installed them with suffixes, so I have ruby-yarv and ruby-trunk globally available. That's a quite good setup.

Before compiling some Ruby, we need to change the change the compile-script slightly. So open up the file in $YARV_SRC/tool/compile.rb and disable the peephole_optimization and inline_const_cache, since JRuby doesn't really support these features yet. If you use Ruby trunk, you also need to do a global search-and-replace in this file, from YARVCore to VM.

Now you're finished to compile something. The goal for this execution have been to get an iterative fibonacci sequence running, so save this code in a file called fib.rb:

def fib_iter_ruby(n)
  i = 0
  j = 1
  cur = 1
  while cur <= n
    k = i
    i = j
    j = k + j
    cur = cur + 1
  end
  i
end

puts fib_iter_ruby(5000)

Next, compile this file:

ruby-yarv $YARV_SRC/tool/compile.rb -o fib.rbc fib.rb

Modify as needed if you use Ruby trunk instead. Now, how to run this? With the y-flag to JRuby:

jruby -y fib.rbc

As you can see, simple files like this just work. Of course, there are much missing yet, but basic interpretation works really nice. Some more additions to the YARV machine will come quite soon, probably. But the next step will be to implement a compiler for YARV too. From that point it should be possible to execute Ruby-files in JRuby in several different ways; by regular interpretation, YARV compilation, YARV interpretation, Java bytecode AOT compiling, Java bytecode JIT compiling and also possibly Rubinius interpretation. The future is, as always, interesting.

Rubinius and Lisp

Yesterday on the #rubinius-channel, me, Evan and MenTaLguY had the beginnings of a discussion about a Lisp dialect for the Rubinius VM. Since this sound exactly like the kind of thing I've been thinking about for some time, I got excited. But anyway, Pat Eyler covered the issue much better and did a small interview with us that can be read here. Ruby is wonderful. Rubinius is sweet. With Lisp added to the mix, it will be better.

I'll probably follow this up with some more information and ideas, as soon it's fleshed out.

That Rails Thing

I need to join the fray. Pat Eyler has announced a competition together with APress. The first installment is about how Rails has made me a better programmer. Details to be found here.

So, let's get at it. Rails has inspired and affected me in numerous ways. You got to realize that I have been programming web applications in Java for way too many years. Before that I've done both ASP and PHP. Nothing ever felt natural. I've tried many of the LISP frameworks (and Uncommon Web is really cool). I've embraced continuation-based frameworks. But none felt so powerful, yet nonintrusive. Rails embodies the best parts of Ruby.

For me, Rails is very interesting for several reasons. First and foremost is the fact that it is an incredibly good framework. It's really amazing how good it is. And that in itself acts as an inspiration. To know that software can be this good makes me want to strive to perfect my API's, make my libraries more usable, and finding new and novel ways to improve my DSL's.

But that's only part of it. The flip side of this is that Rails is not perfect. In fact, there are myriad ways it can be improved. And that gives me hope, because it also means that no matter how good my code gets, it can always get better. And that means I'll never be totally bored!

In a very literal sense, Rails has made me a better programmer in another way. Since Ruby on Rails was the first Ruby-application to make it into Karolinska Institutet, it means I can thank Rails for being able to write more Ruby at work. And writing Ruby instead of Java must make you a better programmer, neh?

Of course, Rails is an excellent testing tool for seeing how far we have left with JRuby. It goes without saying that you become a better programmer by implementing a language...

I have several applications and libraries that I keep in mind when writing my code. Those libraries act as a gauge against which I measure my code. Code quality, testing ability, interface, sheer trickyness; in all these areas, Rails is one of the top frameworks I know.

Five Things.

Oh well. It has finally hit me, as I knew it would sooner or later. Charles O Nutter managed to nail me.

If you don't know what this is, it's called the "5 Things Meme" and it means I'll tell you 5 things about me that you probably don't know, and then pass the baton on to 5 other people. We'll see about that.

• I am an "academic failure" in that I haven't even finished the Swedish equivalent to High School (in Swedish it's Gymnasium). There is no thing about computers I have learned in school. And even though I have no education, I actually work centrally at Karolinska Institutet, which is one of the more esteemed universities in Sweden.
• I am an amateur musician. I play guitar, some piano, sing quite a lot and play a few other instruments (but not very good). I create some music, but nothing that has been finished enough to make public. Until recently I was a singer in a Swedish barbershop chorus called the Entertainmen. We came 14:th on the World Barbershop Competition in Salt Lake City 2 years ago. I used to sing Lead, but switched to Bari.
• I train martial arts. I have been doing so for most of my life, but not with enough regularity to attain a black belt yet. What I've been training for the last few years is called Bujinkan Budo Taijutsu, and it seems I will stay with it.
• I am a vegetarian, and has been for the past 10 years. I don't eat meat, fowl or fish, but I do eat egg and dairy products.
  
• I live in a former mental hospital, in a beautiful park 20 minutes from central Stockholm. The hospital was built 1908 and looks very much like a castle.

So. Now I've done my share. What's left is to give this burden to someone else. Since I don't like the concept of chain *anything* I will introduce a small twist by tagging 5 people who probably won't do it, for several reasons. But if they do it, it will be all the more interesting. So, without further ado:

• Steve Yegge
• Jonathan Schwartz
• why the lucky stiff
• Matz
• Martin Fowler

I will present at JFokus

A few weeks ago I noted that Sun in Sweden announced on the JavaForum meetup that they were going to have a full day Java conference in January. This conference is called JFokus and will have 4 different tracks with 4 speakers on each track. I will feature in one of these tracks, talking about testing Java code with JRuby and RSpec, and also how to build a DSL for doing database integration testing using Ruby. I believe it will be very interesting, and the rest of the day looks incredible nice too.

So, if you're in Sweden, Jan 30:th, look into it!

More information can be found at the official site: http://www.jfokus.se/.

Profile support in JRuby trunk

As noted a few posts ago, Sandbox has been ported and run in JRuby. You did need to use a special version, only available from a branch in the JRuby source tree. But now this support has been added to trunk. This means you can check out the latest version of JRuby trunk, gem install javasand, and everything will work.

For more info, see my previous post on this: here.

A Personal blog

Due to lots and lots and lots of stuff happening in my life right now, I haven't had the time to write anything substantial here in a while. Hopefully, this will change.

I have also decided to start a new blog, for more... personal musings. Books, music, movies. Stuff like that. Nothing that would interest the regular readers of this blog, probably. But anyway, here's the address: http://olabini.blogspot.com.

Ah, right: Have a merry christmas and a happy new year, everyone!

Another neat trick

Another sweet thing, that I've actually wondered idly about from time to time, is how you could get "provided?"-style parameters in Ruby. If you have used a language with this functionality, like Common Lisp for example, you know that this can be highly useful.

And just the other day, while reading Hal Fultons excellent (but poorly proof read) The Ruby Way (2nd ed) I found the way. Ergo, it looks like this:

def foo(bar, baz = (baz_provided=true; 42))
 p [bar,baz,baz_provided]
end

foo(13,47)
foo(13)
foo(13,42)

This will provide the output:

[13, 47, nil]
[13, 42, true]
[13, 42, nil]

which illustrates the usage pretty well. The magic, of course, lies in the fact that default parameters are evaluated, just the same as anything else. You could do some very crazy things inside that default value specification if you wanted. Anyway, just a nugget.

A very small Ruby method

I haven't had much time or inclination for blogging lately. Not much happening at the moment, actually. But I came up with one small thing I wanted to document. Just a practical thing for certain situations. Basically it's a with-method, that works fairly well. It's nothing magical and the trick is basic. It's more or less an alias, actually:

module Kernel
 def with(obj = nil, &block)
   (obj || self).instance_eval &block
 end
end

with("abc") do
 puts reverse
end

"abc".with do
 puts reverse
end

As you can see, insstance_eval can be used like JavaScript or VB's with. This is nice for the simple reason of documentation. I find this usage much easier to read and understand than most usages of instance_eval that I've seen.

So, that's it for today. I'll probably be back soon with some recent JRuby developments too.

The Curse of Camping: The story continued

It's time for another installment in the Camping saga. The last time I wrote about 1.5, and some things that didn't work properly in JRuby. This time, the announcement is once again this: Camping 1.5 works in JRuby. It works very well, actually. To show this, I will make it very easy for you to follow along in how to achieve it.

First, have JRuby installed, either from trunk or version 0.9.2 if that version has been released when you read this.

Secondly, gem install ActiveRecord, Camping and all their requirements.

Thirdly, gem install ActiveRecord-JDBC.

Fourth, put you MySQL JDBC driver on your classpath (or any other JDBC driver you care to use).

Download blog.rb from the Camping examples here. Modify it by adding the line

require 'jdbc_adapter'

after the other require's at the head of the file.

Finally, you need to create a database file, since JRuby doesn't support the standard choice of camping (which is SQLite3). To do this, create a file that's named (on Linux) $HOME/.campingrc, or on Win32 %APPDATA%/Campingrc. (%APPDATA% is on most Win32-boxes something like c:\Documents and Settings\olagus\Application Data). This file should contain your database definition in standard ActiveRecord fare. My example looks like this:

database:
 adapter: jdbc
 driver: com.mysql.jdbc.Driver
 url: jdbc:mysql://localhost:3306/blog
 username: blog
 password: blog

Of course, you should use your own individual settings and driver information.

Camping does only require that the database exists. It will add migrate the blog application automatically, and as soon as you've started the application by doing:

jruby $JRUBY_HOME/bin/camping blog.rb

it will listen on 3301 and be ready to serve an example blog application.

Have fun and enjoy!

Freaky freaky Sandbox has come to JRuby

I would like to announce the first release of JavaSand, which brings _why's spooky Sandbox to JRuby.

For those of you who hasn't played with Sandbox, it allows you to create a new area for executing Ruby where you have control on which classes are loaded and available. It's good for lots of things. It can be made very secure, for example. Or you can load several applications in different Sandboxes and they won't interfere with each other in any way.

Now, in MRI, this is done through some fairly involved swapping of symbol tables. In JRuby things are quite a lot easier. Actually, it's only a matter of loading the right classes and remove all methods not allowed.

So, how do you get to use sandbox from JRuby? First of all, you need a special JRuby, that can be checked out and compiled from svn://svn.codehaus.org/jruby/branches/sandbox. This functionality will come to regular JRuby to, but not until after 0.9.2 has been released. So, when you have this special version downloaded, you just install the RubyGem called javasand, and you're set to go. If you would like a fun example to try out, you can download sandbox_server directly from _why's code repository http://code.whytheluckystiff.net/svn/sandbox/trunk/bin/sandbox_server. To run it:

jruby -rubygems sandbox_server

and you can connect to your machine at port 5000, press enter, and you have an IRB session. Wow!

So, the project JavaSand is part of JRuby-extras and the source is in that repository, at RubyForge! Go ahead and test it now!

The FINAL OpenSSL post?

Possibly.

I've checked in all functionality I will add to OpenSSL support in JRuby at this point. Of course, there will be more, but not concentrated in a spurt like this. Tomorrow I will modify the build process and then merge everything I've done into trunk.

Let's back up a little. What have I accomplished? This: All OpenSSL tests from MRI run (except PKCS#7). That includes tests of SSL server and SSL client. Simple https-request also works. This is sweet. Everything else there is tests for in Ruby works. But... this is also the problem. Roughly half of Ruby's OpenSSL library is not tested at all. And since the current OpenSSL initiative from my part is based on tests, I haven't done anything that isn't tested for.

So, some things won't work. There is no support for Diffie-Hellman keys right now, for example. Will be easy to add when the time comes, but there isn't any testing so I haven't felt the need.

The only thing not there, as I said, is PKCS#7. That was just too involved. I'll take care of that some other time, when someone says they want it... Or someone else can do it? =)

So, what this boils down too is that JRuby trunk will have OpenSSL support sometime tomorrow. Hopefully it will be useful and I can get on to other JRuby things. I have a few hundred bugs I would like to fix, for example...

Oh yeah, that's true. Tomorrow will also be YAML day. I'll probably fix some bugs and cut a new release of JvYAML. It's that time, the bug count is bigger than it was, and JRuby needs some fixes. So that's the order of day for tomorrow. First OpenSSL and then YAML. Any comments on this, please mail or comment directly here.

G'night.

JavaForum: last night

And now it's over. Last night was the last presentation for me in a while. Over 200 people where there and the place was packed. I must say I'm very pleased with how my part went. I felt really good, much flow and the demonstrations went perfect.

Rob Harrop and Adryan Colyer talked about Spring 2.0 and AOP. It was fairly interesting, but since I've done some work with AOP before I felt that the level was a bit to introductory. No real meat.

JavaForum announced a day focused on Java in January, with four tracks and some great speakers. Looks very nice.

JRuby in Stockholm, JavaForum on Tuesday

On Tuesday it's time for the big JavaForum in Stockholm. I'm looking very much forward to it and have spent some hours today sharpening the presentation. Of course it will be very interesting to see Rob Harrop too.

In other news, something quite exciting is probably just around the corner...

JRuby presence at JavaOne 2007

The call for papers went out two days ago, and we already have several good proposals in line that will be submitted to JavaOne. Hopefully we will get many interesting talks this year. As far as I know, there are at least 5 JRuby-specific proposals submitted, and there will hopefully be more.

I am sending in two different proposals this year, one about JRubyEE (it's called "Agile Enterprise Development with JRuby"). That will be interesting to do and I really believe there is much for Java enterprise development to use in JRuby.

The second proposal is for a BOF about several JVM languages called "Lambda on the JVM: JRuby, Jython and Lisp". I'll talk about how to utilize different JVM alternative languages to do interesting and useful things. Since this is my venue, I will specialize on languages that are derived from Lisp (implicitly or explicitly). (I include both Ruby and Python in this category).

Hopefully at least one of the proposals will get through.

The obligatory meta-blog post.

So, this is post number 100, since I started blogging. And that's almost exactly one year ago. Isn't that interesting? I've been writing something here each 3.65'th day. Often it's been uninteresting, very often indifferent, but sometimes I have manage to write something that people liked. So, I'm happy about this small anniversary.

So, I plan to do what almost every blogger do at least once: turn the writing process inwards and investigate why people blog, why the person in question blogs, and perhaps, if we are really META, why the current blog post is actually written. Hopefully I'll know why I've written this by the end of this post. Otherwise, it'll probably be something indifferent or boring.

Let's begin. "Why blog?", the general question. When I started out, I had no real goal. I knew that I wanted to get better at technical writing and that was largely it. I hoped I would get some new contacts and interesting discussions but the base reason was for the writing ability. As such, I wouldn't have to write public; I could hone my writing skill in my basement and never publish anything to the (sometimes scary) scrutinizing eyes of the public. But that's the problem. If I didn't publish it, I wouldn't have as much incentive to become better. Now I absolutely have to get better at writing, otherwise I will feel ashamed about the filth that will for all time be remembered by Google.

That's reason number one. The second reason, that I've come to realize this last year but didn't know when I started out, is that I learn by writing about a subject. If it's a technical topic I basically have to research the topic quote thoroughly, so I won't look like a fool. And that is really good, that is incredibly good. I have learned very much by writing about things I find interesting. The first learning when I research and think about what I should write, and the second learning when people read it and comment and correct me.

Reason number three can be thought of as hubris. I believe that some of the things I know can be useful for other to know, and I believe that some of the problems and bugs I uncover in open software should be documented somewhere so others won't have to go through the same bug finding escapades as I did on that subject.

And here's where we come to the point. The point is that if you're working with software (or with anything where information sharing is important, really), you should write about. Because the writing benefits you and it benefits me. If you find a problem, write about it. Doesn't matter if someone actually reads it or not, for the most important aspects of writing is about yourself.

Steve Yegge wrote about this in his "Blog Or Get Of The Pot", and also in one of this older blogs ("Why you should blog" I believe it's called).

Coda: Why did I write this post? To make it clear in my head why I blog. To make it obvious and explicit why I do it. So that's what this post is about, and it's a recursive post since it talks about itself. I'm satisfied.

Some notes from the Stockholm Rails meet

Last night (Wednesday) about 35 developers with an unhealthy interest in Rails met up in Stockholm, at Valtech's offices, to share some experiences and talk shop.

It was very interesting and fun to meet people I can relate to. We ended up talking programming languages for a few hours after the main event ended.

Peter Marklund did a presentation on a CRM system in Rails, and Christian and Albert from Adocca talked about caching, and handling a Rails application that needs to scale into the millions. Nice stuff.

Last, I did an improvised presentation on LPW, a small Rails application that is fed its main data through Web Services instead of ActiveRecord. I believe it went fairly well, even though I had no slides and almost no preparation... =)

Speakings this and next week

Just a quick notice. I will be talking this wednesday at a Rails group in Stockholm. The subject will (for once) not be about JRuby, at least not primarily. Instead I will talk about an application called LPW, which is interesting since it doesn't use a database as the main data feed, but a bunch of Web Services that are implemented by a third party and deployed with EJB's in Axis. I had all kinds of challenges, but in the end the result became very good. I will talk a little about what you can expect when trying to interface with Java in this way, and good tricks on how to handle an alternative to ActiveRecord as model.

I've written a bit about LPW sometime this summer, if anyone is interested.

Next week is the JavaForum meet up in Stockholm, and I will speak after Rob Harrop. The subject is JRuby and the day is Tuesday. As far as I know, the meet up is full.

The presentation will be pretty standard JRuby fare for Java developers.

Further OpenSSL progress

Sometimes it feels like the progress I've done with the OpenSSL library is almost swallowed up by all the new functions I've found that needs to be implemented. It's an uphill battle.

implementation doesn't handle the But, I'm happy to say that the latest digression is finished. I have successfully implemented the X509_STORE-family of functions plus all dependencies. In the end I had to create a wrapper for X509Certificate and my own PEM-reader and PEM-writer, since BouncyCastle'sOpenSSL specific aux trust information that can be appended to these certificates.

But anyway, that means there is a Java implementation of X509_STORE, X509_STORE_CTX, X509_VERIFY_PARAM, X509_LOOKUP, X509_LOOKUP_METHOD, X509_TRUST, X509_PURPOSE and a whole slew of others too. About the only thing I didn't implement was X509_POLICY_TREE. That was just too much. I'll tackle that hurdle if it seems Ruby needs it.

So, what is the status then? Coming in at almost 10 000 lines of code there is only three real parts left to do. Or, three parts left that have MRI test cases, I should say. Since there are quite a few files not tested in the MRI implementation. Like DH keys. Wow. But I ignore that for now. The current goal is the OpenSSL::X509::Store-family, the OpenSSL::PKCS7-family, and the SSL-stuff, the rest of the way. So, wish me luck.

An Emacs diversion: Font sizes

After a few weeks of very intense JRuby-OpenSSL hacking, I felt the need to do something different, so I've spent a few hours with my slightly rusty Emacs Lisp skills, trying to fix something that I really need. Namely, control over font-size and fonts in Emacs, on Linux. I want it inside Emacs and customizable by EL. To my surprise I couldn't find anything like that anywhere.

For me personally, it's necessary when presenting, since I usually code with a small font in Emacs, the code will be totally unreadable when presenting. And since I don't have a fancy MacBook Pro, I need to be able to zoom in and out inside Emacs.

Presto, it wasn't easy, but I've managed it. For some reason, font handling seems quite backward in Emacs. I had to extract the current font, and then split it and join the new array together again. Not neat and my way of doing it is not the best. But, for your pleasure, here is the code to do it, and also some code that establishes a font ring of the standard fonts in different sizes that can be walked through:

(defun inc-font-size ()
 (interactive)
 (let* ((current-font (cdr (assoc 'font (frame-parameters))))
        (splitted (split-string current-font "-"))
        (new-size (+ (string-to-number (nth 7 splitted)) 1))
        (new-font (concat (nth 0 splitted) "-"
                          (nth 1 splitted) "-"
                          (nth 2 splitted) "-"
                          (nth 3 splitted) "-"
                          (nth 4 splitted) "-"
                          (nth 5 splitted) "-"
                          (nth 6 splitted) "-"
                          (number-to-string new-size) "-*-"
                          (nth 9 splitted) "-"
                          (nth 10 splitted) "-"
                          (nth 11 splitted) "-*-"
                          (nth 13 splitted))))
   (if (> (length splitted) 14)
       (dotimes (n (- (length splitted) 14))
         (setq new-font (concat new-font "-" (nth (+ n 14) splitted)))))
   (set-default-font new-font t)
   (set-frame-font new-font t)))

(defun dec-font-size ()
 (interactive)
 (let* ((current-font (cdr (assoc 'font (frame-parameters))))
        (splitted (split-string current-font "-"))
        (new-size (- (string-to-number (nth 7 splitted)) 1))
        (new-font (concat (nth 0 splitted) "-"
                          (nth 1 splitted) "-"
                          (nth 2 splitted) "-"
                          (nth 3 splitted) "-"
                          (nth 4 splitted) "-"
                          (nth 5 splitted) "-"
                          (nth 6 splitted) "-"
                          (number-to-string new-size) "-*-"
                          (nth 9 splitted) "-"
                          (nth 10 splitted) "-"
                          (nth 11 splitted) "-*-"
                          (nth 13 splitted))))
   (if (> (length splitted) 14)
       (dotimes (n (- (length splitted) 14))
         (setq new-font (concat new-font "-" (nth (+ n 14) splitted)))))
   (set-default-font new-font t)
   (set-frame-font new-font t)))

(defvar *current-font-index* 0)

(defconst *font-ring* '(
                       "-urw-nimbus mono l-regular-r-normal--15-*-88-88-p-*-iso8859-1"
                       "-urw-nimbus mono l-regular-r-normal--17-*-88-88-p-*-iso8859-1"
                       "-Adobe-Courier-Medium-R-Normal--14-*-100-100-M-*-ISO8859-1"
                       "-Adobe-Courier-Medium-R-Normal--16-*-100-100-M-*-ISO8859-1"
                       "-Adobe-Courier-Medium-R-Normal--18-*-100-100-M-*-ISO8859-1"
                       "-Adobe-Courier-Medium-R-Normal--20-*-100-100-M-*-ISO8859-1"
                       "-Adobe-Courier-Medium-R-Normal--22-*-100-100-M-*-ISO8859-1"
                       "-Adobe-Courier-Medium-R-Normal--24-*-100-100-M-*-ISO8859-1"
                       "-Adobe-Courier-Medium-R-Normal--26-*-100-100-M-*-ISO8859-1"
                       "-Adobe-Courier-Medium-R-Normal--28-*-100-100-M-*-ISO8859-1"
                       "-Adobe-Courier-Medium-R-Normal--30-*-100-100-M-*-ISO8859-1"
                       "-Adobe-Courier-Medium-R-Normal--32-*-100-100-M-*-ISO8859-1"
                       "-Adobe-Courier-Medium-R-Normal--34-*-100-100-M-*-ISO8859-1"
                       "-Adobe-Courier-Bold-R-Normal--14-*-100-100-M-*-ISO8859-1"
                       "-Adobe-Courier-Bold-R-Normal--16-*-100-100-M-*-ISO8859-1"
                       "-Adobe-Courier-Bold-R-Normal--18-*-100-100-M-*-ISO8859-1"
                       "-Adobe-Courier-Bold-R-Normal--20-*-100-100-M-*-ISO8859-1"
                       "-Adobe-Courier-Bold-R-Normal--22-*-100-100-M-*-ISO8859-1"
                       "-Adobe-Courier-Bold-R-Normal--24-*-100-100-M-*-ISO8859-1"
                       "-Adobe-Courier-Bold-R-Normal--26-*-100-100-M-*-ISO8859-1"
                       "-Adobe-Courier-Bold-R-Normal--28-*-100-100-M-*-ISO8859-1"
                       "-Adobe-Courier-Bold-R-Normal--30-*-100-100-M-*-ISO8859-1"
                       "-Adobe-Courier-Bold-R-Normal--32-*-100-100-M-*-ISO8859-1"
                       "-Adobe-Courier-Bold-R-Normal--34-*-100-100-M-*-ISO8859-1"
                       "-Misc-Fixed-Medium-R-SemiCondensed--10-*-75-75-C-*-ISO8859-1"
                       "-Misc-Fixed-Medium-R-SemiCondensed--12-*-75-75-C-*-ISO8859-1"
                       "-Misc-Fixed-Medium-R-SemiCondensed--13-*-75-75-C-*-ISO8859-1"
                       "-Misc-Fixed-Medium-R-Normal--13-*-75-75-C-*-ISO8859-1"
                       "-Misc-Fixed-Medium-R-Normal--14-*-75-75-C-*-ISO8859-1"
                       "-Misc-Fixed-Medium-R-Normal--15-*-75-75-C-*-ISO8859-1"
                       "-Misc-Fixed-Medium-R-Normal--16-*-75-75-C-*-ISO8859-1"
                       "-Misc-Fixed-Medium-R-Normal--17-*-75-75-C-*-ISO8859-1"
                       "-Misc-Fixed-Medium-R-Normal--18-*-75-75-C-*-ISO8859-1"
                       "-Misc-Fixed-Medium-R-Normal--19-*-75-75-C-*-ISO8859-1"
                       "-Misc-Fixed-Medium-R-Normal--20-*-75-75-C-*-ISO8859-1"
                       "-Schumacher-Clean-Medium-R-Normal--12-*-75-75-C-*-ISO8859-1"
                       "-Schumacher-Clean-Medium-R-Normal--13-*-75-75-C-*-ISO8859-1"
                       "-Schumacher-Clean-Medium-R-Normal--14-*-75-75-C-*-ISO8859-1"
                       "-Schumacher-Clean-Medium-R-Normal--15-*-75-75-C-*-ISO8859-1"
                       "-Schumacher-Clean-Medium-R-Normal--16-*-75-75-C-*-ISO8859-1"
                       "-Schumacher-Clean-Medium-R-Normal--17-*-75-75-C-*-ISO8859-1"
                       "-Schumacher-Clean-Medium-R-Normal--18-*-75-75-C-*-ISO8859-1"
                       "-Sony-Fixed-Medium-R-Normal--14-*-100-100-C-*-ISO8859-1"
                       "-Sony-Fixed-Medium-R-Normal--16-*-100-100-C-*-ISO8859-1"
                       "-Sony-Fixed-Medium-R-Normal--18-*-100-100-C-*-ISO8859-1"
                       "-Sony-Fixed-Medium-R-Normal--20-*-100-100-C-*-ISO8859-1"
                       "-B&H-LucidaTypewriter-Medium-R-Normal-Sans-14-*-100-100-M-*-ISO8859-1"
                       "-B&H-LucidaTypewriter-Medium-R-Normal-Sans-16-*-100-100-M-*-ISO8859-1"
                       "-B&H-LucidaTypewriter-Medium-R-Normal-Sans-18-*-100-100-M-*-ISO8859-1"
                       "-B&H-LucidaTypewriter-Bold-R-Normal-Sans-20-*-100-100-M-*-ISO8859-1"
                       "-B&H-LucidaTypewriter-Bold-R-Normal-Sans-24-*-100-100-M-*-ISO8859-1"
                       "-B&H-LucidaTypewriter-Bold-R-Normal-Sans-30-*-100-100-M-*-ISO8859-1"
                       "-B&H-LucidaTypewriter-Bold-R-Normal-Sans-34-*-100-100-M-*-ISO8859-1"
                           ))

(defun font-next ()
 (interactive)
   (let ((len (length *font-ring*))
         (next-index (+ *current-font-index* 1)))
     (if (= next-index len)
         (setq next-index 0))
     (setq *current-font-index* next-index)
     (message (concat "setting " (nth *current-font-index* *font-ring*)))
     (set-default-font (nth *current-font-index* *font-ring*) t)
     (set-frame-font (nth *current-font-index* *font-ring*) t)))

(defun font-prev ()
 (interactive)
   (let ((len (length *font-ring*))
         (next-index (- *current-font-index* 1)))
     (if (= next-index 0)
         (setq next-index (- len 1)))
     (setq *current-font-index* next-index)
     (set-default-font (nth *current-font-index* *font-ring*) t)
     (set-frame-font (nth *current-font-index* *font-ring*) t)))

(defun font-current ()
 (interactive)
 (cdr (assoc 'font (frame-parameters))))

(defun font-set (ix)
 (setq *current-font-index* ix)
 (set-default-font (nth *current-font-index* *font-ring*) t)
 (set-frame-font (nth *current-font-index* *font-ring*) t))

(provide 'fontize)

I also bound these methods to keys, like this:

(global-set-key [?\C-+] 'inc-font-size)
(global-set-key [?\C--] 'dec-font-size)
(global-set-key [?\M-+] 'font-next)
(global-set-key [?\M--] 'font-prev)

Hope this helps someone in the same situation.

Nooks and Crannies of Ruby

There are many small parts of Ruby, tips, tricks and strange things. I thought that I would write about some of the more interesting of these, since some of them are common idioms in the Ruby community. The basis for the information is as always from the Pick-axe, but how these things are used in real life comes from various places.

The splat operator

The asterisk is sometimes called the splat operator when not used for multiplication. It is used in two different places for opposite cases. When on the right hand side of an expression, it is used to convert an array into more than one right hand value. This makes splicing of lists very easy and nice to do.

a,b,c = *[1,3,2]

Second, it's used at the left hand side to collect more than one right hand value into an arra

*a = 1,3,2

This makes no difference if you're calling a method or assigning variables. What matters is as usual with programming languages; that there is a left hand side and a right hand side (lhs and rhs from now on):

def foo(a,*b)
p b
end

foo 1,2,3,*[4,5,6]

This is all old news, and not very exciting. It's useful and the basis for some niceties, but nothing overwhelming. The thing that is really nice about the rhs version of the splat operator is what it does if the value it's applied to isn't an array. Basically, the interpreter first checks if there is a to_ary-method available. If not, it goes for the to_a method. Now, Kernel has a default to_a-method so all objects will respond to to_a. This method is deprecated to call directly, though, but if called through splat or Kernel#Array it doesn't generate a warning. So:

a = *1

will result in the same thing as

a = 1

except for jumping through some unnecessary hoops underneath the covers. But say that you have an object that implements Enumerable and you want to do something with. Maybe transform a Hash into an array of 2-element arrays, you can do it like this:

*a = *{:a=>1,:b=>2}

Now, this still isn't that useful. Oh, it's slightly useful but there is a method in Hash that does this too. But say that we have a file object:

*a = *open('/etc/passwd')

Since File includes Enumerable, it also has a to_a method which creates the array by using each to iterate and collect all elements. In this case all the lines in the file.

def foo(*args)
bar(*args)
end

Camping uses the splat operator at many places, mostly with the common idiom to take any arguments offered and passing them all on as separate arguments again:

Symbols and to_proc

I hesitate to use the word neat, but I can't really find anything that better describes the sweet, sweet combination of symbols and to_proc. I'm going to show you a small example of how it's used before I explain this very common practice:

[1e3,/(foo)/,"abc",:hoho].collect &:to_s

Now, this code will not run without a small addition to your code base. But first of all, let's just walk through the code. First we define a literal array that contains four elements of different type. One Float, one Regexp, a String and a Symbol. Then we call collect to make a new array out of this. But where we usually provide collect with a block, we instead see the ampersand that symbolizes that we want to turn a Proc-object into a block argument for a method. But what comes next is not a variable, but a symbol. So, what happens? Well, the ampersand checks if the value provided to it is a Proc, and if not it calls to_proc on the value in question, if such a method is defined. And how should this method look? Like this:

class Symbol
 def to_proc
lambda { |o| o.send(self) }
 end
end

Now, this method is nothing much. But it employs some fun trickery. It first creates a Proc by calling Kernel#lambda with a literal block. This block takes one argument, and the block calls the method send on the argument with itself as argument. As self in this case would be a symbol, and specifically the symbol :to_s in the above example, the end result is that the Proc returned will call to_proc on each object yielded to the block. So, with this explanation it's easier to understand what the first example does. In effect it is exactly the same as

[1e3,/(foo)/,"abc",:hoho].collect {|v| v.to_s}

but without that nasty duplication of the v-argument. It's not a big saving, but many small savings...

I recommend installing facets, which include numerous small, nice solutions like this. They can also be required separately, so if you have facets installed, just require 'facet/symbol/to_proc' to get this specific functionality included.

Using operators as method names

Ruby allows much more operators to be redefined than most languages. This makes some interesting tricks possible, but most importantly it can make your code radically more readable. An excellent example of this can be found in the net/ldap-library (available as ruby-net-ldap from RubyGems). Now, LDAP uses something called filters for searching, and the syntax for filters are basically prefix notation with ampersand, pipe and exclamation mark for and, or and not, respectively. Now, with the net/ldap-library you can define a combined filter like this:

include Net
f = (LDAP::Filter.eq(:cn,'*Ola*') & LDAP::Filter.eq(:mail,'*ologix*')) |
    LDAP::Filter.eq(:uid,'olagus')

This defines a filter that basically says: find all entries where cn is '*Ola*' and mail is '*ologix*' or uid is 'olagus'. This is very readable thanks to the infix operators, that for everyone who knows LDAP will be easy to understand.

The next example comes from Hpricot, where _why puts the slash to good use:

doc = Hpricot(open("http://redhanded.hobix.com/index.html"))
(doc/"span.entryPermalink").set("class", "newLinks")

Note how neatly doc/"span..." fits in, and it looks like XQuery, or any other path query syntax. But it's just regular Ruby code and the slash is just method call. I'm really sad that /. isn't allowed as a method in this way... =)

Now, ackording to the Pickaxe, all of these infix operators will be translated from arg1 op arg2 into arg1.op(arg2). But Ruby still needs to be able to parse everything. This means that most operators need to have one required argument. Trying this with a home defined *-operator will not work:

x = a *

But, an experimental syntax for importing packages in JRuby actually used this effect:

import java.util.*

This is just a simple exploatation of the fact that * is a regular method name and used like this will be parsed by Ruby like that too, which means it doesn't need an argument. So, which operators are available for your leisure? Ackording to the Pickaxe, these are [], []=, **, !, ~, + (unary), - (unary), *, /, %, +, -, >>, <<, &, ^, |, <=, <, >, >=, <=>, ==, ===, !=, =~, !~.
Note that the method names when implementing the unary + and - is +@ and -@:

class String
 def -@
swapcase
 end
end

The most important thing to remember when reusing operators like this is to not overdo it. Use it where it makes sense and is natural but not elsewhere. Remember that Ruby code should follow the principle of least surprise. The above example of using unary minus to return a swapcased version of the string is probably not obvious enough to warrant its use, for example.

Using lifecycle methods to simplify daily life

Inversion of control is all the rage in the Java world right now, but using callbacks of call kinds have always been a great way to make readable and compact. The Observer pattern is used in many places, and I suspect it's implemented without any knowledge of the pattern in most places.

Ruby contains a few callback methods and lifecycle hooks that make life that much easier for the Ruby library writer. Probably the most useful of these are Module#included. Basically, this is a method you define like this:

module Enumerable
def self.included(mod)
puts "and now Enumerable has been used by #{mod.inspect}..."
end
end

It will be called every time a module is included somewhere else.

There are other callbacks that can be useful. Module#method_added, Module#method_removed, Module#method_undefined and counterparts for Kernel with singleton prefixed. Class#inherited is interesting. Through this you can actually keep track of all direct subclasses of your class and with some metaprogramming trickery (basically writing a new inherited for each subclass that does the same thing) you can get hold of the complete tree of subclasses. If you want that for some reason. I would for example use this approach for Test::Unit, rather than iterating over ObjectSpace. But I guess that's a matter of taste.

Class variables versus Class instance variables

This is one thing that always trips people up. Including me. Class variables are special variables that are associated with a class. They are referenced with two at-signs and a name, like @@name. So far, it's simple. But classes are also instances of Class, which means that these instances can have regular one-at-sign instance variables. These are not the same thing. Not at all. Something like this:

class Foo
@@borg = []
@me = nil

def initialize
@me = self
Foo::add_borg
end

def self.add_borg
@@borg << @me
end
end

will result in a @@borg-list filled with nils. This is because the first @me refers to an instance variable in the Foo instance of Class; not the @me instance variable associated with an instance of the Foo-class.

Condensed lesson: Class have instance variables of themselves, these are rarely useful; they usually contribute to hard-to-find-errors. And don't confuse them with class variables which is a totally different kind of beast.

Shortcuts: __FILE__ and ARGF

Ruby contains a myriad of shortcuts, many influenced from Perl and other invented to make it easier to write condensed programs. The regexp result globals are always good to have, but there are other that can be very useful too. Two that I like most are __FILE__ and ARGF. __FILE__ is also part of a very, very common idiom that the Pickaxe details. Combined with the global $0 it makes it easy to differ execution when a file is required, and when it's executed. Basically, $0 contains the name of the file that has been executed. In C this would be argv[0]. __FILE__ is the full filename of the file the code can be found in. If these are the same, the current file is the one asked to execute. This is useful in many places. I use it often in gemspecs:

if $0 == __FILE__
Gem::manage_gems
Gem::Builder.new(spec).build
end

If I run the file above with gem build, this part will not execute, but if I execute the file directly, it will run.

Matz sometimes likes to show how to implement the UNIX utility cat in Ruby:

puts *ARGF

This combines tip number uno in this blog entry with the constant ARGF. ARGF is a nice special object that when you reference it will open all the files named in ARGV. If you have any options in your ARGV you'd better remove them before referencing ARGF, though. Basically what you get when referencing ARGF is a file handle to the files named on the command line. And since a File has Enumerable and thus to_a, splat will read all the lines in all the files and combine them into an array and then splay the array into the call to puts which will print each line. Here you are, cat!

There are other globals and constants available, but most aren't as useful as the previously named. For example you can use __END__ on an empty line, and the code interpolation will stop there and the rest of the file will be available as the constant DATA. I haven't seen anyone use this. It's a remnant from when Ruby was a tool to replace Perl, and the other scripting tools in UNIX.

Everything is runtime

Basically, the whole difference in Ruby compared to compiled languages is that everything happens at runtime. Actually, this difference can be seen when looking at Lisp too. In Common Lisp there are three different times when code can be evaluated: at compile-time, load-time and eval-time. In Java class-structure is fixed. You can't change class structure based on compile parameters (oh boy, sometimes I miss C-style macros). But in Ruby, everything is runtime. Everything happens at that time (except for constants... this is a different story). This means that class definitions can be customized based on environment. A typical example is this:

class Foo
include Tracing if $DEBUG
end

This class will include some methods when the -d flag is provided, and others when it's not. Basically there isn't much syntax in Ruby that couldn't be implemented in the language itself. A class declaration can be be duplicated with

Class.new(:name) do
#class declarations go here
end

And almost all parts of a method-definition with def can be provided with define_method. The glaring mismatch (blocks) will be corrected with 1.9. Except for that, it's just sugar. If statements could be implemented with duck typing/polymorphism:

class TrueClass
def if(t,f)
t.call
end
end

class FalseClass
def if(t,f)
f.call if f
end
end

x = true

x.if lambda{ puts "true" }, lambda{ puts "false"}

And that's the real Lisp inheritage of Ruby. There really isn't any essential syntax. Everything can be implemented with the basics of receiver, message, arguments, and blocks. Just remember that. It's the basis for all useful metaprogramming. There is no compile-time. Everything can change. "There is no spoon".

Announcing ActiveRecord-Mimer 0.0.1

The initial version of ActiveRecord-Mimer have been released.

The project aims to provide complete ActiveRecord support for the Mimer SQL database engine. This initial release provides the basis for that. Most operations work, including migrations. The only exceptions are rename_column and rename_table which isn't supported by the underlying database engine.

The project resides at RubyForge: http://rubyforge.org/projects/ar-mimer

and can be installed with RubyGems by
gem install activerecord-mimer

The code is released under an MIT license.

Another OpenSSL woe.

My interesting OpenSSL implementation exercise continues. I am now very close. Very, very close. I'm actually so close that SSLSocket and SSLServer actually works, provided that you use Anonymous Diffie-Hellman (which is suicidal, but that's another story). All of this have been submitted to my openssl-branch. What's missing is the X509-store and PKCS#7. And the X509-store doesn't really look good. Not good at all. It's needed for full SSL support. But the bad thing is this: there isn't any Java libraries that duplicate the functionality. Nada. At least not that I can find. The functionality needed is to read and write X509_store-formatted files and directories, to be able to add certificates and CRL's and to verify against these a certificate, based on various interesting OpenSSL rules.

I wouldn't say that I mislike OpenSSL. I wouldn't say that I hate it either. It's very impressive in many ways. But boy. It seems I have to port a substantial part of it to Java, and I'm not looking forward to it. I need to to do both a port, and add support for KeyStore and CertStore so the Java SSLEngine also can use the information. Will this be an interesting exercise? Oh yes.

So, without further ado, this is the plea of this blog post: If you know of any easier way to do this, please tell me. Now! (where "this" is the X509_STORE-family of functions.)

JRuby versus Camping: Round 2

As undoubtedly some of you have discovered while trying, JRuby doesn't run Camping anymore. The culprit is a small feature in Ruby that we don't support yet. For some reason this feature is the preferred idiom for option parsing and Camping 1.5 introduced it, which means that Camping 1.5 will fail miserably in JRuby right now. There are more or less two ways around it, though. The first one is easy; just use a Camping version that is

opts.on("-p", "--port NUM", "Port for web server (defaults to #{conf.port})")
     { |conf.port| }

To fix this particular instance, just change it into this:

opts.on("-p", "--port NUM", "Port for web server (defaults to #{conf.port})")
     { |v| conf.port=v }

Not much of a difference, really. On all places where assignment to a non-obvious node is done in the manner above, just replace it with a regular assignment, and Camping will run.

Now, we are planning on fixing this syntax, but there is much development going down right now, and this change requires some changes in the parser, which is always interesting. But it will be there.

Introducing TIJuAVA - Java with Type Inference

Every time I've written Java code lately, I've been painfully aware of how much unnecessary code I write every time. And most of this is Java's fault. This blog post is a very small thought experiment. TIJuAVA does not exist as software. Yet. If I someday have the time I would love to implement it, but there are more pressing needs right now.

So, what are the rules? Basically, all valid Java programs are valid TIJuAVA programs. Some valid TIJuAVA programs are not valid Java programs. Simply put, the main difference is that you don't need to declare a type for any local variables or member variables. Type declarations are only necessary in method declarations. You can declare local variables and member variables if you want to, and in certain very unlikely circumstances you will need too.

Let's take a very simple example. This code is taken from the JRuby source code, but I have added one or two things to make it easier to showcase:

package org.jruby.util.collections;

import java.util.ArrayList;
import java.util.Collection;
import java.util.Iterator;

public class IdentitySet {
   private items = new ArrayList();

   public void add(Object item) {
       items.add(item);
   }

   public void remove(Object item) {
       iter = items.iterator();
       while (iter.hasNext()) {
           storedItem = iter.next();
           if (item == storedItem) {
               iter.remove();
           }
       }
   }

   public boolean contains(Object item) {
       iter = items.iterator();
       while (iter.hasNext()) {
           storedItem = iter.next();
           if (item == storedItem) {
               return true;
           }
       }
       return false;
   }

   private Collection getItems() {
       return items;
   }

   private void something(java.util.AbstractSet inp) {
       val1 = inp;
       for(iter = val1.iterator();iter.hasNext();) {
           System.err.println(iter.next());
       }
   }
}

This code doesn't really show all that can be done with this approach, and if I were to show a real example, this blog would be unbearably filled with code. So, this is just a tidbit.

The TIJuAVA system would need to be implemented as a Java two-pass compiler. Basically, the first pass finds all variable names that need to have a type inferred, and then walks through the information it's got, basic on method signatures and methods called on the variable. In almost all cases it will be possible to come to one conclusion on which type to use. The compiler would then generate regular Java byte code, basically the same bytecode that would have been generated had you written the types by hand.

Of course, most people use IDE's to write code nowadays. Wizards and code generators and what not. So why something like this? Well, even though your IDE writes your code for you, it is still there, and you still have to understand it at some level. If not when writing, you would still need to read it. And boy does type declarations clutter things. Especially generics. And here is one interesting tidbit. Generic types would also be possible to infer in most cases.

Another thing that could be easily added is some kind of in-place literal syntax for lists and maps. This would be more like a macro feature, but the list syntax would mostly just be a call to Array.asList, which isn't to bad.

An objection that I anticipate is from people who think that the code will be less readable by removing the type pointers. This should be more of a problem when you have large methods, but everyone these days use refactorings so they won't have methods with a LOC over 20. And if that's the case, the local variables should be easily understood by the operations that are used on them.

So. Someday, when I have time, this may be reality. If anyone is interested, that is.