First two days of TSSJS

It has been two long days; not because I've been going to sessions all day long, but because I've reworked my presentations quite heavily. But now both the BOF and the TS are finished, and I think they went well. I had to keep the level to Ruby, JRuby and Rails introductionary material, though, since most developers here didn't seem to know what is possible with these technologies.

But it's been great; I've gotten good feedback and had some really interesting conversations with lots of people.

We have been doing the town each night, and I've found that I like Barcelona very much. Except for the food: this country doesn't seem to be good for vegetarians at all. Very annoying. I'm going for beer and wine instead of food the rest of the week. =)

One day left, though, and it's bound to be nice. Me and Martin are both on a developers panel about the state of programming languages in 2020; I have no idea what to say, and I'm thinking about just ad-libbing it. I know my own position in these questions fairly well, and the current Yegge-debate have made my opinions even more explicit.

But now it's time to see the town again.

Interviewed by AkitaOnRails

Yesterday I spent 2 hours chatting with Fabio Akita, of AkitaOnRails (the largest Rails blog in Brazil); the result is a long interview that was published today. It's got some good stuff, and some Ola-stuff, which you should recognize by now.

And I note that he calls me a workaholic; but he got this interview prepared in less than a day too, and also translated it into Portuguese.

You can find it at http://www.akitaonrails.com/pages/olabini.

First weeks at ThoughtWorks

I've finally started. I've finally moved to London. I've been working for two weeks at ThoughtWorks now, and it's been quite crazy. Everything is very nice and I'm having loads of fun. Of course, it's also lots of hard work, and I feel that I'm stretching my capacity considerably more than I ever did at Karolinska Institutet. That's great, and I feel that I'm really doing something real now. We have so many interesting things going on, and I wish I could tell you all about it.

What I can tell you is that I'm working quite much on Mingle, and I'm also spending time on other JRuby related issues. I've been planning on getting SQL Server and Oracle working as good as possible with AR-JDBC, and I've spent time on Derby performance. Hopefully I'll continue the database work this week, since especially SQL Server and Oracle is very important.

The most important work for this week is probably to prepare for TheServerSide in Barcelona. I still haven't had time to prepare my demos, so it's about time now. I hope to see many of you there.

In conclusion, my first weeks at ThoughtWorks have been awesome. I really like the pople, and everything is just neat. I like being able to walk to work and working in the very nice TW office on High Holborn. I'm very happy about it all.

Book update

As you know, I am writing on a book about JRuby on Rails. A few minutes ago I finished the first draft of chapter 14. That means that there are just 3 chapters and 3 appendixes left to write (chapter 1, 2 and 15). So the writing is going very well, but it's taking a heavy toll on me personally. I seriously don't recommend writing a book like this in your spare time, while at the same time switching employer, moving abroad and try to be a core developer in an open source project which is getting lots of attention.

So, in summary: it's going well, it still looks like it will be out in October, and I'm deadly tired.

JRuby 1.0

The JRuby community is pleased to announce the release of JRuby 1.0!

Homepage: http://www.jruby.org/
Download: http://dist.codehaus.org/jruby/

JRuby 1.0 is a major milestone for our project. Our main goal for 1.0 has been
Ruby compatibility. We feel this goal has been reached. When we see
companies like ThoughtWorks offering commercial support; we know this goal
has been reached. Please download JRuby and take it for a test drive. Try
running your Ruby and Ruby on Rails applications with it.

Give us feedback. Join our community. Help us continue to improve JRuby.

It is important to notice that JRuby 1.0 is not the end all of Ruby interpreters. It's not perfect. This is just the beginning. We now have a very good base to work from. This is were the real work begins. Join us. It will be a fun ride, and JRuby will just get better!

What's wrong with this code?

Today I will introduce to you a method from ActiveRecord. The method takes a parameter called type and that value can bu for example :primary_key, :string or :integer. Now, in the first line there is a call to native_database_types. Generally, that call returns a structure that looks somewhat like this:

def native_database_types #:nodoc:
  {
   :primary_key => "int(11) DEFAULT NULL auto_increment PRIMARY KEY",
   :string      => { :name => "varchar", :limit => 255 },
   :text        => { :name => "text" },
   :integer     => { :name => "int", :limit => 11 },
   :float       => { :name => "float" },
   :decimal     => { :name => "decimal" },
   :datetime    => { :name => "datetime" },
   :timestamp   => { :name => "datetime" },
   :time        => { :name => "time" },
   :date        => { :name => "date" },
   :binary      => { :name => "blob" },
   :boolean     => { :name => "tinyint", :limit => 1 }
 }
end

The method itself looks like this.

def type_to_sql(type, limit = nil, precision = nil, scale = nil) #:nodoc:
  native = native_database_types[type]
 column_type_sql = native.is_a?(Hash) ? native[:name] : native
 if type == :decimal # ignore limit, use precison and scale
    precision ||= native[:precision]
   scale ||= native[:scale]
   if precision
     if scale
       column_type_sql << "(#{precision},#{scale})"
     else
       column_type_sql << "(#{precision})"
     end
   else
     raise ArgumentError, "Error adding decimal column: precision cannot be empty if scale if specified" if scale
   end
   column_type_sql
 else
   limit ||= native[:limit]
   column_type_sql << "(#{limit})" if limit
   column_type_sql
 end
end

There is something very wrong with this implementation. Of course, there could exist many errors here, but what I'm thinking about right now is a violation of the usual way methods should work. And in effect, that problem with this method have caused ActiveRecord-JDBC to implement some very inefficient code to handle this method. And it gets called a lot in ActiveRecord. I'll get back later today with a pointer to what's wrong here, and I will also discuss some of what I've done in AR-JDBC to handle this situation. I hope for many suggestions here! =)

This is what's wrong

I must say, I got some really good responses to my post about what was wrong with the code I posted. Most of those responses concerned the design of the code, and I agree, this part of Rails could have been done much better. But what I was thinking about was actually a bug. And Lars Westegren (my former colleague) nailed it at the first try. Let me show two important excerpts from this code:

column_type_sql = native.is_a?(Hash) ? native[:name] : native

and here:

column_type_sql << "(#{limit})" if limit

Obviously, double left arrow is append, and for all cases where there is a limit, this append will change the String. This is one of the cases where it's kind of annoying that strings are mutable. If I cache away the values that native_database_types should return, then the next time anyone wants a string SQL type, that will generate VARCHAR(255)(255). The next time again, VARCHAR(255)(255)(255). And so one. So either I need to recreate the hash every time, or I need to do a deep clone of it every time. Neither of these options are very good, and it seems the deep clone option isn't fast enough, even when done in Java, so I decided to go with a hash literal instead. Was that the right choice? I don't know. It improves performance, but on the other hand it churns objects and creates new objects all the time. All of this because of some sloppy coding in Rails.

What's the lesson learned? Never modify your arguments, unless that is an explicit part of the contract for that method and part of the documentation.

Testing with JRuby on Rails and ActiveRecord-JDBC

This will be a highly uninflammatory blog post, as contrast to the last one. Specifically, there is a slight problem when running the command

jruby -S rake

in a a JRuby on Rails-application. This problem is caused by some hard coded values in the database Rake definitions for Rails. But don't despair, there is a simple solution to this. It's not as simple as it should be (invisible) but it's easy enough. Provided you have JRUBY_HOME set and your version of AR-JDBC is 0.3.1, execute this command from your Rails application root

cp $JRUBY_HOME/lib/ruby/gems/1.8/gems/ActiveRecord-JDBC-0.3.1/lib/tasks/jdbc_databases.rake lib/tasks

Since the hard coded values are hard to override, the jdbc_databases.rake file just hacks Rake to be able to redefine tasks and then redefines the core tasks. This shouldn't affect a bi-Ruby installation, since the overriding only happens on JRuby, not on MRI. If someone has a better way to do this, please tell me. =)

There can be only one, a tale about Ruby, IronRuby, MS and whatnot

(Updated: added a quote from John Lam about not being able to look at the MRI source code)

After RailsConf in Portland, there has flared up a discussion centered around IronRuby and Microsoft. We discussed many of these points in depth at the conference, and I'll elaborate some on my views on the issues in a bit.

But first I would like to talk some about the multitude of Ruby implementations springing up. I firmly believe that a language evolves in phases. The first phase, germination, is the period where a language needs one consistent implementation (or a spec). It's during this phase when most "alpha geek adoption" happens. Many important libraries are written, but most applications are not in the main economic center. Ruby have been in this phase for a long time, but the fact that new implementations are springing up left and right is a sure sign that Ruby is entering phase 2: implementation. For adoption to happen, there need to exist several competing implementations, all of them good. This is the evolutionary stage, where it's decided what kind of features an implementation should provide. Should we have green or native threads? Are all the features of the original implementation really that necessary? (Continuations, ObjectSpace). Is there cruft in the standard library that needs to be weeded out? (timeout.rb). All of these questions get answered when other people implement the language. The last phase, which I guess could be called adoption, is when the language have several working implementations, all good enough to deliver high end applications on, when many applications are written in the language, and there exists a plethora of libraries, systems and support for the language.

What this means is that for a language to be successful, there needs to exist competing implementations. They need to implement their features in different ways and make different choices during development. Otherwise, the language will die. (This is obviously not enough, since Smalltalk fulfilled this admirably and still never got widespread adoption.). But I still believe it's incredibly important for a language to evolve with many implementations, which is why I find Rubinius, JRuby, YARV and IronRuby to be extremely important projects for the welfare of Ruby. I want Ruby to be successful. I want Ruby to be the next major language for several reasons. But most importantly: I want Ruby to be a better language tomorrow, than it is today. The only way that's going to happen is by having lots of people implement the language.

So, that's enough of the introductory flame bait. This describes one half of why IronRuby is an important project, and why we can't let it fail. The other side of the coin is the same reason JRuby is important. .NET as a platform have some wildly useful features. There are many developers who swear by .NET for good reason. And what's more important, there are lots of large enterprises with such a vested interest in .NET, that they will never choose anything else. Now, for the welfare of all programmers in the world, I personally believe the world would be a better place if those .NET-environments also used Ruby. So that's the other coin of why IronRuby is important.

The most well read blog about the current Microsoft/Ruby controversy is Martin Fowlers article RubyMicrosoft. Go read it now, and then I'll just highlight the points I find most important.

First: John Lam is committed to creating a "compliant" Ruby implementation. I have no doubts that he can do it. But there are a few problems lurking.

For example, what is a compliant Ruby implementation? Since there exists no spec, and no comprehensive test suite, the only way to measure compliance is to check how close the behavior matches MRI. But this have some problems too. How do you check behavior? Well, you need to run applications. But how do you get so far as you can run applications?
What JRuby did was that we looked at MRI source code.

John Lam can not look at MRI source code. He cannot look at JRuby source code. He cannot look at Rubinius source code. If he does, he will be terminated.

So, the next best alternative: accepting patches from the community, which can look at Ruby source? Nope, no cigar. Microsoft is not really about Open Source yet. Their license allows us to look at their source code, and also to fork it and do what we want with it. But that's only half of what open source is about. The more important part is that you should be able to contribute back code without having to fork. You can't do that with IronRuby, since Microsoft is too scared about being sued for copyright infringement.

There was some doubt about Lam actually being banned from looking at MRI source code. This is the first quote that said it is so. It's from the discussion "Virtual classes and 'real' classes -- why?" on Ruby-core, this quote posted at 29/03/07:

Is this how things are actually implemented? (BTW I'm not lazy here - we cannot look for ourselves).

I am going to make a bold statement here. Under the current circumstances, I don't believe it's possible for John Lam and his team to create a Ruby implementation that runs Rails within at least 18 months. And frankly, that's not soon enough.

As I said above, I have all confidence that John can do great stuff if he has the right resources. But creating a Ruby implementation is hard enough while having all the benefits of the open source community.

The two points I want to make with this point is this: The Ruby community must damned well get serious about creating a good, complete specification and test suite. It's time to do it right now, and we need it. It's not a one-man job. The community needs to do it. (And yes, the two SoC projects are a very good start. But you still need to be able to run RSpec to take full advantage of them; and let's face it, the RSpec implementation uses many nice Ruby tricks.)

The second point is simpler: Microsoft needs to completely change how they handle Open Source. Their current strategy of trying to grow it into the organization will not work (at least not good enough). They need to turn around completely, reinvent themselves and make some really bold moves to be able to meet the new world. If they can't do this, they are as dead as Paul Graham claims.

Solving mounting problem on MacOS X

This is a highly specific post, but I thought I'd write about this so that if someone else have the same problem, they can try to solve it my way.

I got a new computer, Intel-based MBP last week. I upgraded it to latest OSX version (10.4.9 I believe), and immediately installed everything I needed. While ending my last job and returning the laptop associated with that post, I made backups to a 500GB USB LACIE hard drive. Very nice indeed, and half my life is now on that hard drive. (I have most of it on other places too, but not so easily accessible).

The first thing I managed to do was to shut off the hard drive without unmounting it correctly from OSX. I got the warning et all, but well, done is done. It was stupid, I know. Starting out with messing up everything. So what happened? Well, the next time I tried to get OSX to find my hard drive, by inputing the USB connection and switching on the power. Nothing happened. The drive would spin up, but no mount points or nice icons on the desktop. After some investigation I found that the Disk Manager HANGS when the LACIE is turned on and connected. I also found that when I switched the power off, my console says something about not being able to repair disk /dev/disk2s1. Interesting. After a few hours investigation on the Internet I despaired, and decided to try my own ingenuity instead.

I won't tell you about everything I did to find this solution. It would get severely boring very fast. So, here is the solution:

1. Attach the device and turn on the power.
2. Open up a terminal and create a new directory in /Volumes, (eg sudo mkdir /Volumes/LACIE2).
3. Mount the drive explicitly on this mount point (sudo mount -t msdos /dev/disk2s1 /Volumes/LACIE2).
4. Delete ALL .DS_Store files on the disk (cd /Volumes/LACIE2; find . -name 'DS_Store' -exec rm -rf \{\} \;).
5. Wait for a while. At this point you should have two LACIE drives on your Desktop, one fake and one real. Unmount the real one by dragging it to the trash can.
6. Turn of the external hard drive, or detach it.
7. Reboot.
8. Attach/Turn on the external hard drive.

This is a process that works for me, and my drive is now back to working mode. It seems that OSX stores some information in the DS_Store files somewhere on the disk that got corrupted for me. Hopefully this information can help someone else with the same problem.

Finally in London

Yesterday I landed in London, and let me tell you: it's been tough getting here. Packing and moving and arranging all takes lots of time and energy. And it's not helping being sick while pulling it off, either. But I'm finally here, and right now sitting at a hotel room in Holborn, close to Russel Square. Hopefully I'll be able to land an apartment soon too, and then I'll get back to my former speed, hopefully.

In the meantime, tomorrow is my first day at ThoughtWorks. It's bound to be interesting, but I don't expect any downtime. We have so many interesting things going, that it will be full speed ahead from day one.

Although I'm more tired than I should be, if someone feels like meeting up this or next week for a beer and talk JRuby, it would be fun.