RbYAML in Google Summer of Code

Great news for all Ruby implementations around. A project to bring RbYAML up-to-date and perform better has been accepted for Google Summer of Code. Long Sun is the name of the student, and me and Xue Yong Zhi will jointly mentor this effort.

In fact, I'm very excited about this news. RbYAML was an incredibly important piece of the puzzle to get JRuby to finally work with RubyGems, and that kickstarted our possibilities to start testing numerous other applications. I soon ported RbYAML to Java, and created the JvYAML and JvYAMLb projects, to get better efficiency. Sadly, this left RbYAML without any TLC. That changed a while back when Rubinius picked up the project to get their YAML support going, and now that Long Sun will work on it, hopefully we will finally get an extremely compliant and bug free YAML implementation for Ruby.

This will obviously benefit Rubinius, but it will also be very good for both JRuby and IronRuby. The work will be test-driven which means a more complete test suite will be built around YAML in Ruby.

If you're interested in following the project, it's now hosted at Google Code (due to problems with RubyForge from China) at http://code.google.com/p/rbyaml/. Long Sun will also blog about his progress here: http://rbyaml.blogspot.com/.

Exciting news indeed.

Ruby Design meeting

Yesterday marked the first of the Ruby design meetings, where most of the Ruby implementers got together on IRC and started hashing out the solutions to several current concerns, and worked on getting more cooperation in the design of Ruby features.

This is slated to become a weekly meeting, and it's a huge deal. This will make the lives of all Ruby implementations much easier, and the meeting yesterday actually accomplished some very nice things.

There were representatives from JRuby, Rubinius and macruby present, and of course also Matz, Koichi, Nobu and Tanaka from the Ruby core team.

Some of the highlights was a decision to start working on a common API for MultiVM, initial acceptance to add the RubySpecs (coming from Rubinius originally) into the 1.8 and 1.9 build process, meaning that regression testing will be much better from now on, and also having most implementations using the same specs for compliance testing. In reality, this takes us one step closer to a real executable specification that everyone agrees on and has official blessing from Matz.

In conjunction with this, a decision to set up continuous integration for 1.8 and 1.9 was made. The exact practicalities is still to be decided, but the decision to get it done is also very important.

All in all, these are excellent news, and I'm feeling extremely hopeful about more cooperation between the Ruby implementors.

If you're interested in exactly what happened, you can find the agenda, action items and log here: http://ruby-design.pbwiki.com/Design20080421.

JtestR 0.2 released!

And so, JtestR 0.2 has finally been released. The highlights include support for Expectations and TestNG, RSpec stories and lots of other goodies.

Here is the release announcement:

JtestR allows you to test your Java code with Ruby frameworks.

Homepage: http://jtestr.codehaus.org
Download: http://dist.codehaus.org/jtestr

JtestR 0.2 is the current release of the JtestR testing tool. JtestR integrates JRuby with several Ruby frameworks to allow painless testing of Java code, using RSpec, Test/Unit, Expectations, dust and Mocha.

Features:
- Integrates with Ant and Maven
- Includes JRuby 1.1, Test/Unit, RSpec, Expectations, dust, Mocha and ActiveSupport
- Customizes Mocha so that mocking of any Java class is possible
- Background testing server for quick startup of tests
- Automatically runs your JUnit and TestNG codebase as part of the build

Getting started: http://jtestr.codehaus.org/Getting+Started

New and fixed in this release:
JTESTR-10 It should be possible to run TestNG tests
JTESTR-12 Buildr support
JTESTR-13 CC.rb should be able to run JtestR tests
JTESTR-17 Tests should be groupable and runnable per groups
JTESTR-21 Support RSpec stories
JTESTR-28 JtestR should include expectations
JTESTR-30 code coverage support
JTESTR-31 Autoloading of Java constants
JTESTR-32 Can't load IA 32-bit .so on a IA 32-bit platform
JTESTR-33 JtestR should use latest version of JRuby
JTESTR-34 Errors when project is in path with spaces on Windows XP
JTESTR-37 Can't expect a specific Java exception correctly
JTESTR-38 Problem with mocking Java classes
JTESTR-39 RSpec story runner seems to require rubygems
JTESTR-40 Package missing or.jruby.exceptions.RaiseException
JTESTR-43 It should be possible to get the generated mock class without instantiation
JTESTR-44 New output files start with a whitespace
JTESTR-45 RSpec raise_error and Test/Unit assert_raise and assert_nothing_raised handled JRuby NativeException stuff correctly.

Team:
Ola Bini - ola.bini@gmail.com
Anda Abramovici - anda.abramovici@gmail.com

Connecting languages (or polyglot programming example 1)

Today I spent some time connecting two languages that are finding themselves popular for solving wildly different kinds of problems. I decided I wanted to see how easy it was and if it was a workable solution if you would want to take advantage of the strengths of both languages. The result is really up to you. My 15 minutes experiment is what I'll discuss here.

If you'd like, you can see this as a practical example of the sticky part where two languages meet, in language-oriented programming.

The languages under consideration is Ruby and Erlang. The prerequisite reading is this eminent article by my colleague Dennis Byrne: Integrating Java and Erlang.

The only important part is in fact the mathserver.erl code, which you can see here:

-module(mathserver).
-export([start/0, add/2]).

start() ->
Pid = spawn(fun() -> loop() end),
register(mathserver, Pid).

loop() ->
receive
  {From, {add, First, Second}} ->
      From ! {mathserver, First + Second},
      loop()
end.

add(First, Second) ->
mathserver ! {self(), {add, First, Second}},
receive
  {mathserver, Reply} -> Reply
end.

Follow Dennis' instructions to compile this code and start the server in an Erlang console, and then leave it there.

Now, to use this service is really easy from Erlang. You can really just use the mathserver:add/2 operation directly or remotely. But doing it from another language, in this case Ruby is a little bit more complicated. I will make use of JRuby to solve the problem.

So, the client file for using this code will look like this:

require 'erlang'

Erlang::client("clientnode", "cookie") do |client_node|
server_node = Erlang::OtpPeer.new("servernode@127.0.0.1")
connection = client_node.connect(server_node)

connection.sendRPC("mathserver", "add", Erlang::list(Erlang::num(42), Erlang::num(1)))

sum = connection.receiveRPC

p sum.int_value
end

OK, I confess. There is no erlang.rb yet, so I made one. It includes some very small things that make the interfacing with erlang a bit easier. But it's actually still quite straight forward what's going on. We're creating a named node with a specific cookie, connecting to the server node, and then using sendRPC and receiveRPC to do the actual operation. The missing code for the erlang.rb file should look something like this (I did the minimal amount here):

require 'java'
require '/opt/local/lib/erlang/lib/jinterface/priv/OtpErlang.jar'

module Erlang
import com.ericsson.otp.erlang.OtpSelf
import com.ericsson.otp.erlang.OtpPeer
import com.ericsson.otp.erlang.OtpErlangLong
import com.ericsson.otp.erlang.OtpErlangObject
import com.ericsson.otp.erlang.OtpErlangList
import com.ericsson.otp.erlang.OtpErlangTuple

class << self
def tuple(*args)
  OtpErlangTuple.new(args.to_java(OtpErlangObject))
end

def list(*args)
  OtpErlangList.new(args.to_java(OtpErlangObject))
end

def client(name, cookie)
  yield OtpSelf.new(name, cookie)
end

def num(value)
  OtpErlangLong.new(value)
end

def server(name, cookie)
  server = OtpSelf.new(name, cookie)
  server.publish_port

  while true
    yield server, server.accept
  end
end
end
end

As you can see, this is regular simple code to interface with a Java library. Note that you need to find where JInterface is located in your Erlang installation and point to that (and if you're on MacOS X, the JInterface that comes with ports doesn't work. Download and build a new one instead).

There are many things I could have done to make the api MUCH easier to use. For example, I might add some methods to OtpErlangPid, so you could do something like:

pid << [:call, :mathserver, :add, [1, 2]]

where the left arrows sends a message after transforming the arguments.

In fact, it would be exceedingly simple to make the JInterface API downright nice to use, getting the goodies of Erlang while retaining the Ruby language. And oh yeah, this could work on MRI too. There is an equivalent C library for interacting with Erlang, and there could either be a native extension for doing this, or you could just wire it up with DL.

If you read the erlang.rb code carefully, you might have noticed that there are several methods not in use currently. Say, why are they there?

Well, it just so happens that we don't actually have to use any Erlang code in this example at all. We could just use the Erlang runtime system as a large messaging bus (with fault tolerance and error handling and all that jazz of course). Which means we can create a server too:

require 'erlang'

Erlang::server("servernode", "cookie") do |server, connection|
 terms = connection.receive
 arguments = terms.element_at(1).element_at(3)
 first = arguments.element_at(0)
 second = arguments.element_at(1)

 sum = first.long_value + second.long_value
 connection.send(connection.peer.node, Erlang::tuple(server.pid, Erlang::num(sum)))
end

The way I created the server method, it will accept connections and invoke the block for every time it accepts a connection. This connection is yielded to the block together with the actual node object representing the server. The reason the terms are a little bit convoluted is because the sendRPC call actually adds some things that we can just ignore in this case. But if we wanted, we could check the first atoms and do different operations based on these.

You can run the above code in server, and use the exact same math code if you want. For ease of testing, switch the name to servernode2 in both server and client, and then run them. You have just sent Erlang messages from Ruby to Ruby, passing along Java on the way.

Getting different languages working together doesn't need to be hard at all. In fact, it can be downright easy to switch to another language for a few operations that doesn't suit the current language that well. Try it out. You might be surprised.

Pragmatic Static Typing

I have been involved in several discussions about programming languages lately and people have assumed that since I spend lots of time in the Ruby world and with dynamic languages in general I don't like static typing. Many people in the dynamic language communities definitely expresses opinions that sound like they dislike static typing quite a lot.

I'm trying to not sound defensive here, but I feel the need to clarify my position on the whole discussion. Partly because I think that people are being extremely dogmatic and short-sighted by having an attitude like that.

Most of my time I spend coding in Java and in Ruby. My personal preference are to languages such as Common Lisp and Io, but there is no real chance to use them in my day-to-day work. Ruby neatly fits the purpose of a dynamic language that is close to Lisp for my taste. And I'm involved in JRuby because I believe that there is great worth in the Java platform, but also that many Java programmers would benefit from staying less in the Java language.

I have done my time with Haskell, ML, Scala and several other quite statically typed languages. In general, the fact that I don't speak that much about those languages is that I have less exposure to them in my day-to-day life.

But this is the thing. I don't dislike static typing. Absolutely not. It's extremely useful. In many circumstances it gives me things that really can't have in a dynamic language.

Interesting thought: Smalltalk is generally called a dynamic language with very late binding. There are no static type tags and no type inference happening. The only type checking that happens will happen at runtime. In this regard, Smalltalk is exactly like Ruby. The main difference is that when you're working with Smalltalk, it is _always_ runtime. Because of the image based system, the type checking actually happens when you do the programming. There is no real difference between coding time and runtime. Interestingly, this means that Smalltalk tools and environments have most of the same features as a static programming language, while still being dynamic. So a runtime based image system in a dynamic, late bound programming language will actually give you many of the benefits of static typing at compile time.

So the main take away is that I really believe that static typing is extremely important. It's very, very useful, but not in all circumstances. The fact that we reach for a statically typed programming language by default is something we really need to work with though, because it's not always the right choice. I'm going to say this in even stronger words. In most cases a statically typed language is a premature optimization that gets extremely much in the way of productivity. That doesn't mean you shouldn't choose a statically typed language when it's the best solution for the problem. But this should be a conscious choice and not by fiat, just because Java is one of the dominant languages right now. And if you need a statically typed language, make sure that you choose one that doesn't revel in unnecessary type tags. (Java, C#, C++, I'm looking at you.) My current choice for a static language is Scala - it strikes a good balance in most cases.

A statically typed language with type inference will give you some of the same benefits as a good dynamic language, but definitely not all of them. In particular, you get different benefits and a larger degree of flexibility from a dynamic language that can't be achieved in a static language. Neal Ford and others have been talking about the distinction between dynamic and static typing as being incorrect. The real question is between essence and ceremony. Java is a ceremonious language because it needs you to do several dances to the rain gods to declare even the simplest form of method. In an essential language you will say what you need to say, but nothing else. This is one of the reasons dynamic languages and type inferenced static languages sometimes look quite alike - it's the absence of ceremony that people react to. That doesn't mean any essential language can be replaced by another. And with regads to ceremony - don't use a ceremonious language at all. Please. There is no reason and there are many alternatives that are better.

My three level architecture of layers should probably be updated to say that the stable layer should be an essential, statically typed language. The soft/dynamic layer should almost always be a strongly typed dynamic essential language, and the DSL layers stays the same.

Basically, I believe that you should be extremely pragmatic with regards to both static and dynamic typing. They are tools that solve different problems. But out industry today have a tendency to be very dogmatic about these issues, and that's the real danger I think. I'm happy to see language-oriented programming and polyglot programming get more traction, because they improve a programmers pragmatic sensibilities.

JvYAMLb finally released as separate project

So I've finally made the time to extract JvYAMLb from JRuby. That means that JvYAML is mildly deprecated. Of course, since JvYAMLb only uses ByteLists it might not be the best solution for everyone.

If you're interested in downloading the 0.1 release, you can do it at the Google Code download site http://code.google.com/p/jvyamlb/downloads/list.

Presentations this and next week

Wow. JRuby 1.1 is out! Go get! (And incidentally, wouldn't JRuby be the perfect way for Google to support Ruby in Google Apps Engine?)

I'll be doing a talk tonight at .NETAkademien about Ruby and Rails. It's mostly an introduction.
You can find more info here.

On Thursday I'll speak about JRuby at Developer Summit in Stockholm. I'll be in the Dynamic Languages track and you can read the abstract and more information here.

And finally, next Monday I'll talk about JRuby at the Stockholm Ruby User Group. You can find out more information about that event here.

Overall, it feels like the conference season is starting up again. I'll be presenting at JavaOne in a month too.

I've been extremely quiet here lately. It's been a bit slow in JRuby land, actually, and my focus has been elsewhere. Hopefully I'll be able to start posting some really cool stuff soon - I have something up my sleeve that should be possible to talk about soon.