onsdag, augusti 23, 2006

RailsConf Europe

So, it has finally been decided. Me and my four colleagues will go to London for RailsConf. It feels really good, especially since the lineup with speakers is great.

Hopefully I'll get the chance to meet up with some people too. I know at least one JRuby'ist who'll be there.

fredag, augusti 18, 2006

Timeline for the next few weeks.

  • JvYAML:
    I have finally been able to actually get some work done on JvYAML again. The plan is to have the 0.2 release out in two weeks time. There are three components left to finish. I have already completed the Emitter and the Serializer. What's missing is the Representer, JavaBean support and JRuby integration. But when this is finished, we will have a complete pure Java YAML library for JRuby
  • Mongrel:
    I really want Mongrel to work in JRuby. We have been spending some time getting the HTTP-parser ported to Java, and this is more or less finished now. Earlier today I managed to run all the Mongrel test cases for the parser successfully in JRuby, so we're clearly on the way. What we need right now is to get the first parts of the Extension API into JRuby proper. When that's done we can start testing Mongrel for real.
  • ActiveRecord-JDBC:
    I have a goal to rewrite parts of this to abstract away differences between different drivers (we have some specialized code for Oracle, some for MySQL, and that's no good in the long run.) I'm also planning on writing some test cases using Migrations, so we can test the driver against many different databases. What I really want to do is to test MySQL, Oracle, MS SQL Server and PostgreSQL enough to say that JRuby can use them. Especially MS SQL would be good, since MRI only can use ODBC if not running production on a Win32 machine.
  • JRuby performance and bug correction:
    Right now these are the big points. We need to correct as many bugs as possible, nibble away at performance, and also improve documentation. I will try to take a look at these issues if time allows.
  • CLaJ:
    This is a new project that I have been thinking about and planning for the last few months. I won't say much more about it right now, but that it is something I'm really passionate about (even more than JRuby). Expect more information in a blog post soon.

torsdag, augusti 17, 2006

The JRuby Tutorial #2: Going Camping

So, for the 2nd, (actually third installment, but the JIRB one doesn't really count), I was thinking that we should take a look at a few different subjects. The red thread will be a simple web based Blog application built with Camping. Camping is a microframework for web development, by whytheluckystuff, is insanely small and incredibly powerful. It uses another library called Markaby, which generates markup based on pure Ruby code. I will show the application in smaller parts, with explanations, and at the end include a link to the complete source file.

First of all we have to have a working JRuby built from the latest version of trunk. After JRuby is working we need to install a few gems. Follow these commands and you'll be good:
jruby %JRUBY_HOME%\bin\gem install camping --no-ri --no-rdoc --include-dependencies
jruby %JRUBY_HOME%\bin\gem install ActiveRecord-JDBC --no-ri --no-rdoc
This installs Camping, ActiveRecord, ActiveSupport, Markaby, Builder, Metaid and ActiveRecord-JDBC. We don't generate RDoc or RI for these gems, since that's one part of JRuby that still is pretty slow.

The Blog application
The first thing the blog application needs is a database. I will use MySQL, but other databases may work through the JDBC-adapter, but there is still some work to be done in this area. I will have my MySQL server on localhost, so change the configuration if you do not have this setup. You'll need a database for the blog application. I've been conservative and named the database "blog" and the user "blog" with the password "blog". Easy to remember, but not so good for production.

Update: As azzoti mentioned, you have to set your classpath to include the MySQL JDBC-driver, which can be downloaded from http://www.mysql.org/.

Now, open up a new Ruby file and call it blog.rb. The name of the file is important; it has to have the same name as the Camping application. Now, first of all we include the dependencies:
require 'rubygems'
require 'camping'
require 'camping/session'
require_gem 'ActiveRecord-JDBC'
require 'active_record/connection_adapters/jdbc_adapter'
require 'java'

include_class 'java.lang.System'
These statements include Rubygems, Camping, Camping session support, ActiveRecord-JDBC and Java support. It also includes the Java class named java.lang.System for later use. The next step is to include some configuration for our application.
Camping.goes :Blog
Blog::Models::Base.establish_connection :adapter => 'jdbc', :driver => 'com.mysql.jdbc.Driver',
:url => 'jdbc:mysql://localhost:3306/blog', :username => 'blog', :password => 'blog'

module Blog
include Camping::Session
end
These statements first names our application with the Camping.goes-statement. This includes some fairly heavy magic, including reopening the file and rewriting the source to include more references to the Camping framework. But this line is all that is needed. The next line establishes our connection to the database, and it follows standard JDBC naming of the parameters. Of course, these should be in a YAML file somewhere, but for now we make it easy. The last part makes sure we have Session support in our Blog application.

Now we need to define our model, and this is easily done since Camping uses ActiveRecord:
module Blog::Models
def self.schema(&block)
@@schema = block if block_given?
@@schema
end

class Post < Base; belongs_to :user; end
class Comment < Base; belongs_to :user; end
class User < Base; end
end
The first part of this code defines a helper method that either sets the schema to the block given, or returns an earlier defined schema. The second part defines our model, which includes Post, Comment and User, and their relationships.

The schema is also part of the application, and we'll later see that Camping can automatically create it if it doesn't exist (that's why we didn't need to create any tables ourselves, just the database).
 Blog::Models.schema do
create_table :blog_posts, :force => true do |t|
t.column :id, :integer, :null => false
t.column :user_id, :integer, :null => false
t.column :title, :string, :limit => 255
t.column :body, :text
end
create_table :blog_users, :force => true do |t|
t.column :id, :integer, :null => false
t.column :username, :string
t.column :password, :string
end
create_table :blog_comments, :force => true do |t|
t.column :id, :integer, :null => false
t.column :post_id, :integer, :null => false
t.column :username, :string
t.column :body, :text
end
end
This defines the three tables needed by our blog system. Note that the names of the tables include the name of the application as a prefix. This is because Camping expects more than one application to be deployed in the same container, using the same database.

When we have defined the schema, it's time to define our controller actions. In Camping, each action is a class, and each action class define a method for get, one for post, etc. These classes will be defined inside the module Blog::Controllers. The first action we create will be the Index action. It looks like this:
     class Index < R '/'
def get
@posts = Post.find :all
render :index
end
end
This defines a class that inherits from an anonymous class defined by the R method. What it really does, is bind the Index action to the /-path. It uses ActiveRecord to get all posts and then renders the view with the name index.

The Add-action adds a new Post, but only if there is a user in the @state-variable, which acts as a session. If something is posted to it, it creates a new Post from the information and redirects to the View-action:
     class Add
def get
unless @state.user_id.blank?
@user = User.find @state.user_id
@post = Post.new
end
render :add
end
def post
post = Post.create :title => input.post_title, :body => input.post_body,
:user_id => @state.user_id
redirect View, post
end
end
As you can see, there's not much to it. Instance variables in the controller will be available to the view later on. Note that this action doesn't inherit from any class at all. This means it will only be available by an URL with it's name in it.

We need a few more controllers. View and Edit are for handling Posts. Comment adds new comments to an existing Post. Login and Logout are pretty self explanatory. And Style returns a stylesheet for all pages. Note that Style doesn't render anything, it just sets @body to a string with the contents to return.
     class View < R '/view/(\d+)'
def get post_id
@post = Post.find post_id
@comments = Models::Comment.find :all, :conditions => ['post_id = ?', post_id]
render :view
end
end

class Edit < R '/edit/(\d+)', '/edit'
def get post_id
unless @state.user_id.blank?
@user = User.find @state.user_id
end
@post = Post.find post_id
render :edit
end

def post
@post = Post.find input.post_id
@post.update_attributes :title => input.post_title, :body => input.post_body
redirect View, @post
end
end

class Comment
def post
Models::Comment.create(:username => input.post_username,
:body => input.post_body, :post_id => input.post_id)
redirect View, input.post_id
end
end

class Login
def post
@user = User.find :first, :conditions => ['username = ? AND password = ?', input.username, input.password]
if @user
@login = 'login success !'
@state.user_id = @user.id
else
@login = 'wrong user name or password'
end
render :login
end
end

class Logout
def get
@state.user_id = nil
render :logout
end
end

class Style < R '/styles.css'
def get
@headers["Content-Type"] = "text/css; charset=utf-8"
@body = %{
a {
text-decoration: none;
}
a:hover {
text-decoration: underline;
}
body {
font-family: Utopia, Georga, serif;
}
h1.header {
background-color: #fef;
margin: 0; padding: 10px;
}
div.content {
padding: 10px;
}
div.post {
background-color: #ffa;
border: 1px solid black;
padding: 20px;
margin: 10px;
}
}
end
end
Also note how easy it is to define routing rules with the help of regular expressions to the R method.

Next up, we have to create our views. Since Camping uses Markaby, we do it in Ruby, in the same file. Views are methods in the module Blog::Views with the same name as referenced inside the controllers call to render. There is a special view called layout which get called for all views, if you don't specify otherwise in the call to render. It looks like this:
     def layout
html do
head do
title 'Blog'
link :rel => 'stylesheet', :type => 'text/css',
:href => '/styles.css', :media => 'screen'
end
body do
h1.header { a 'Blog', :href => R(Index) }
div.content do
self << yield
end
end
end
end
As you can see, standard HTML tags are defined by calling a method by that name. The contents of the tag is created inside the block sent to that method, and if it makes sense to give it content as an argument, this works too. Title, for example. If you give a block to it, it will evaluate this and add the result as the title, but in this case it's easier to just provide a string. Note how a link is created, by the method R (another method R this time, since this is in the Views module). Finally, the contents of the layout gets added by appending the result of a yield to self.

The index view is the first we'll see when visiting the application, and it looks like this:
     def index
if @posts.empty?
p 'No posts found.'
else
for post in @posts
_post(post)
end
end
p { a 'Add', :href => R(Add) }
p "Current time in millis is #{System.currentTimeMillis}."
end
I've also added a call that writes out the current time in milliseconds, from Java's System class, to show that we're actually in Java land now, and potentially could base much of our application on data from Java. We check if there are any posts, and if so iterate over them and write them out with a partial called _post. We also create a link to add more posts. The rest of the views look like this:
     def login
p { b @login }
p { a 'Continue', :href => R(Add) }
end

def logout
p "You have been logged out."
p { a 'Continue', :href => R(Index) }
end

def add
if @user
_form(post, :action => R(Add))
else
_login
end
end

def edit
if @user
_form(post, :action => R(Edit))
else
_login
end
end

def view
_post(post)

p "Comment for this post:"
for c in @comments
h1 c.username
p c.body
end

form :action => R(Comment), :method => 'post' do
label 'Name', :for => 'post_username'; br
input :name => 'post_username', :type => 'text'; br
label 'Comment', :for => 'post_body'; br
textarea :name => 'post_body' do; end; br
input :type => 'hidden', :name => 'post_id', :value => post.id
input :type => 'submit'
end
end
And the three partials:
     def _login
form :action => R(Login), :method => 'post' do
label 'Username', :for => 'username'; br
input :name => 'username', :type => 'text'; br

label 'Password', :for => 'password'; br
input :name => 'password', :type => 'text'; br

input :type => 'submit', :name => 'login', :value => 'Login'
end
end

def _post(post)
div.post do
h1 post.title
p post.body
p do
a "Edit", :href => R(Edit, post)
a "View", :href => R(View, post)
end
end
end

def _form(post, opts)
p do
text "You are logged in as #{@user.username} | "
a 'Logout', :href => R(Logout)
end
form({:method => 'post'}.merge(opts)) do
label 'Title', :for => 'post_title'; br
input :name => 'post_title', :type => 'text',
:value => post.title; br

label 'Body', :for => 'post_body'; br
textarea post.body, :name => 'post_body'; br

input :type => 'hidden', :name => 'post_id', :value => post.id
input :type => 'submit'
end
end
In my opinion, this code is actually much easier to read than HTML and most of it is fairly straight forward. One interesting part is the add and edit methods, which checks if a user is logged in, otherwise uses the _login-partial instead of showing the real content.

Finally, we will define a create-method for Camping, which is responsible for creating the tables for our model:
 def Blog.create
Camping::Models::Session.create_schema
unless Blog::Models::Post.table_exists?
ActiveRecord::Schema.define(&Blog::Models.schema)
Blog::Models::User.create(:username => 'admin', :password => 'camping')
end
end

This first creates a table for the session information, and then checks if the Post-table exists; if not all the tables in the schema defined before is created.

Now you have seen the complete application. If you have no interest in writing this by hand, the complete code can be found here.

Running the application
To run a Camping application, you need to run the camping executable that has been installed into your %JRUBY_HOME%\bin on your application file. In my case I run it like this:
jruby %JRUBY_HOME%\bin\camping blog.rb
in the directory where my blog.rb exists and I very soon have a nice application at http://localhost:3301/blog which works wonderfully. Startup is a little bit slow, but as soon as WEBrick has started listening the application is very snappy. You can try changing your blog.rb-file too; Camping will automatically update your application without having to restart the server. As I said above, I included a call to System.currentTimeMillis, to show that we are actually using Java in this blog-application. If that isn't apparent from the call to System, remember that we are actually using JDBC to talk to our database, and very soon you will be able to use the ActiveRecord-JDBC adapter to connect to any databases Java can talk too. That's a brigth future.

The JRuby Tutorial #1.5: Using JIRB to check Java behaviour

I was planning on this issue to be on Camping, but I realized that it would be good to show how to use IRB with JRuby to make life as a Java developer easier. Since Java is a compiled language, it's exceedingly hard to just test things out. My hard drive is completely littered with 5-line programs testing one aspect of the Java libraries or whatnot. If I could have an easy way to interface with the libraries without having to go through a compile cycle, this would ease my life very much. And actually, the last months I have been using JIRB in this way. Of course there is BeanShell and Groovy, but I feel they get in the way. And I'm very fond of IRB, of course.

IRB stands for Interactive Ruby, and JIRB is the name of the program for starting IRB with JRuby. Starting it should be as easy as:
%JRUBY_HOME%\bin\jirb
and after two or three seconds you will get the Ruby prompt:
irb(main):001:0>
At this point, the first thing to do is to require the java libraries, so we can have something to work with:
irb(main):001:0> require 'java'
=> true
I would first like to show how I would go about finding the behaviour of the caret in a Java regexp. (I know I could find it in the API, but this is just to show how you could do it.)

First of all, I will include the relevant class and see what I can do:
irb(main):002:0> include_class 'java.util.regex.Pattern'
=> ["java.util.regex.Pattern"]
irb(main):003:0> Pattern.new
NameError: wrong # of arguments for constructor
from D:\Project\jruby-trunk\src\builtin\javasupport.rb:231:in `initialize'
from (irb):1:in `new'
from (irb):1:in `binding'
from D:/project/jruby-trunk/lib/ruby/1.8/irb.rb:150:in `eval_input'
from D:/project/jruby-trunk/lib/ruby/1.8/irb.rb:70:in `signal_status'
from D:/project/jruby-trunk/lib/ruby/1.8/irb.rb:189:in `eval_input'
from D:/project/jruby-trunk/lib/ruby/1.8/irb.rb:70:in `each_top_level_statement'
from D:/project/jruby-trunk/lib/ruby/1.8/irb.rb:190:in `loop'
from D:/project/jruby-trunk/lib/ruby/1.8/irb.rb:190:in `catch'
from D:/project/jruby-trunk/lib/ruby/1.8/irb.rb:190:in `eval_input'
from D:/project/jruby-trunk/lib/ruby/1.8/irb.rb:70:in `start'
from D:\project\jruby-trunk\bin\jirb:13:in `catch'
from D:/project/jruby-trunk/lib/ruby/1.8/irb.rb:71:in `start'
from D:\project\jruby-trunk\bin\jirb:13
irb(main):004:0> Pattern.methods
=> ["matches", "quote", "compile", "setup_constants", "setup", "singleton_class", "setup_attributes", "new_instance_for", "[]", "const_missing" ...]
irb(main):005:0> Pattern.methods.grep /compile/
=> ["compile"]
Ok, first I include the class, then I try it out. Whoopsie, there was no empty constructor for Pattern. I'm don't really remember what methods are in there, though, so I check out which methods Pattern have. There is a good many of them (and I haven't written them all out either), so I try again by grepping for a method name I believe is in there. Once I've found the method I'm looking for, I call it with the pattern I want to compile, and inspect the result in various ways. I vaguely remember a matcher method, and it seems to work as I remember:
irb(main):008:0> pt = Pattern.compile("^foo")
=> ^foo
irb(main):009:0> pt.inspect
=> "^foo"
irb(main):010:0> pt.java_class
=> java.util.regex.Pattern
irb(main):011:0> pt.class
=> Pattern
irb(main):012:0> m = pt.matcher("foo")
=> java.util.regex.Matcher[pattern=^foo region=0,3 lastmatch=]
Ok, now I have something to work with. Good, and the standard output from JRuby gives me some insight into the internals of the matcher too; interesting stuff. There is a find method, and a matches method on the Matcher:
irb(main):013:0> m.find
=> true
irb(main):014:0> m.find
=> false
irb(main):015:0> m.matches
=> true
irb(main):016:0> m.search
NoMethodError: undefined method `search' for java.util.regex.Matcher[pattern=^foo region=0,3 lastmatch=foo]:#
from (irb):1:in `method_missing'
from (irb):1:in `binding'
from D:/project/jruby-trunk/lib/ruby/1.8/irb.rb:150:in `eval_input'
from D:/project/jruby-trunk/lib/ruby/1.8/irb.rb:70:in `signal_status'
from D:/project/jruby-trunk/lib/ruby/1.8/irb.rb:189:in `eval_input'
from D:/project/jruby-trunk/lib/ruby/1.8/irb.rb:70:in `each_top_level_statement'
from D:/project/jruby-trunk/lib/ruby/1.8/irb.rb:190:in `loop'
from D:/project/jruby-trunk/lib/ruby/1.8/irb.rb:190:in `catch'
from D:/project/jruby-trunk/lib/ruby/1.8/irb.rb:190:in `eval_input'
from D:/project/jruby-trunk/lib/ruby/1.8/irb.rb:70:in `start'
from D:\project\jruby-trunk\bin\jirb:13:in `catch'
from D:/project/jruby-trunk/lib/ruby/1.8/irb.rb:71:in `start'
from D:\project\jruby-trunk\bin\jirb:13
I also belived there was a search method, and also a way to get back to the initial state of the Matcher. Hmm. Time to check out the methods of this object too:
irb(main):017:0> m.methods
=> ["hashCode", "start", "matches", "useAnchoringBounds", "region", "reset"...] irb(main):019:0> m.reset
=> java.util.regex.Matcher[pattern=^foo region=0,3 lastmatch=]
irb(main):020:0> m.find
=> true
irb(main):021:0> m.group 0
=> "foo"
irb(main):022:0>
I found the method I was looking for; reset, calls it and tries find again, and then gets the group with index 0, which is the complete match.

Some Swinging
JIRB works great for trying out Swing interactively. I'm just going to show a tidbit of this, and since this is one of the more common examples of JRuby, you will be able to find many different demonstrations of this on the net.

We first require java-support, and including the required classes for this example:
irb(main):001:0> require 'java'
=> true
irb(main):002:0> include_class(['javax.swing.JFrame','javax.swing.JLabel','javax.swing.JButton','javax.swing.JPanel'])
=> ["javax.swing.JFrame", "javax.swing.JLabel", "javax.swing.JButton", "javax.swing.JPanel"]

Then we create a frame and some panels:
irb(main):003:0> frame = JFrame.new("JRuby panel")
=> javax.swing.JFrame[frame0,0,0,0x0,invalid,hidden,layout=java.awt.BorderLayout,title=JRuby panel,resizable,normal,defaultCloseOperation=HI
DE_ON_CLOSE,rootPane=javax.swing.JRootPane[,0,0,0x0,invalid,layout=javax.swing.JRootPane$RootLayout,alignmentX=0.0,alignmentY=0.0,border=,fl
ags=16777673,maximumSize=,minimumSize=,preferredSize=],rootPaneCheckingEnabled=true]
irb(main):004:0> basePanel = JPanel.new
=> javax.swing.JPanel[,0,0,0x0,invalid,layout=java.awt.FlowLayout,alignmentX=0.0,alignmentY=0.0,border=,flags=9,maximumSize=,minimumSize=,pr
eferredSize=]
irb(main):005:0> labelPanel = JPanel.new
=> javax.swing.JPanel[,0,0,0x0,invalid,layout=java.awt.FlowLayout,alignmentX=0.0,alignmentY=0.0,border=,flags=9,maximumSize=,minimumSize=,pr
eferredSize=]
irb(main):006:0> buttonPanel = JPanel.new
=> javax.swing.JPanel[,0,0,0x0,invalid,layout=java.awt.FlowLayout,alignmentX=0.0,alignmentY=0.0,border=,flags=9,maximumSize=,minimumSize=,pr
eferredSize=]
After this, we add some labels and some buttons to the panes, and then add the panes to the base pane, and the base pane to the frame.
irb(main):007:0> labelPanel.add JLabel.new("Isn't JRuby cool?")
=> javax.swing.JLabel[,0,0,0x0,invalid,alignmentX=0.0,alignmentY=0.0,border=,flags=8388608,maximumSize=,minimumSize=,preferredSize=,defaultI
con=,disabledIcon=,horizontalAlignment=LEADING,horizontalTextPosition=TRAILING,iconTextGap=4,labelFor=,text=Isn't JRuby cool?,verticalAlignm
ent=CENTER,verticalTextPosition=CENTER]
irb(main):008:0> labelPanel.add JLabel.new("JIrb is very useful")
=> javax.swing.JLabel[,0,0,0x0,invalid,alignmentX=0.0,alignmentY=0.0,border=,flags=8388608,maximumSize=,minimumSize=,preferredSize=,defaultI
con=,disabledIcon=,horizontalAlignment=LEADING,horizontalTextPosition=TRAILING,iconTextGap=4,labelFor=,text=JIrb is very useful,verticalAlig
nment=CENTER,verticalTextPosition=CENTER]
irb(main):009:0> buttonPanel.add JButton.new("Pushbutton without action")
=> javax.swing.JButton[,0,0,0x0,invalid,alignmentX=0.0,alignmentY=0.5,border=javax.swing.plaf.BorderUIResource$CompoundBorderUIResource@1354
355,flags=296,maximumSize=,minimumSize=,preferredSize=,defaultIcon=,disabledIcon=,disabledSelectedIcon=,margin=javax.swing.plaf.InsetsUIReso
urce[top=2,left=14,bottom=2,right=14],paintBorder=true,paintFocus=true,pressedIcon=,rolloverEnabled=true,rolloverIcon=,rolloverSelectedIcon=
,selectedIcon=,text=Pushbutton without action,defaultCapable=true]
irb(main):010:0> basePanel.add labelPanel
=> javax.swing.JPanel[,0,0,0x0,invalid,layout=java.awt.FlowLayout,alignmentX=0.0,alignmentY=0.0,border=,flags=9,maximumSize=,minimumSize=,pr
eferredSize=]
irb(main):011:0> basePanel.add buttonPanel
=> javax.swing.JPanel[,0,0,0x0,invalid,layout=java.awt.FlowLayout,alignmentX=0.0,alignmentY=0.0,border=,flags=9,maximumSize=,minimumSize=,pr
eferredSize=]
irb(main):012:0> frame.add basePanel
=> javax.swing.JPanel[,0,0,0x0,invalid,layout=java.awt.FlowLayout,alignmentX=0.0,alignmentY=0.0,border=,flags=9,maximumSize=,minimumSize=,pr
eferredSize=]
Then it's just to pack the contents and show the frame:
irb(main):013:0> frame.pack
=> nil
irb(main):014:0> frame.show
=> nil
To continue playing the Swing, just poke away at the methods. You can always change it if something goes wrong. After you're finished you can dump most of the structure very easy. Just remember to call show again to see some of the updates. Certain changes call redraw directly, but mostly it's always a good choice to call show after updates on the structures.

The JRuby Tutorial #1: Getting started

This is the first installment in a series of tutorials I will write about how to use JRuby for fun and profit. I'm planning on covering a wide range of things you can do with JRuby, from Web frameworks with JDBC to writing your own JRuby extensions in Java. This time I'll take some time covering the basics; what you can do and how you can do it.

I'm writing these tutorials on Windows 2003, but most operations are not OS-centric, so hopefully people with other systems can use these instructions unchanged.

This tutorial will not teach neither Java or Ruby. A basic knowledge about both programming languages are assumed, but most examples will be in Ruby and focus on Ruby development with help from Java (the issue on JRuby extensions is a notable exception.)

What is JRuby
To quote the homepage: "JRuby is a 100% pure-Java implementation of the Ruby programming language." Almost complete support for all Ruby constructs work, and the idea is that you should be able to just use your Ruby scripts with JRuby instead of MRI (Matz' Ruby Implementation, the original Ruby, written in C). But the goal goes beyond that. Java is a powerful platform and there are millions of lines of Java code being written each month, that the world will have to live with for a long time from now. By leveraging Java the platform with the power of the Ruby programming language, programmers get the best from both worlds.

Limitations and caveats
Of course, nothing is simple, so JRuby can't do everything that MRI can. Some of these limitations are due to basic constraints in the Java platform. For example, JRuby will never support a fine-grained timed, until Java does it. Some File-operations just won't work through the Java API. Other limitations are due to differences in implementation, and will probably go away Any Time Now (TM). Some of these include the lack of support for continuations (which Ruby 2.0 won't support anyway), that JRuby's threads are real operating system threads instead of Green Threads (which Ruby 2.0 will have too). Some problems exist with extending Java classes from Ruby and having the result show up from Java. These issues are under development and will be available as soon as enough man hours get put on the problem.

Downloading and installing
The first step in downloading JRuby is to decide whether you should go with a released version (which would be 0.9.0 at this time of writing), or run JRuby from trunk. The last few months, development on JRuby have been quite hectic. Much has been done, and trunk already is quite far away from 0.9.0, both in terms of features, performance and bug fixes. This tutorial will be based on trunk, and if there are any special considerations regarding version, this will be noted in the relevant issue.

To download JRuby from trunk, you need to install a Subversion client. One of these can be found at http://subversion.tigris.org and is very easy to install. After you've downloaded and installed Subversion you're ready to decide where you want JRuby to call home. After you've decided this, you have to set an environment variable called JRUBY_HOME to this directory. For example, I would do this:
set JRUBY_HOME=d:\project\jruby-trunk
in a command window. (In the long run, it would be better to put this somewhere so that it always get set. In Win32 I go to the System configuration on my Control Panel, choose the Advanced tab and press Environment Variables.)

Now we're ready to check out the latest JRuby version:
svn co svn://svn.codehaus.org/jruby/trunk/jruby %JRUBY_HOME%
After you have downloaded JRuby you have to build it, so you have to have a recent version of Ant installed. To build and test JRuby, just enter these commands:
cd %JRUBY_HOME%
ant test
The testing will take a while, but it's good to make sure your installation is fully functional. After you've got the final "BUILD SUCCESSFUL", you can be reasonably sure that your JRuby is functional.

Using Java from Ruby
OK, so now that we've got JRuby up and running, we should try a simple example to see that it actually works correctly and that we can use Java classes to. Write this into a file called testJava.rb:
require 'java'
include_class 'java.util.TreeSet'
set = TreeSet.new
set.add "foo"
set.add "Bar"
set.add "baz"
set.each do |v|
puts "value: #{v}"
end

and run with the command
jruby testJava.rb
You will get this output:
value: Bar
value: baz
value: foo
So, what's happening in this small script is that we first include JRuby's java-support. This provides lots of interesting capabilities, but the most important one is the method Kernel#include_class, which is used to get hold of a Java-class. In most cases you can just include the classes you're interested in and use them as is. In our case, we instantiated the TreeSet as usual in Ruby, by calling new on the class. After this we added some strings to the set, and since JRuby automatically transforms Ruby-Strings into Java Strings, these get sorted by TreeSet in ASCII-order. You may have noticed that we could use a standard each-call to iterate over the entries in the set. This is one of the nice features of JRuby. Even though you're actually working with a Java Collection, it looks and feels (and quacks) like a regular Ruby collection object. All Java classes that implements java.util.Collection automatically gets an each method and includes Enumerable.

The regular include_class function does not work correctly if you try to import a Java class with the same name as an already existing Ruby class. This won't work, for example:
include_class 'java.lang.String'
since there already exists a class called String in Ruby. The first way of handling this problem is to rename the class when including it, by adding a block to the call to include_class:
include_class('java.lang.String') {|package,name| "J#{name}" }
In this case, the class added to the Ruby name space is called JString and doesn't collide with the Ruby String.

The second solution uses the fact that include_class includes the classes mentioned into the current name space, so you can call include_class from inside a module definition, and the classes will be available there:
module Java
include_class 'java.lang.String'
end
The Java String is now available as the class Java::String. If you don't want to include all classes specifically, you can also include a complete package, with include_package. It works exactly like include_class, but takes a package name instead, and makes all classes in that package (but not sub packages) available as classes in the current context.

This is mostly it. JRuby tries as hard as possible to follow the Ruby way, and make Java integration as natural as possible. The ideal is that you should be able to guess how it works, and it will work that way.

Conclusion
The last 6 months, JRuby have become a very real alternative to MRI. As performance gets better and better, the promise of integration with legacy Java systems will get more and more convincing. And using Ruby for tools and tasks where Java only incurs overhead is a very compelling way for Java developers to get the most out of their current tools.

The next issue is slated to be about using Camping from JRuby, together with ActiveRecord and JDBC. Camping is microframework, that together with ActiveRecord lets you write small applications very compact and efficient.

onsdag, augusti 16, 2006

Byte-lexing JRuby.

Since I begun the effort to recreate our lexer with JFlex, me, Charles and Wes Nakamura have discussed how MRI's lexer can be so much faster than JRuby's. One thing discussed was that it could maybe be because JRuby lexes with Readers, instead of handling bytes directly. I didn't believe this would be a big difference, but I set out to test it, anyway. Basically, I replaced all references to chars with bytes, and Readers to InputStreams in our Lexer. There were a few other places that had to be changed. I haven't done anything else to the lexer (there were many places were we could do some good optimization). The results amazed me. My first test case was completely focused on only the parsing step. There is no evaluation in these numbers. First, trunk JRuby:

Did 10_000 parses in 4336
Did 10_000 parses in 3885
Did 10_000 parses in 3866
Did 10_000 parses in 3835
Did 10_000 parses in 3836
Did 10_000 parses in 3815
Did 10_000 parses in 3836
Did 10_000 parses in 3896
Did 10_000 parses in 3845
Did 10_000 parses in 3926
-- Full time for 1_000_000 parses: 39076

and with my byte enhancements:

Did 10_000 parses in 3595
Did 10_000 parses in 3214
Did 10_000 parses in 3215
Did 10_000 parses in 3205
Did 10_000 parses in 3204
Did 10_000 parses in 3184
Did 10_000 parses in 3205
Did 10_000 parses in 3235
Did 10_000 parses in 3204
Did 10_000 parses in 3215
-- Full time for 1_000_000 parses: 32476

These times show about 18% increase in parsing speed, just by working with raw bytes instead of characters.

After I integrated the changes and fixed a few bugs, I finally could test the gain in regular JRuby evaluation speed. My test case this time consists of requiring webrick and IRB. First, trunk JRuby:

real 0m5.191s
user 0m0.000s
sys 0m0.010s

And with the byte-patch:

real 0m4.743s
user 0m0.010s
sys 0m0.010s

So, for this particular test case, we get about 5-8%, depending on circumstances. I would never have imagined that it was such an expense to work with Readers instead of InputStreams. The question now is; is it worth it? Should we sacrifice full unicode (or at least non-astral plane) reading for speed? I'm not sure.

söndag, augusti 13, 2006

Lexing Ruby.

It has become apparent that JRuby's hand coded Lexer is a liability, in the long run. The code is hard to maintain and probably suboptimal with regards to speed. So two weeks ago I decided to check out the possibility of a Lexer generator. I haven't had much time, but the last days I've started reimplementing the JRuby lexer with the help of JFlex.

The first parts have been really easy, actually. I already have a simple version tied in with JRuby, working. There's not much of the syntax there yet, but some of it works really fine. Fine enough to check performance and see the road ahead. So, I have two test cases, the first one looks like this:


class H < Hash
end

puts H.new.class
puts H[:foo => :bar].class

and the second like this:

class H < Hash
def abc a, b, *c
1_0.times {
puts "Hellu"
}
end

def / g, &blk
2.times do
puts "well"
end
end
end

H.new.abc 1, 2
H.new/3

The first one test a corner case in subclassing core JRuby classes. The second one is just nonsense to test different parts of the syntax. Both of these parse and run correctly on JRuby with the JFlex-based lexer. It's not much, but it's something to build on.

Regarding performance, I've created a test program that uses the JRuby parser to parse a specified file 10 x 10_000 times, reporting each time and the total. I've run it on both the first and the second example, and both are about 8 to 10 percent faster with the new lexer. I also expect performance to improve more for bigger files. Right now the lexer keeps track on line number, offset and column number, which is also a performance drain. Removing it gives about 2-3 percent more.

fredag, augusti 11, 2006

Speed in JRuby.

I had really wanted my next post here to be a tutorial about how to get Camping working with ActiveRecord-JDBC and JRuby, but I managed to find two quite serious bugs related to blocks while trying things out. One of the bugs is really strange, and manifests in some of the Markaby CSS-Proxy internals. So, Markaby works, if you refrain from add css class names to tags.

Since Monday I've been busy with various things in JRuby. I've looked around for easy performance fixes, which there are loads of. Many of them didn't give much, but a few things actually had a noticeable difference. One of these places was in the loading process, where it's important to do as few searches as possibly, since most of them are really expensive. Actually, in a few situations JRuby actually parsed and ran a file several times. This happened in WEBrick among others.

I've also implemented a Java version of Enumerable, which was very trick to get working at first, since JRuby don't really support calling methods that yield from Java. So, I had to devise a way of doing this. The generic case seems to be to hard to do right now, but the specific case of calling each and collect the results in a List are finally working well.

I devised a test case where all Enumerable-operations are used 2 or 3 times, and ran this test case 1_000.times. The results for trunk JRuby was ~7s, while the Java implementation of Enumerable took that down to ~5s, which is nice.

Anyway. That's mostly all. I'm off on vacation now.

JvYAML status

I have finally started work on JvYAML again, and I hope to have a new release together within 2 weeks, which will contain JavaBean materialization and the completed emitter. Hopefully I'll manage to integrate this with JRuby not long after.

It's been really hard to get going with this, since I have so many different projects going simultaneously.

måndag, augusti 07, 2006

Some JRuby tidbits

I've spent some time this weekend taking a look at various JRuby issues.

  • Camping:
    The most interesting was trying to get Camping to work. Camping is a microframework for certain kinds of web applications. Very neat, but uses lots of Ruby tricks to function. As such it's a really good test of JRuby capabilities, but it's quite hard to debug. I have got it working really good for basic applications, in the process finding a bug in our Hash#[] implementation that fortunately was easy to fix. But today I've unearthed something really hairy. The test case demands two classes with two methods each, and it's some really strange block tweaking that happens. I hope Charlie is up to the challenge. Markaby (which Camping uses), needs this functionality to work.
  • Mongrel:
    I've started to take a serious look at the Mongrel support. Danny's work on the parser seems really great, so I've added basic JRuby integration to our Subversion, and plan to try getting it to work with Mongrel-0.4 later on today.
  • ActiveRecord-JDBC:
    Last week I added some Oracle functionality to it, but this still needs some work. I'm thinking that we need to factor out some driver-specific functionality soon. Anyway, I released the a version 0.0.1-gem so that people easily can use what functionality we have.
  • JvYAML:
    I've finally begun writing on the Emitter in ernest again. I hope to have most of it finished by Friday, before I go on vacation.
Expect a tutorial on using JRuby and Camping together very soon; hopefully tomorrow. And in a few days after, maybe we can showcase a functional Mongrel in JRuby too?

torsdag, augusti 03, 2006

Announcing Swedish Rails

I would like to announce my plugin Swedish Rails.

Rails provide many goodies and helpers for you, but some of these are dependant on your user interface being in english. Date controls, for example. Pluralization is another. And have you ever tried to capitalize a string with Swedish letters in them? Then you know it doesn't work that well. Until now, that is. You install this plugin, make sure all your Swedish source uses UTF-8, and most of these problems disappear. Downcase, upcase, swapcase and capitalize work as expected. Integer#ordinalize gives Swedish ordinals. And month names and weekday names are in Swedish.

This plugin isn't only for people looking to create Swedish Web interfaces. It can also act as a map for creating a plugin like this for any western language. Just replace all the text-strings in this plugin, change the module name and you're set to go. (Of course you'll have to know the language you're porting too, though).

The plugin can be found at:
http://svn.ki.se/rails/plugins/swe_rails
and some information here:
http://opensource.ki.se/swe_rails.html

The plugin itself is fairly simple. Most of it monkey patches different parts of Ruby proper or aspects of Rails. One of the more interesting parts (which could also be usable by itself) translates all internal month and weekday names in Swedish. This is probably the most interesting when doing Rails GUI's, since you don't have to create your own date_select tag, and all the other variations on this. I've also added som inflector rules, but only for the pluralize helper (so don't start naming database tables in Swedish, please!). The problem is that english pluralization rules are pretty simple. Just tuck an -s on the end and hope for the best. Swedish uses 5 or six declinations for nouns, and most of these are irregular. This makes it slightly hard to describe inflection rules for Swedish, but I gave it a try anyway.

So, enjoy! I just wish I had done this a year ago. Then I wouldn't have had to write sublty different versions of this code all over the place.

onsdag, augusti 02, 2006

Rails, SOAP4R and Java.

I've spent the last three weeks working part time on a project called LPW at work. LPW is a set of web services that talk with the Swedish student databases. The libraries are implemented in Java and deployed with Axist. My project was to create a web system that students can use to access various LPW services. We had a an old implementation of this, written in Java with uPortal as a framework, but for the new implementation we decided that using Ruby on Rails would be interesting and probably worthwhile.

I've spent about 40 hours on the implementation. I did everything except the user interface. I got finished HTML-pages and integrated these with the system. Initially we expected the complete development to take about 120-150 hours with Ruby and about 250 if we did it in Java. In retrospect, I'm pretty sure the Java version would have taken more than that; probably between 350 and 400 hours. Since I wrote the first version in Java I can say this pretty accurately.

So, did I have any interesting experiences while writing this system? Oh yes; otherwise I wouldn't be writing. First of all, I managed to release two plugins in the process of this project. More information about them can be found in older blog entries. I also tried Mongrel for the first time, and I just have to say that I will never go back to WEBrick.

But the most interesting part with using Ruby was that I would have to get SOAP4R and Axis to work together. In the process I found some interesting things. Let me describe the relevant layout of my application.

The project goal was to implement 4-6 different services, which are all available as separate WSDL-files. I generated the drivers for these with wsdl2ruby. After generating the clients for each, I renamed the driver.rb into for example addr_driver.rb, and created a file called lpw_valueobjects.rb where I put all valueobjects found in defaultDriver.rb. I then repeated this process for each service. I then put all these files into the directory RAILS_ROOT/vendor/lpw.

Now, hitting one or two web services each time a student goes to a page isn't really realistic, so I decided early to implement some way of caching the results of the service calls. Since the data in question is pretty static this works well.

So, to integrate the services into Rails, I created a base class called LPWObject in the models-directory. In this class I defined self.wsdl_class and a few other helpers that let me transparently handle the caching of service-classes without actually having them inside the LPWObject itself. This becomes important when I want to save the results in the session. Ruby can't marshal classes which have singleton methods, and WSDL2Ruby depends heavily on singleton methods for the service client.

The first problem with getting Ruby and Java to work over Soap was with swedish characters. When I added something like "Gävlegatan 74" to an attribute and then tried to send this over soap, Soap4R transformed the attribute type into Base64 instead of xsd:String. Suffice to say, Axis didn't really like this. The solution was to add $KCODE="UTF8" to environment.rb. This let's Soap4R believe that åäöÅÄÖ is part of regular strings.

The next problem came when I tried to save some of the value objects into the session. After looking for a long while, I found that if the value object had an attribute called "not", WSDL2Ruby didn't generate an
attr_accessor :not
for this until at runtime, which creates singleton methods on the value object. The solution was to add these accessors by myself. I'm not sure why wsdl2ruby does it like this, but probably there is some weird interaction with the not keyword in Ruby.

The final problem - which I'm not sure if I should blame Axis or Soap4R for - came when one of the value objects contained a byte array with a PDF-file. For some reason Axis sends the regular response XML looking fine, but before and after there are some garbled data. It looks like Axis actually sends the byte data as an attachment too, not just inside the byte array. Soap4R didn't handle this at all, and I got an "illegal token" from the XML parser. The solution to this problem is the worst one, and I should really send a bug report about this, but I haven't had the time yet. Anyway, to fix it, I monkeypatched SOAP::HTTPStreamHandler, and aliased the method send. Inside my new send method I first called the old one, then use a regexp to extract only the xml-parts of the response and reset the receive_string to this. It fixes my problem and works fairly well, but it isn't pretty.

So, in conclusion, Soap4R and Axis seem to work good together, except for a few corner cases. I'm really happy about a project that could've taken 10 times longer to complete, though.