The contract of IO#read

It's interesting. After Charlie made an immense effort and rewrote our IO system, basically from scratch, I have started to find bugs. But these are generally not bugs in the IO code, but bugs in Ruby libraries that depend on the way MRI usually works. One of the more annoying ones are IO#read(n), where n is the length you want to read.

This method is not guaranteed to return a string of length n, even if we haven't hit EOF yet. You can NEVER be sure that what you get back is the length you requested. Ever. If you have code that doesn't check the length of the returned string from read, you are almost guaranteed to have a bug just waiting to happen.

Of course, it might work perfectly on your machine and every other machine you test it on. The reason for this is that read(n) will usually return n bytes, but that depends on the socket implementation or file reading implementation of the operating system, it depends on the size of the cache in the network interface, it depends on network latency, and many other things. Please, just make sure to check the return values length before going ahead and using it.

Case in point: net/ldap has this exact problem. Look in net/ber.rb and you will see. There are also two - possibly three - bugs (couple of years old too) that reports different failures because of this.

One thing that makes this problem hard to find is the fact that if you insert debug statement, you will affect the timing in such a way that the code might actually work with debug statement but not without them.

Oh, did I mention that StringIO#read works the same way? It has exactly the same guarantees as IO#read.

Silly Io experiment

I have known for some time that Io is quite powerful as a language. I wanted to check if it was also suited for creating DSLs. Since I'm lazy I didn't want to create a new DSL just for this, so I decided to appropriate the associations language from ActiveRecord.

The process I followed was this - first come up with something that looks passably nice, then try to implement it. The only requirements was that the names of the associations were saved away somewhere, once per prototype for each model. This is the syntax I came up with (well OK, I didn't work very long on it...):

Post := IoRecord {
has many authors
belongs to blog
belongs to isp
}

Author := IoRecord {
has many blogs
has many posts
has one name
}

Actually, this looks almost readable, right? It's all valid Io code. The question is, how do we get it to do what we want? Without further ado, here's an implementation that gives us enough:

IoRecord := Object clone do(
 init := method(
   self belongings := list
   self hasOnes := list
   self hasManies := list
   self hasFews := list
 )

 appender := method(msg,
   blk := block(
     call sender doMessage(msg) append(call message next name)
     call message setNext(call message next next)
   )
   blk setIsActivatable(true)
 )

 collector := method(
   meths := call argCount / 2
   waiter := Object clone
   for(index, 0, meths-1,
     waiter setSlot(call argAt(index*2) name,
       appender(call argAt(index*2+1)))
   )
   waiter
 )

 belongs := collector(
   to, belongings
 )

 has := collector(
   many, hasManies,
   one, hasOnes
 )

 curlyBrackets := method(
   current := self clone
   call message setName("do")
   current doMessage(call message)
 )
)

Since I'm a bad person and really enjoys metaprogramming, I had to get rid of most of the duplication the first implementation contained. Let's say it like this: the first version didn't have the collector and appender methods. And boy do they make a difference. This is real metaprogramming. Actually, these two methods are actually macros, almost as powerful as Common Lisps. Notice that the words we send in to collector doesn't actually get evaluated. This is one of the reasons we don't need to use symbols - we can just take the unevaluated messages and take their name. In the appender macro we're doing something really funky where we use setNext.

The final effect of that setNext call is that in something like "has many foobar", Io will not try to evaluate foobar at all. In fact, even before Io tries this, we remove the foobar message from the chain and inserts the next message instead.

Oh, right, and the Io lexer actually inserts a synthetic message called "curlyBrackets" when it finds curly brackets. The brackets themselves actually act exactly like parenthesis. You can try this out in Io by changing the closing parenthesis to a closed curly bracket instead - Io doesn't care. Your brackets doesn't have to match up in type, just in cardinality.

Of course, this is just the beginning. Io will blow your mind.

The thing I'm finding myself missing is a good literal syntax for hashes and lists. I'm thinking about implementing something for that. Also, using atPut is starting to get annoying. I want square brackets access. Shouldn't be too hard, since Io does the same thing with square brackets as it does with curly brackets. Almost, at least.

And btw, the Io standard library... Not sure what I think about it. I miss many things from the Ruby core library actually. Now, Io combined with the Ruby libraries, that might be fun?

Emacs Inferior mode for the Io language

For anyone who like the Io language and work with Emacs, here is a small Inferior mode for Emacs and Io.
It's adapted from the Ruby Inferior mode. Download it here. To use it, just do "(require 'inf-io)" and then M-x run-io.

Enjoy.