How large is your .emacs?

I've been reading lots of blogs and opinions about emacs the last few days. What strikes me is all of these people who brag about how large their .emacs files have become. So let me make this very clear:

If your .emacs file is longer than a page YOU ARE DOING IT WRONG.

Why? Well. Unless you are a casual Emacs user, your .emacs should not be regarded as a configuration file. Rather, the .emacs file is actually the entry point to the source repository of your own version of Emacs. In effect, when you configure Emacs you create a fork, which has it's own source that you need to maintain. This is not configuration. This is programming and you should approach it like you do all programming. What does that mean? Modularization. Clean code. Code comments. Source control. Tests. But modularization and source control are the ones that are most important for my Emacs configuration. I have loads of files in ~/emacs and every kind of extension I do has it's own kind of file or directory to put it in. The ~/emacs directory is checked out from source control, and has got customizations for different platforms. That's why my .emacs file is 4-5 lines long. Two for setting customizations that are specific to this computer, and the rest to load the stuff inside of ~/emacs. And that's all.

So how do you handle modularization of Emacs Lisp code? This won't be a tutorial. Just a few advices that might make things easier.

In no specific order:

• (load "file.el") will allow you to just load another file.
• (require 'cl) will give you lots of nice functionality from Common Lisp
• I recommend you have one place where you add all your load paths. Mine look something like this:

  (labels ((add-path (p)
          (add-to-list 'load-path
                       (concat emacs-root p))))
  (add-path "emacs/jde/lisp")
  (add-path "emacs/nxml")
  (add-path "emacs/own")             ;; Personal elisp code
   (add-path "emacs/lisp")            ;; Various elisp code, just dumped here
  )

• Why do it like this? Well, it gives you an easier way to add full paths to your load path without repeating lots of stuff. This depends on you defining emacs-root somewhere - do define it, it can be highly useful.
• Set custom-file. (The custom-file is the file where Emacs saves customizations. If you don't set a specific file for this, you will end up getting all customizations saved into .emacs which you really don't want.) The code for this is simple. Just do (setq custom-file "the-file-name.el")
• Use hooks and advice liberally. They allow you to attach new functionality without monkey patching.
  
• If you ever edit XML, NEVER use Emacs builtin XML editor. Instead download the excellent NXML package.
• Learn how to use Info and customizations
• Use Ido mode

Feel free to add other good advice in the comments. These were just a small smattering of stuff I like and which helps your environment quite seriously. But the most important part is the whole thing about keeping your .emacs extremely small!

Addendum: As Phil just pointed out (and which was part of my plan from the beginning) is that Autoloads should be used as much as possible. Also, make sure to bytecompile as much as possible.

Rubinius and Lisp

Yesterday on the #rubinius-channel, me, Evan and MenTaLguY had the beginnings of a discussion about a Lisp dialect for the Rubinius VM. Since this sound exactly like the kind of thing I've been thinking about for some time, I got excited. But anyway, Pat Eyler covered the issue much better and did a small interview with us that can be read here. Ruby is wonderful. Rubinius is sweet. With Lisp added to the mix, it will be better.

I'll probably follow this up with some more information and ideas, as soon it's fleshed out.

An Emacs diversion: Font sizes

After a few weeks of very intense JRuby-OpenSSL hacking, I felt the need to do something different, so I've spent a few hours with my slightly rusty Emacs Lisp skills, trying to fix something that I really need. Namely, control over font-size and fonts in Emacs, on Linux. I want it inside Emacs and customizable by EL. To my surprise I couldn't find anything like that anywhere.

For me personally, it's necessary when presenting, since I usually code with a small font in Emacs, the code will be totally unreadable when presenting. And since I don't have a fancy MacBook Pro, I need to be able to zoom in and out inside Emacs.

Presto, it wasn't easy, but I've managed it. For some reason, font handling seems quite backward in Emacs. I had to extract the current font, and then split it and join the new array together again. Not neat and my way of doing it is not the best. But, for your pleasure, here is the code to do it, and also some code that establishes a font ring of the standard fonts in different sizes that can be walked through:

(defun inc-font-size ()
 (interactive)
 (let* ((current-font (cdr (assoc 'font (frame-parameters))))
        (splitted (split-string current-font "-"))
        (new-size (+ (string-to-number (nth 7 splitted)) 1))
        (new-font (concat (nth 0 splitted) "-"
                          (nth 1 splitted) "-"
                          (nth 2 splitted) "-"
                          (nth 3 splitted) "-"
                          (nth 4 splitted) "-"
                          (nth 5 splitted) "-"
                          (nth 6 splitted) "-"
                          (number-to-string new-size) "-*-"
                          (nth 9 splitted) "-"
                          (nth 10 splitted) "-"
                          (nth 11 splitted) "-*-"
                          (nth 13 splitted))))
   (if (> (length splitted) 14)
       (dotimes (n (- (length splitted) 14))
         (setq new-font (concat new-font "-" (nth (+ n 14) splitted)))))
   (set-default-font new-font t)
   (set-frame-font new-font t)))

(defun dec-font-size ()
 (interactive)
 (let* ((current-font (cdr (assoc 'font (frame-parameters))))
        (splitted (split-string current-font "-"))
        (new-size (- (string-to-number (nth 7 splitted)) 1))
        (new-font (concat (nth 0 splitted) "-"
                          (nth 1 splitted) "-"
                          (nth 2 splitted) "-"
                          (nth 3 splitted) "-"
                          (nth 4 splitted) "-"
                          (nth 5 splitted) "-"
                          (nth 6 splitted) "-"
                          (number-to-string new-size) "-*-"
                          (nth 9 splitted) "-"
                          (nth 10 splitted) "-"
                          (nth 11 splitted) "-*-"
                          (nth 13 splitted))))
   (if (> (length splitted) 14)
       (dotimes (n (- (length splitted) 14))
         (setq new-font (concat new-font "-" (nth (+ n 14) splitted)))))
   (set-default-font new-font t)
   (set-frame-font new-font t)))

(defvar *current-font-index* 0)

(defconst *font-ring* '(
                       "-urw-nimbus mono l-regular-r-normal--15-*-88-88-p-*-iso8859-1"
                       "-urw-nimbus mono l-regular-r-normal--17-*-88-88-p-*-iso8859-1"
                       "-Adobe-Courier-Medium-R-Normal--14-*-100-100-M-*-ISO8859-1"
                       "-Adobe-Courier-Medium-R-Normal--16-*-100-100-M-*-ISO8859-1"
                       "-Adobe-Courier-Medium-R-Normal--18-*-100-100-M-*-ISO8859-1"
                       "-Adobe-Courier-Medium-R-Normal--20-*-100-100-M-*-ISO8859-1"
                       "-Adobe-Courier-Medium-R-Normal--22-*-100-100-M-*-ISO8859-1"
                       "-Adobe-Courier-Medium-R-Normal--24-*-100-100-M-*-ISO8859-1"
                       "-Adobe-Courier-Medium-R-Normal--26-*-100-100-M-*-ISO8859-1"
                       "-Adobe-Courier-Medium-R-Normal--28-*-100-100-M-*-ISO8859-1"
                       "-Adobe-Courier-Medium-R-Normal--30-*-100-100-M-*-ISO8859-1"
                       "-Adobe-Courier-Medium-R-Normal--32-*-100-100-M-*-ISO8859-1"
                       "-Adobe-Courier-Medium-R-Normal--34-*-100-100-M-*-ISO8859-1"
                       "-Adobe-Courier-Bold-R-Normal--14-*-100-100-M-*-ISO8859-1"
                       "-Adobe-Courier-Bold-R-Normal--16-*-100-100-M-*-ISO8859-1"
                       "-Adobe-Courier-Bold-R-Normal--18-*-100-100-M-*-ISO8859-1"
                       "-Adobe-Courier-Bold-R-Normal--20-*-100-100-M-*-ISO8859-1"
                       "-Adobe-Courier-Bold-R-Normal--22-*-100-100-M-*-ISO8859-1"
                       "-Adobe-Courier-Bold-R-Normal--24-*-100-100-M-*-ISO8859-1"
                       "-Adobe-Courier-Bold-R-Normal--26-*-100-100-M-*-ISO8859-1"
                       "-Adobe-Courier-Bold-R-Normal--28-*-100-100-M-*-ISO8859-1"
                       "-Adobe-Courier-Bold-R-Normal--30-*-100-100-M-*-ISO8859-1"
                       "-Adobe-Courier-Bold-R-Normal--32-*-100-100-M-*-ISO8859-1"
                       "-Adobe-Courier-Bold-R-Normal--34-*-100-100-M-*-ISO8859-1"
                       "-Misc-Fixed-Medium-R-SemiCondensed--10-*-75-75-C-*-ISO8859-1"
                       "-Misc-Fixed-Medium-R-SemiCondensed--12-*-75-75-C-*-ISO8859-1"
                       "-Misc-Fixed-Medium-R-SemiCondensed--13-*-75-75-C-*-ISO8859-1"
                       "-Misc-Fixed-Medium-R-Normal--13-*-75-75-C-*-ISO8859-1"
                       "-Misc-Fixed-Medium-R-Normal--14-*-75-75-C-*-ISO8859-1"
                       "-Misc-Fixed-Medium-R-Normal--15-*-75-75-C-*-ISO8859-1"
                       "-Misc-Fixed-Medium-R-Normal--16-*-75-75-C-*-ISO8859-1"
                       "-Misc-Fixed-Medium-R-Normal--17-*-75-75-C-*-ISO8859-1"
                       "-Misc-Fixed-Medium-R-Normal--18-*-75-75-C-*-ISO8859-1"
                       "-Misc-Fixed-Medium-R-Normal--19-*-75-75-C-*-ISO8859-1"
                       "-Misc-Fixed-Medium-R-Normal--20-*-75-75-C-*-ISO8859-1"
                       "-Schumacher-Clean-Medium-R-Normal--12-*-75-75-C-*-ISO8859-1"
                       "-Schumacher-Clean-Medium-R-Normal--13-*-75-75-C-*-ISO8859-1"
                       "-Schumacher-Clean-Medium-R-Normal--14-*-75-75-C-*-ISO8859-1"
                       "-Schumacher-Clean-Medium-R-Normal--15-*-75-75-C-*-ISO8859-1"
                       "-Schumacher-Clean-Medium-R-Normal--16-*-75-75-C-*-ISO8859-1"
                       "-Schumacher-Clean-Medium-R-Normal--17-*-75-75-C-*-ISO8859-1"
                       "-Schumacher-Clean-Medium-R-Normal--18-*-75-75-C-*-ISO8859-1"
                       "-Sony-Fixed-Medium-R-Normal--14-*-100-100-C-*-ISO8859-1"
                       "-Sony-Fixed-Medium-R-Normal--16-*-100-100-C-*-ISO8859-1"
                       "-Sony-Fixed-Medium-R-Normal--18-*-100-100-C-*-ISO8859-1"
                       "-Sony-Fixed-Medium-R-Normal--20-*-100-100-C-*-ISO8859-1"
                       "-B&H-LucidaTypewriter-Medium-R-Normal-Sans-14-*-100-100-M-*-ISO8859-1"
                       "-B&H-LucidaTypewriter-Medium-R-Normal-Sans-16-*-100-100-M-*-ISO8859-1"
                       "-B&H-LucidaTypewriter-Medium-R-Normal-Sans-18-*-100-100-M-*-ISO8859-1"
                       "-B&H-LucidaTypewriter-Bold-R-Normal-Sans-20-*-100-100-M-*-ISO8859-1"
                       "-B&H-LucidaTypewriter-Bold-R-Normal-Sans-24-*-100-100-M-*-ISO8859-1"
                       "-B&H-LucidaTypewriter-Bold-R-Normal-Sans-30-*-100-100-M-*-ISO8859-1"
                       "-B&H-LucidaTypewriter-Bold-R-Normal-Sans-34-*-100-100-M-*-ISO8859-1"
                           ))

(defun font-next ()
 (interactive)
   (let ((len (length *font-ring*))
         (next-index (+ *current-font-index* 1)))
     (if (= next-index len)
         (setq next-index 0))
     (setq *current-font-index* next-index)
     (message (concat "setting " (nth *current-font-index* *font-ring*)))
     (set-default-font (nth *current-font-index* *font-ring*) t)
     (set-frame-font (nth *current-font-index* *font-ring*) t)))

(defun font-prev ()
 (interactive)
   (let ((len (length *font-ring*))
         (next-index (- *current-font-index* 1)))
     (if (= next-index 0)
         (setq next-index (- len 1)))
     (setq *current-font-index* next-index)
     (set-default-font (nth *current-font-index* *font-ring*) t)
     (set-frame-font (nth *current-font-index* *font-ring*) t)))

(defun font-current ()
 (interactive)
 (cdr (assoc 'font (frame-parameters))))

(defun font-set (ix)
 (setq *current-font-index* ix)
 (set-default-font (nth *current-font-index* *font-ring*) t)
 (set-frame-font (nth *current-font-index* *font-ring*) t))

(provide 'fontize)

I also bound these methods to keys, like this:

(global-set-key [?\C-+] 'inc-font-size)
(global-set-key [?\C--] 'dec-font-size)
(global-set-key [?\M-+] 'font-next)
(global-set-key [?\M--] 'font-prev)

Hope this helps someone in the same situation.

Three ways to add Ruby Macros

As most of my readers probably have realized at this point, I have a few obsessions. Lisp and Ruby happens to be two of the more prominent ones. And regarding Lisp, macros is what especially interest me. I have been doing much thinking lately on how you could go about adding some kind of macro facility to Ruby and these three options are the result.

I should begin by saying that none of these options are entirely practical right now. All of them have some serious problems which I frankly haven't been able to come up with an answer for yet. But that doesn't stop me from blogging about my ideas, of course. Another thing to notice is that this is not about hygienic macros. This is the full-blown, power, blow-the-moon away version of macros.

MacRuby - Direct defmacro in Ruby
The first approach rests on modifying the language itself. You can add a defmacro keyword which takes a name and a code block to execute. Each time the compiler/interpreter finds a macro-definition, it will remember the name. When that name is found in the code later on each place will be marked. Then, before execution begins, all places where the call to the macro are will be replaced by the output from sending in the subnodes at that place by the output of calling the macro. An example of a simple macro:

 defmacro log logger, level, *messages
if $DEBUG
  :call, logger, level, *messages
else
  :nop
end
end

log @l, :debug, "value is: #{very_expensive_operation()}"

What's interesting in this case is that the messages will not be evaluated if the $DEBUG flag is not set. This is because the value returned from the macro will be spliced into the AST only if that flag is set. Otherwise a no-op will be inserted instead. Obviously, for this kind of code to work, the interpreter would need to change substantially. There is also a big problem with it, since it's very hard to fit this model into the object-oriented system of Ruby. As I think about it now, it seems macros would be the only non-OOP feature in Ruby, if added in this way. Another big problem with this model is that it is really not that intuitive what the resulting code from the macro will be. As soon as something more advanced needs to be returned, it will be very hard getting it straight in your head. One solution to this would be to do it the standard CL way. First write the output from the macro in several different instances. Then transform this to the AST code through a tool that parses the code. Then transform this into the macro. This process would be helped by tools, of course.

Back-and-Lisp-Ruby - Write macros in Lisp, translate Ruby back and forth
Another way to achieve this power in Ruby would be to separate the macro language from the main language. In effect, the macros would be a classic pre-processor. To offer the same power level as Lisp and others, the best way would be to write the macros themselves in a Lisp dialect, then transform Ruby in a well-defined way to Lisp and back again. (See the next version for more about this idea.) In this situation the same macro as before could look like this:

 (defmacro log (logger level &rest messages)
 (if $DEBUG
     `(,level ,logger ,@messages)
     '()))

The main difference in this code is that the macro and the output from the macro is Lisp. We have gotten rid of the ugly :call and :nop return values, and to me this seems quite readable. Of course, I'm not sure everyone else feels the same way. And we still have the same problem with Object Orientedness. It's missing.

RoCL - Ruby over Common Lisp
The final idea is to build a Ruby runtime within Common Lisp and transform Ruby into Common Lisp before running it. The macros could either be added as Ruby code or Lisp code. Everything will be transformed into the equivalent code in Lisp, maybe using CLOS as the Object-system, or building something based on Ruby's. Of course, the semantics of many things would change, and many libraries would need to rewritten. But in the end, there would be incredible power available. Especially if we can make it go both ways, so that Common Lisp can use Ruby libraries.

An example transformation could look like this. From this Ruby:

 class String
  def revert(a, *args)
    if block_given?
      yield a
    else
      args + [a]
    end
  end
end

"abc".revert "one" do |x|
  puts x
end

This is nonsense code, if you hadn't noticed. =)

 (with-class "String" nil
            (def revert (a block &rest args)
              (if block
                  (apply block a)
                  (+ args [a]))))
(revert "abc" "one" #'(lambda (x)
                        (puts self x)))

Conclusions
It is very hard to actually retrofit macros into Ruby after the fact. I'm still not sure it can be done and keep enough of Ruby's semantics to make it meaningful. It seems that we need a new language. But if I had to choose among these approach, the RoCL one seems the most interesting and also the most fun to implement. If I have a motto it would have to be something in the line of "best of all worlds". I want the best from Ruby, Java, Lisp, Erlang and everything I can find.

The Dark Ages of programming languages

We seem to be living in the dark ages of programming languages. I'm not saying this to bash everything; I'm actually being totally objective right now. Obviously, our situation right now is much better than it was 10 years ago. Or even 5 years ago. I would actually say that it's really much better now, than 1 year ago. But programming is still way too painful in almost all cases. We are doing so much stuff by hand that obviously should be done be computer.

I spend quite much time learning new languages now and then, to try to find something that's really good for me. So far, the best contestants are Ruby, Erlang, OCaml and Lisp, but all of those have their share of problems too. They just suck less than the alternatives.

• Ruby... I really like Ruby. Ruby is such an improvement that I really want to do almost everything in it nowadays. I think in Ruby half the time and in Lisp the other half. But it's not enough. It is still clunky. I want tail calls. I want real macros. I want blazing speed and complete integration with good libraries for everything and more. I'm just a sucker for power, and I want more of it in Ruby.
• Erlang and OCaml. These languages are really great. For specific applications. Specifically, Erlang is totally superior for concurrent programming. And OCaml is incredibly fast, very typesafe and has great GUI libraries. So, if I was asked to do something massively concurrent I would probably choose Erlang, and OCaml if it was GUI programming. But otherwise... Well, Erlang does have some neat functional properties, but not any nice macro support. It doesn't have a central code repository and many other things you expect from a general purpose language. OCaml suffers from the same things.
• Lisp is the love of my life. But as so many people before me has noted, all the implementations are bad in some way or another. Scheme is lovely; for research. Common Lisp is so powerful, but it needs users. Lots of them, creating libraries for every little data format there can be, creating competing implementations of particularly important API's; like databases.

Conclusion. Nothing is good enough, right now. I see two two paths ahead. Two ways that could actually end in the "100-year language".

The first path is one new language. This language will be based on all the best features of all current languages, plus a good amount of research output. I have a small list what this language would need to be successful as the next big one:

• It needs to be multiparadigm. I'm not saying it can't choose one paradigm as the base, but it should be possible to program in it functionally, OOP, AOP, imperative. It should be possible to build a declarative library so you can do logic programming without leaving the language.
• It should have static type inference where possible. It should also allow optional type hints. This is so important for creating great implementations. It can also increase readability in some cases.
• It needs all the trappings of functional languages; closures, first-order functions and lambdas. This is essential, to avoid locking the language into an evolutionary corner.
• It needs garbage collection. Possibly several competing implementations of GC's, running evolutionary algorithms to find out which one is best suited for long running processes of the program in question.
• A JIT VM. It seems almost a given right now that Virtual Machines are a big win. They can also be made incredibly fast.
• Another JIT VM.
• A non-VM implementation. Several competing implementations for different purposes is important to allow competition and experimentation with new features of implementation.
• Great integration with legacy languages (Java, Ruby (note, I'm counting on all Rubyists moving to this new language when it gets out, making Ruby legacy), Cobol). This is obvious. There are to many things lying around, bitrotting, that we will never get rid of.
• The language and at least one production quality implementation needs to be totally open-source. No lock-in of the language should be possible.
• Likewise, good company support is essential. A language needs money to be developed.
• A centralized code/library repository. This is one of Java's biggest failings. Installing a new library in Java is painful. We need something like CPAN, ASDF, RubyGems.
• The language needs great, small and very orthogonal libraries. The libraries included with the language needs to be great, since they have to be small but still pack all the most needed punch.
• Concurrency must be a breeze. There should be facilities in the language itself for making this obvious. (Like Erlang or Gambit Scheme).
  
• It should be natural to do meta-programming in it (in the manner of Ruby).
• It should be natural to solve problems bottom-up, by implementing DSL's inside or outside the language.
• The languages needs a powerful macro facility that isn't to hard to use.
  
• Importantly, for the macro facility, the language needs to have a well-defined syntax tree of the simplest possible kind, but it also needs to have optional syntax.

So, that's what I deem necessary (but maybe not sufficient) for a really useful, good, long term programming language. When I read this list, it doesn't seem that probables that this language will show up any time soon, though. Actually, it seems kinda unrealistic.

So maybe the other way ahead is the right one? The other way I envision is that languages become easier and easier to create, and languages have their strength in different places. Along this path I envision the descendants of Ruby and Erlang exploiting what they're good at and eschewing everything else. But for this strategy to work, the first thing implemented in each language needs to be a seamless way to integrate to other languages. Maybe there will come an extremely good glue-language (not like Perl or Ruby, but a language that only will serve as glue between programming languages), and all languages will implement good support for that language. For example you could code a base Erlang concurrent framework, which uses G (the glue language) to implement some enterprise functionality in Java sandboxes, and some places where Ruby through G will implement a DSL, which have subparts where Ruby uses G to run Prolog knowledge engines.

If you had to choose among the two futures, I am frankly more inclined towards the one-language one. But the multi-language way seems much more probable. And since I'm trying to choose way now, I'm placing my bets on the second option. We are not ready to implement G yet, but I do think that as many p-language techs as possible should do their best to learn how languages can cooperate in different ways, to prepare this project.