fredag, september 22, 2006

The Dark Ages of programming languages

We seem to be living in the dark ages of programming languages. I'm not saying this to bash everything; I'm actually being totally objective right now. Obviously, our situation right now is much better than it was 10 years ago. Or even 5 years ago. I would actually say that it's really much better now, than 1 year ago. But programming is still way too painful in almost all cases. We are doing so much stuff by hand that obviously should be done be computer.

I spend quite much time learning new languages now and then, to try to find something that's really good for me. So far, the best contestants are Ruby, Erlang, OCaml and Lisp, but all of those have their share of problems too. They just suck less than the alternatives.

  • Ruby... I really like Ruby. Ruby is such an improvement that I really want to do almost everything in it nowadays. I think in Ruby half the time and in Lisp the other half. But it's not enough. It is still clunky. I want tail calls. I want real macros. I want blazing speed and complete integration with good libraries for everything and more. I'm just a sucker for power, and I want more of it in Ruby.
  • Erlang and OCaml. These languages are really great. For specific applications. Specifically, Erlang is totally superior for concurrent programming. And OCaml is incredibly fast, very typesafe and has great GUI libraries. So, if I was asked to do something massively concurrent I would probably choose Erlang, and OCaml if it was GUI programming. But otherwise... Well, Erlang does have some neat functional properties, but not any nice macro support. It doesn't have a central code repository and many other things you expect from a general purpose language. OCaml suffers from the same things.
  • Lisp is the love of my life. But as so many people before me has noted, all the implementations are bad in some way or another. Scheme is lovely; for research. Common Lisp is so powerful, but it needs users. Lots of them, creating libraries for every little data format there can be, creating competing implementations of particularly important API's; like databases.
Conclusion. Nothing is good enough, right now. I see two two paths ahead. Two ways that could actually end in the "100-year language".

The first path is one new language. This language will be based on all the best features of all current languages, plus a good amount of research output. I have a small list what this language would need to be successful as the next big one:
  • It needs to be multiparadigm. I'm not saying it can't choose one paradigm as the base, but it should be possible to program in it functionally, OOP, AOP, imperative. It should be possible to build a declarative library so you can do logic programming without leaving the language.
  • It should have static type inference where possible. It should also allow optional type hints. This is so important for creating great implementations. It can also increase readability in some cases.
  • It needs all the trappings of functional languages; closures, first-order functions and lambdas. This is essential, to avoid locking the language into an evolutionary corner.
  • It needs garbage collection. Possibly several competing implementations of GC's, running evolutionary algorithms to find out which one is best suited for long running processes of the program in question.
  • A JIT VM. It seems almost a given right now that Virtual Machines are a big win. They can also be made incredibly fast.
  • Another JIT VM.
  • A non-VM implementation. Several competing implementations for different purposes is important to allow competition and experimentation with new features of implementation.
  • Great integration with legacy languages (Java, Ruby (note, I'm counting on all Rubyists moving to this new language when it gets out, making Ruby legacy), Cobol). This is obvious. There are to many things lying around, bitrotting, that we will never get rid of.
  • The language and at least one production quality implementation needs to be totally open-source. No lock-in of the language should be possible.
  • Likewise, good company support is essential. A language needs money to be developed.
  • A centralized code/library repository. This is one of Java's biggest failings. Installing a new library in Java is painful. We need something like CPAN, ASDF, RubyGems.
  • The language needs great, small and very orthogonal libraries. The libraries included with the language needs to be great, since they have to be small but still pack all the most needed punch.
  • Concurrency must be a breeze. There should be facilities in the language itself for making this obvious. (Like Erlang or Gambit Scheme).
  • It should be natural to do meta-programming in it (in the manner of Ruby).
  • It should be natural to solve problems bottom-up, by implementing DSL's inside or outside the language.
  • The languages needs a powerful macro facility that isn't to hard to use.
  • Importantly, for the macro facility, the language needs to have a well-defined syntax tree of the simplest possible kind, but it also needs to have optional syntax.
So, that's what I deem necessary (but maybe not sufficient) for a really useful, good, long term programming language. When I read this list, it doesn't seem that probables that this language will show up any time soon, though. Actually, it seems kinda unrealistic.

So maybe the other way ahead is the right one? The other way I envision is that languages become easier and easier to create, and languages have their strength in different places. Along this path I envision the descendants of Ruby and Erlang exploiting what they're good at and eschewing everything else. But for this strategy to work, the first thing implemented in each language needs to be a seamless way to integrate to other languages. Maybe there will come an extremely good glue-language (not like Perl or Ruby, but a language that only will serve as glue between programming languages), and all languages will implement good support for that language. For example you could code a base Erlang concurrent framework, which uses G (the glue language) to implement some enterprise functionality in Java sandboxes, and some places where Ruby through G will implement a DSL, which have subparts where Ruby uses G to run Prolog knowledge engines.

If you had to choose among the two futures, I am frankly more inclined towards the one-language one. But the multi-language way seems much more probable. And since I'm trying to choose way now, I'm placing my bets on the second option. We are not ready to implement G yet, but I do think that as many p-language techs as possible should do their best to learn how languages can cooperate in different ways, to prepare this project.

19 kommentarer:

Isaac Gouy sa...

It needs to be multiparadigm.
When someone mentions multiparadigm I ask if they've tried Mozart/Oz (it's just a bad habit I've developed).

Anonym sa...

And you can also do some nifty things in Objective-C like higher order messaging stuff!

A well written intro by the way is Programming in Objective-C ( )

chromatic sa...

Hey, that's the feature list for Perl 6!

Julian Morrison sa...

I'd say rather, we were in the dark ages, and the church of C++ and Java ruled. Now we're in the "industrial revolution". Nobody knows what to believe. Every couple years turns the world on its head. There's no perennial standard.

As to the language of the future: it will be functional, and I'd consider the candidates Erlang, Mozart/Oz and Haskell. Each while presently having downsides, provides a really solid base to build upon. If I had to bet, I'd put money on Erlang.

Functional purity: it's the principle of "shared nothing" applied to code. It has exactly the advantages touted for shared-nothing, on a function by function level.

Live replaceable code: all real large apps re-implement this, usually badly. Stopping your app to change your code is a process measurable in dollars.

Concurrency: future CPUs will probably be parallel. Networks will be ubiquitous within and between machines.

Composable transactions: the only sensible way to structure actions that can't be allowed to partially complete.

Languages that won't make the cut: C++ and Java (OO has hit its ceiling), Ruby (probably the last gasp of pure OO), Lisp (too low level, not free of side effects, and macros are a step outside the language that breaks "equational reasoning" - the language of the future will do the same job with higher-order functions and casual parsing).

Anonym sa...


I don't think Erlang has a chance of getting any kind of large support in the future because honestly it is just backwards feeling in many places that are extremely important, like strings. It also has pretty ugly syntax.

For these reasons, I don't think Erlang will be the future. Haskell probably won't because it is *so* far out there from traditional languages.

And while I don't believe Ruby will be the language of the future either, the way it implements OO I do believe will be carried on. It's OO system is rather forgiving and doesn't force you to write a lot of framework to do simple things (ala Java).

I've been thinking about this topic lately, and I do think that the language of the future will have Pythonic/Ruby syntax, it will have type-inference ala OCaml, and will have concurrency like Erlang. If you could get all those features in one language, watch out.

Julian Morrison sa...

Anonymous @ 1:15:00, you should draw a distinction between essence and surface. Erlang can have its syntax cleaned. It can be sped up. It can get a port of Haskell's fast packed strings. Contrast: Ruby can never be purely functional.

Gour sa...

I'd say Haskell is pretty close except the fact that as it is recently said on one mailing list:

"Haskel (like Lisp) is well designed (on a strong mathematical foundation) to be compiled into efficient machine code, as opposed to a well designed interpreted language like Lua, or badly designed interpreted language like JavaScript.

But Haskel is about as far away from a "consumer programming language" as you can get -- it's quite weird by most people's standards, and has been known to cause people's heads explode.
In the ideal world, more people would have less explosive heads, but unfortunately Haskel is just too powerful and esoteric for most people to handle. (Unfortunately, so many people use shitty languages like PHP, so their programs explode instead of their heads.)
In the hands of a great programmer (like James Clark), it's astounding what can be done in a few lines of Haskel, and it's interesting to compare the amount of Java code it takes the same programmer to implement the same algorithm..."


Mark Haniford sa...

As to the language of the future: it will be functional, and I'd consider the candidates Erlang, Mozart/Oz and Haskell. Each while presently having downsides, provides a really solid base to build upon. If I had to bet, I'd put money on Erlang.

Julian, that's not the way it works in the industry though. If those were the uber-holy grail, then they would already be the uber-holy grail.

What he's talking about doesn't exist yet, and it won't ever exist. That doesn't mean you can't strive for it, but we're talking money.

Anonym sa...

Objective-C, the (only?) language that ships with out-of-the-box Unicode support and has the best GUI dev kit (aka Cocoa) available today. (But, ok, languages like OCAML have a workaround library to do the Unicode job).

And for many many bugs developers are to blame, not the language!!!

Cynos sa...


Tomasz Wegrzanowski sa...

You may be interested in my old post on the same subject.

Anonym sa...

Have you looked at Io?

It doesn't perfectly meet all of your criteria, of course, but there's lots of interesting ideas going into Io.
(Actor-style concurrency, for example)

Anonym sa...

Aren't OOP, AOP and imperative programming the same thing?

Anonym sa...

Nemerle is close.

taw sa...

Now that the anonymous poster mentioned it, you pretty much described Nemerle.

It is multiparadigm (at least functional + object-oriented + imperative + macros), has static type inference without sacrificing runtime type information. It has closures, first-order functions, lambdas, REPL, GC, two mature JIT VMs (Microsoft's and Mono). There aren't any complete native compilers right now, but there are many such projects in progress.

Integration with other languages is quite decent, at least with those that have .NET ports or interfaces. It's not perfect, but it's as good as any other language. It is open source (BSD license), and so it one of VMs. It is supported by some Microsoft grants, but that's not exactly "good company support".

As far as I know it doesn't have centralized code repository. It can reuse all .NET libraries, but I'm not sure whether that's what you want. It uses a lot of macros and metaprogramming and it seems people have been coding useful DSLs in it, and want to make concurrency easy by macros. The macros aren't exactly Lisp, but they're pretty decent for a language with Java-like syntax.

That's very close to what you described. Now if you or someone else ported RubyGems to Nemerle, wrote a few cool programs in it, and started marketing campaign for one of them, DHH-style, it could really become something ;-)

Skandar sa...

I suggest check out Scala programming language its very close of what you would like

Its a multiparadigm language supporting imperative, OOP, functional and concurrent with a fast vm, type-safe, partial type-inference, statically typed and syntax is very similar to java. It also have an eclipse plugin you can use as an IDE.

Nico sa...

I'm with you all the way on this. I would only add this: use an Algol-family syntax, but have an alternate S-expression syntax (in part for macro definition, but there's complications in extending the algol-syntax with macros), borrow from Icon (failure, backtracking) but by default limit backtracking so no resumptions happen (this would meant that like Perl6 you could mix backtracking and non-backtracking code, but without two different syntaxes), *don't* make too much use of non-alphanumeric characters as with IDEs and identifier completion it should be possible to aim for slightly more verbose-looking programs for maintainability (that is, expressiveness is great, but someone new to the language should stand some chance of understanding what's going on).

J sa...

I was bent on creating a new language until I found Io.

It has the power of lisp, the beauty and elegance of smalltalk, transparent futures...

I still find ruby a more practical language at this point, but I yearn for the day that I feel comfortable writing production code in Io.

Anonym sa...

Did you have had a look at Scala ?

Mel (