Tuesday, March 23, 2010

Loosing the Language war

Everyday our log files increase, immense amount of data pours in, from server statistics, to the number of students in the application. The question of how to present this data is rapidly coming to a head.

To begin with we used Erlang, for no other reason than it was our server side language and the need to keep the language count under control is always a pressing issue on the dev team. Erlang is a heroic language it is the reason we have had 100 percent uptime on the product but it was not designed to be used as a parsing language, Why you ask?

Erlang is tail recursive passing a full copy of the parameter each time it is called, which in itself is fine and tail recursion is elegant in its own way, the problem arises with Erlang's representation of strings. Each character uses 8 bytes of memory. That's right -- 8 bytes, not 8 bits! Erlang stores each character as a 32-bit integer, with a 32-bit pointer for the next item in the list (remember, strings are lists of characters.). The reason for this is that Erlang can handle any Unicode character thrown at it, while noble in theory it was not valuable in practise.

So, with erlang out of the game there was a call out for a new language to take its place. Before my job at Monash I worked at a place called Editure and it was amongst other things a perl house, so instinctively I ran with perl, I knew it was basically unix calls with some nice wrapper around, it had a nice library support with CPAN. Sure its OO is tacked on with gaffa tape but I did not need OO for what I was doing, so perl seemed like the perfect choice.

But it still lost.

Perl did the computation at light speed, with the ability to spit the data out in HTML it made for easy automated reports but this was not enough, why? perl is loosely typed and as a result you generally have the slack to code fast and loose, which if fine when you need to hack together a report in an hour, but in six weeks time when you need to revisit your code and you have forgotten the method signature on your hand built datetime to gregorian seconds you will need to find the method instead of having the compiler yell at you for being a moron.

Perl is not a large scale maintainable language, I am sure perl 6 will change that, while it also washes my car for me but until that time perl gets too sprawling too quickly. I can understand entirely even only writing a fractional amount of perl code why Editure picked up its bags and moved into the JVM.


Also? timelocal, built in perl function has a epoch of 1900 where as other builtins have it as 1970, what the hell perl!

So, the race has seemingly been won by Scala, a functional language on the JVM written by Martin Odersky, this is where my knowledge of the language ends, but if it truly has won, I am sure I will be learning it soon.