Open source project breakdown.

I realised today that I actually have a fairly large number of open source projects published online (all on google code. Another thing I realised is that I should fix that).

I also realised that some of these are totally defunct.

I thought this would be a good time to do a quick breakdown of them, explaining what's there, what they do and what their current status is.

JTypebuilder

Code generator for creating immutable data structures in Java. The idea was to define simple datatypes with a record notation and get an immutable class from them with correct equality, hash code and toString implementations, getters for the properties and a builder class for generating instances.

Status: Very very dead. It was at best a weak idea, and I have no interest in pursuing it. Use Scala's case classes instead.

Generators4j

A brief foray into functional programming in Java. Lazy generators with functions like map, filter, etc.

Status: So dead. I didn't get very far before concluding that trying to do this in Java was unusably awful.

Lazy Strings

Experiments in efficient representation of String types, with the aim being to provide a drop in replacement for java.lang.String with a different set of performance characteristics. Started in Java, moved to Scala.

Status: Just resting its eyes. I'm not doing much with this at the moment, but I occasionally peek at it and will probably factor out some of the ideas and turn it into a more tightly focused library.

Ranged Types

Very small Scala library for statically checked numeric ranges.

Status: Awaiting a use case. I occasionally think about picking it up again, but then I wonder why. It's a fun idea, but I don't actually have anything I need to use it for and as far as I can tell neither does anyone else.

SBinary

A small library for binary serialization and deserialization of Scala data types, based on Haskell's Data.Binary.

Status: Very much alive. I've just released version 0.1 RC1, am using it as a dependency in other things and am continuing to tinker with it to improve its usability.

Prefer Scala

A wrapper around the Java preferences API designed to be nicer to use from within Scala and support a wider variety of preferences in a typesafe way. Uses SBinary to serialize Scala types to and from the preference backing store. It's been factored out of the code for Hector's Reminder Service.

Status: Fledgling. I've only just released it. It's very small, and I intend it to remain so, so I expect to push it towards a 1.0 fairly quickly and then have it enter maintenance mode where future updates are just to fix bugs and bring it into line with the latest versions of its dependencies.

Hector's Reminder Service

Unlike the other ones, this one is an application. It's a small cross platform status bar application based on QT which gives you reminder messages on a semi-regular basis. Designed to be unobtrusive and simple and intended for the occasional casual reminder rather than of specific events. Uses "Prefer Scala" for persisting of state between application runs.

Status: Again, quite recent. I have a semi-official version released which works and more or less does what I want. I'm intending to polish that, add a very small number of new features (currently planned are a more expressive way of specifying message intervals, the ability to temporarily suppress a message group and possibly a simple API for other programs to interact with him) and then declare it to be feature complete. Once it's reached that point it will enter a similar state of "Updates are only to fix bugs and match new dependency versions".

Hector's Reminder Service: QT Jambi and Scala

Well, I spent a lot of today putting together the application I mentioned in my recent rant.

The program is called "Hector's Reminder Service". Basically it's a taskbar reminder app. You specify a random lists of messages and their approximate frequency. It gives a little notification message (not a popup window!) on the task bar showing one of those messages about that often. You can configure as many different groups of messages as you like and they'll be scheduled independently.

The code is available here. It's GPLed, mostly because it depends on QT and I couldn't be bothered to figure out the ramifications. Anything I consider reusable will be factored out into a library and released under a more moderate license. I have a few more things to sort out with it (mainly packaging) and will then release a version 0.1 of it.

Currently there's no packaging system set up. If you want to build this you'll need QT Jambi installed - both for the user interface file compiler and for the native libraries. It's currently untested on anything except windows, but now that I've given up on Swing and switched to QT I expect it should by and large work on OSX or any X-windows setup with a compliant toolbar. No doubt there will be problems, but they should be surmountable. Give me a shout if you do want to build it and discover it doesn't work on your platform. I'll do my best to help.

Edit: Actually, unless you're feeling brave you probably don't want to build this. It depends on having Scala 2.7 and jerbil installed as well as the QT Jambi libraries. You can download a prebuilt version from http://hectorreminder.googlecode.com/files/hector.zip , but you'll still need the QT Jambi libraries installed.

Edit 2: I can confirm that Hector does work properly under linux. You need to replace the qtjambi.jar in the lib directory with the one from your jambi install (turns out that's windows specific. Oops). Other than that he works perfectly.

Wait, you believed them when they said "Write once, run anywhere"? That's so cute.

So, I'm writing a tiny little application that sits in your status bar and pops up occasional reminder messages in the corner of your desktop. It uses the Java 6 desktop integration stuff. It's not very exciting - just fun and moderately useful.

I originally wrote this for Victoria with a set of hardcoded messages, so it only needed to run on windows. It worked really well, so I thought I'd turn it into a proper application - it should only be a few hours of coding to do so.

Especially, I thought, as it should run nicely cross platform.

Ha ha. Ha.

First off, Macs are ruled out - no functioning Java 6 yet. I took a brief look at using jdic instead, but it turns out that doesn't support macs either. Argh. So, we're stuck with windows and linux. Ho hum.

Yeah, linux? Not so much.

First off, Swing *never* works properly under linux. As far as writing cross platform GUIs, "Write once, run anywhere" is a blatant and utter lie. If you're using Gnome or KDE with their standard window managers, it will just about limp by. If you're using anything else, good luck.

Anyway, I use xmonad. However I use xmonad with gnome (at least on my laptop), and this is just a status bar feature, so given that I still have the gnome status bar I was optimistic.

Nope. Looks like Swing checks the window manager name, not the availability of the status bar. It's really great the way the Swing/AWT developers actually understand the environment they're developing for, isn't it? On a related note, the resulting toolbar icon looks *really bad* even when I run it under normal gnome. That's probably just a scaling and transparency issue though. I imagine I can sort it out if I try hard enough.

I have to say, despite this the desktop integration stuff is moderately nice. It's a great way of writing once and running anywhere that has windows and the latest version of Java installed.

Sigh.

Anyone willing to put a word in for Groovy?

Groovy seems to come up in conversation a reasonable amount. People appear to be doing some interesting things with it. But every time someone tries to advocate the language it leaves me completely cold. Basically all the advocacy seems to come down to:

• It has "closures"
• It looks like Java...
• ...but it's dynamically typed.

Yay? It has a few features that seem a bit more interesting. It has named (and default?) parameters, which is nice (not exciting, just nice). What little I've seen of its metaclass stuff gives me a simultaneous "yikes" and "ooh, that's kinda neat" reaction. This isn't really intended to be a bash at the language. I just don't know much about it, and none of the information I've seen seems terribly compelling. I'd like to hear some stuff about why it's actually a nice language to use. I probably still won't use it - I have enough languages on my plate as it is - but I'd like to be a little less ignorant about it.

Tell us why your language sucks

Let's play a game. Take your favourite languages and tell us what you hate about them. Like I've said before - all languages suck. Don't pretend yours is perfect. If there aren't things about it which annoy you you're not trying hard enough.

Note: This should not be about bashing languages you don't use or like, or even languages which you use and don't like. Everyone does that already. I don't want to hear yet another rant about why Ruby sucks from a Java programmer, why Java sucks from a Factor programmer, why Lisp sucks from a Javascript programmer, etc. I'm not even interested in hearing about people who are forced to use Java|C|C++|Javascript|Whatever and hate it. Tired, been done to death. It specifically has to be a language you use at least semi-regularly and like. Feel free to post here, to your own blogs, to the inevitable reddit thread, etc. But wherever you post it, have fun. Go on, rant. Get it off your chest. You'll feel better for it. :-)

I've already started with Scala. Now I'll do Haskell. Hopefully someone more experienced with the language will follow up, as I'll freely admit it's not my best language.

So, things I hate about Haskell:

Let's start with the obvious. Monad tutorials. No, not monads. Specifically the tutorials. They're endless, overblown and dear god are they tedious. Further, I've never seen any convincing evidence that they actually help. Read the class definition, write some code, get over the scary name.

Now we've over the one sociological issue, let's get to some actual language ones.

Modularity. Haskell's module system is about the minimum possible you can get away with and still claim to have a module system. Anything less and you basically have C style header files. Now, this isn't a huge problem. To a certain degree type classes mitigate the need for anything functorial. But there are examples where it's an issue. Take ByteStrings for example. Lazy and strict ByteStrings have essentially the same module signatures, but there's no way to write code which is agnostic as to the type of ByteString it uses. They don't live in a type class because it was considered that the type class signature would be too long. This is probably fair, but life would be a lot nicer if you could write functors over the ByteString modules.

In general, Haskell has very poor namespacing. If two modules define something of the same name you have to redefine one of them or refer to one of them qualified. It's not a huge issue, especially as you *can* rename and Haskell modules are anyway not as long as Java packages. It's not like you have to refer to org.haskell.data.collections.map.mypetcat.lookup. M.lookup isn't too onerous. This mainly causes problems with symbolic operators, because things like Foo.++ look a lot worse as infix operators than Foo.bar does. I don't know of a good solution to this problem. Object oriented languages provide a solution to it by basically making the type of the first argument responsible for namespacing. I don't find this especially satisfactory, but it does work.

On a similar note, Haskell type classes basically live in global scope. There are good reasons for this, and it avoids you having to write a whole lot of painful sharing constraints (The example that is thrown at me every time I complain about this is what do you do if you want to unify two Data.Sets, each of which uses a different instance of the Ord type class)? It can be a real pain though.

There's another issue related to type classes. Let's pick an example. Data.List defines both sortBy and sort. The difference is purely that sort uses the standard ordering on the Ord type class while sortBy uses a provided ordering. And you need to define different functions every time you want to be able to use either a type class or something user configurable. Because you can't redefine type classes locally or pass them as first class instances there's basically no way around this. You can normally avoid the issue with selective use of newtype and careful choice of your types, but it's definitely there.

Which brings me to another point - type classes are not first class. The Scala encoding of type classes is clunky in many ways, but a very nice feature is that the things you're using as type classes are real first class members of the language, and you can bring to bear all the usual tools you have for manipulating stuff.

10,000 compiler extensions. GHC introduces so many compiler extensions, and everyone goes wild about them. This is understandable. Some of them are really useful. Some of them it's amazing to manage living without (No multiparameter type classes in H98? Yeargh. Getitaway!!). I really like most of these I've used, but it sometimes make it feel like you need to employ deep and profound type system voodoo to get anything done. It would be very nice to get these unified into something sane and consistent, but Haskell' seems to have stalled. Tuples and records. There's basically no good generic handling for either of these. Each of the different sizes of tuples is a totally different type (with no common type classes for abstracting over them), records are just a thin wrapper over normal datatypes with no extensibility or namespacing. Your record accessors will clash just as much as any other function name.

Something which has bitten me in the past is sharing code between monadic and non-monadic contexts. You essentially need to write pure and monadic versions of a number of functions. It would be nice if generic monadic code could automagically be specialised to the identity monad in a way that didn't uglify pure code using it, or pure code lifted to monadic versions (this is harder I think). This doesn't come up too much, but it means that there's often functions like foo and fooM for pure and monadic versions.

The prelude and standard type classes are a bit painful sometimes. Things which you'd expect to be overloaded into type classes aren't (Data.Monoid defines a function mappend for example, which in the List instance is ++. Why isn't ++ on Data.Monoid?) and sometimes the type classes which are there are poorly thought out (Num shouldn't extend Eq, and it would be nice if + was factored out in order to provide better support for things like vector spaces). I believe there is some work on alternate numeric preludes.

Plus lots of other little things which don't spring to mind at the moment.

Also, one final disclaimer. Please please please don't take this as a "Haskell sucks, don't use it!!!" post. If you do I'll... I don't know, give you a really devastating hurt puppy look or something.

I'd also appreciate it if you don't use this post to start a language war. Remember - you're only allowed to say bad things about languages you actually like. Otherwise you're cheating. :-)

Easy binary serialization of Scala types

I'm going to be prototyping some stuff in Scala at work in the coming week, and wanted a nice way of marshalling things to/from files and across the network. The BytePickle stuff in scala.io does nothing for me, and Java serialization gives me the screaming heebie jeebies, so this prompted me to get off my ass and do something I've been meaning to do for a while - port something akin to Haskell's Data.Binary to Scala using the encoding of type classes I've previously discussed. Well, it's done - it didn't take very long at all. The port is *extremely* loose - in particular I've just written it for imperative use rather than define custom monads for reading and writing in a pure manner (sorry). The project is hosted on google code at http://code.google.com/p/sbinary/

At its heart it's extremely simple:

trait Binary[T]{
  /**
   * Read a T from the DataInputStream, reading no more data than is neccessary.
   */
  def reads(stream : DataInputStream) :T;

  /**
   * Write a T to the DataOutputStream.
   */
  def writes(t : T)(stream : DataOutputStream) : Unit; 
}

object Operations{
  /**
   * Use an implicit Binary[T] to read type T from the DataInputStream.
   */ 
  def read[T](stream : DataInputStream)(implicit bin : Binary[T]) : T = bin.reads(stream);

  /**
   * Use an implicit Binary[T] to write type T to the DataOutputStream.
   */
  def write[T](t : T)(stream : DataOutputStream)(implicit bin : Binary[T]) : Unit =  
    bin.writes(t)(stream);
}

Err. That's it. Did you want more? :-)

There's more to it than that of course, but most of the rest of the code I've written for this is just helper methods, instances and scalacheck tests.

Out of the box this will serialise tuples of any size (that Scala supports. i.e. of 22 elements or fewer), lists, arrays, immutable maps, options, Strings, all the AnyVal types and any combination thereof. Looking at the code should give you an idea of how to define your own Binary instances.

Using it is very simple. It works by knowing the type of thing you want to read or write from the stream and selecting the appropriate logic based on that type (but, unlike Java serialization, if you give it the wrong type it will attempt to read it as that type anyway and probably do crazy things - this is very explicitly using the type to define a compact encoding and doesn't select it based on dynamic information from the stream). e.g.

  import binary.Operations._;
  import binary.Instances._;
  val foo = read[(Int, Option[String], List[Int])](inStream);
  write(foo._2)(outStream);

The read and write methods on Operations take care of selecting an appropriate implicit instance of Binary and combining them to do the right thing.

Note that binary serialization logic is kept entirely external to the class, so it's almost as easy to define for classes from external libraries as it is for your own.

I'm not doing an official release yet - I want to have a play around with this and see how usable it is. Once I have, I might change the API around to improve it. On the other hand, the code works now and does enough (within its very simple objectives) that it's probably useful. I've written a bunch of scalacheck tests for it and am reasonably confident it gets all the current binary instances right. If you want to use it for something, go right ahead! Report back to me and let me know how it goes.

Edit: By the way, this only works properly on 2.6.1 or higher. There were some problems with the implicit arguments implementation prior to then that prevent the instances from working correctly.

Learning Scala

Some questions for people who are learning / have learned Scala: What languages did you know beforehand, and how easy did you find learning Scala in comparison to these? Are there any languages which you found knowing particularly helpful when picking up Scala? An explanation follows: Scala seems to be a relatively hard language to learn for some people, not so much for others. Part of this is its complexity - it really does have a lot of little features - but I'm wondering if more of it might be its approach. It's a language with two major inspirations - object orientation (in the peculiar flavour of it Java practices) and statically typed functional programming, and I'm not sure how easy it is to understand the language unless you understand where it's coming from in this regard. In particular one thing we've observed in #scala from people learning the language is that if you know both Java and Haskell (I presume an ML would work as well?), learning Scala becomes significantly easier. I had almost no trouble picking it up, but I know both. Ricky Clarkson seems to be in a similar boat in terms of Haskell + Java having helped. I presume others are too. On the other hand, people with Java background but not much FP seem to have more trouble and people coming from a predominantly ruby or python background have a harder time yet. (I don't know what happens to people coming from a Haskell with no Java background. I'd expect a similar degree of confusion to the Java with no Haskell background). Some of this is probably in terms of material - a lot of Scala tutorials, etc. out there seem to assume you already know Java. This is probably largely accurate but seems like a mistake in the long-term to me. On the other hand, I'd be really uncomfortable teaching Scala as a first language, so what languages *should* they be learning to prepare the way? Anyone tried learning it on the basis of, say, Ruby + OCaml? So, what do we want people's path into Scala to be? Should we suggest they learn Java first if they don't want a bit of a rough start, or is there a better way?