Well, I've been meaning to set up a site of my own for ages. I finally got around to it. I can now be found at http://www.drmaciver.com/
This blog will be moving to the "programming" category there. I'll update the various aggregator sites that have it on to point there instead.
I came up with a neat idea for changing the syntax for call by name parameters recently (it turned out that it's actually a reversion to an older syntax for it! The new one was there to resolve some problems, but I like the old syntax better so would rather resolve those problems directly). In the discussion of this some problems were pointed out and the feature list sortof spiralled out of control and collided head on with a previous proposal by Andrew Foggin. Here's a summary of the current state of the proposal.
This is kinda a "Well, duh" moment. It's obvious in retrospect, but I completely failed to spot it up front, so I thought I'd share.
I noticed that SBinary performance really really sucked. We're using it at work for saving application state, and reading and writing a file of only 900kb took about two seconds! This was bad.
Some quick performance testing suggested that this was almost entirely IO bound. Reading and writing the corresponding amount of data from a byte array took fairly little time - only about 200ms for writing, 300ms for reading. So, what was I doing wrong?
After a few seconds of head scratching I realised the problem. You see, RandomAccessFile implements the DataInput and DataOutput interfaces. This is useful for small things, but for doing non-trivial binary input and output? Not so much.
The problem is that the reads and writes for these implementations are totally unbuffered. This should have been obvious, but for some reason didn't occur to me. Oops. I'm now buffering reads and writes explicitly (currently in a pretty stupid way, but oh well). It's a lot faster now.
I'm thinking about changing the scope of some of the code for SBinary.
Specifically, you remember that part where I said "SBinary is only for serializing objects and manipulating binary data, and it's going to remain super minimal and specialised and this will never ever change!!"? I'm thinking of changing that. :-)
The reason for this change of heart is that I'm realising how incredibly generic the constructions you put together for SBinary are. You're basically creating a walker for deconstructing and reconstructing your entire object graph. That's pretty damn powerful. In particular I was thinking about how to modify formats to permit sharing (another post on that will be forthcoming) and suddenly thought "haaang on a minute. I've written this code before". It looks suspiciously identical to some Java code I wrote a while back for generic cloning of object graphs*. A simple rebinding of the backend to use a queue of objects rather than input and output streams would give a pretty efficient deep clone mechanism. I've also been thinking of creating a JCR backend which mostly works the same as the binary data (indeed, most data would probably be stored as binary blobs in the JCR), but allows for references to other nodes (and would use this for data sharing).
At the very least, this will result in ditching the explicit dependency on java.io. It will still be used extensively in the back end, but this is only visible in the API for the parts that actually need to interact with it. (most likely approach - have an Input and Output opaque type to replace DataInput and DataOutput. These will just be wrappers around the java.io types, but this won't be visible at first)
If I do do something like this, it would still be with making binary data the priority, and there would definitely be a specialised binary frontend which should be just as convenient as the current API. If it ever looks like feature creep is threatening to destroy that I'll separate out projects and/or cut out the idea entirely.
* In the unlikely event that anyone who worked on that project actually reads this blog, they will probably shudder in horror at the mention of that code. It was very fragile with regards to changes in the rest of the code. But that wasn't actually an issue with the cloning - it was an issue with the post-clone processing. The graph was of database mapped objects and it needed to be partially linearised in order to insert it back into the database due to constraint issues, and this never really worked right.
SBinary and Scalacheck are part of a small set of libraries that make extensive use of implicit arguments in a style very reminiscient of Haskell type classes. I'm hoping this style of programming will get more common in Scala - it's a really useful technique and, in my completely unbiased opinion, both SBinary and Scalacheck are fantastic and you absolutely should use them. :-) But in order to do so you need to really understand how implicit arguments in Scala work.
This post is actually for work, as we're using Scala there and this is a subject which has been confusing one of my colleagues.
As a starting point, in Scala you can declare a method to have multiple argument lists. This isn't a fantastically useful feature, but here's how it works:
scala> def foo(x : Int)(y : Int)
| = x + y
foo: (Int)(Int)Int
scala> foo(1)(2);
res1: Int = 3
scala> foo(1, 2);
:6: error: wrong number of arguments for method foo: (Int)(Int)Int
foo(1, 2);
^
scala> foo(1)
:6: error: missing arguments for method foo in object $iw;
follow this method with `_' if you want to treat it as a partially applied funct
ion
foo(1)
^
i.e. "exactly the same as a single method parameter list but you have to use a different syntax for calling it". Hurray.
This has one advantage though. You can declare the last parameter list of a function to be implicit. The syntax for this works as follows:
scala> def speakImplicitly (implicit greeting : String) = println(greeting)
speakImplicitly: (implicit String)Unit
scala> speakImplicitly("Goodbye world")
Goodbye world
scala> speakImplicitly
:6: error: no implicit argument matching parameter type String was foud.
scala> implicit val hello = "Hello world"
hello: java.lang.String = Hello world
scala> speakImplicitly
Hello world
So, we can call this as normal but, additionally, we can leave out the implicit argument list and the compiler will look for a value in the enclosing scope which has been marked as implicit. If we try to do that and there is no such value in scope then the compiler will complain.
Implicits are totally typesafe, and are selected based on the static type of the arguments. Here are some examples to show how things work.
scala> def speakImplicitly (implicit greeting : String) = println(greeting) speakImplicitly: (implicit String)Unit scala> implicit val aUnit = (); aUnit: Unit = () scala> speakImplicitly:7: error: no implicit argument matching parameter type String was found.
Only an implicit of type String will be selected for an implicit argument of type String.
scala> def speakImplicitly (implicit greeting : String) = println(greeting) speakImplicitly: (implicit String)Unit scala> implicit val hello : Any = "Hello world" hello: Any = Hello world scala> speakImplicitly:7: error: no implicit argument matching parameter type String was found.
Implicit selection happens on the *static* type of variables. It's no use having something of the right dynamic type if the variable isn't typed accordingly.
scala> def speakImplicitly (implicit greeting : String) = println(greeting) speakImplicitly: (implicit String)Unit scala> implicit val foo = "foo"; foo: java.lang.String = foo scala> implicit val bar = "bar"; bar: java.lang.String = bar scala> speakImplicitly:9: error: ambiguous implicit values: both value bar in object $iw of type => java.lang.String and value foo in object $iw of type => java.lang.String match expected type String
If there are multiple implicit arguments of the same type, it will fail as it has no way of choosing between them. But...
scala> def sayThings (implicit args : List[Any]) = args.foreach(println(_))
sayThings: (implicit List[Any])Unit
scala> implicit val nothingNiceToSay : List[Any] = Nil
nothingNiceToSay: List[Any] = List()
scala> sayThings
scala> implicit val hellos : List[String] = List("Hello world");
hellos: List[String] = List(Hello world)
scala> sayThings
Hello world
If you have an implicit argument of a subtype, it will also match as an implicit argument of this type. Moreover, if you have two implicit arguments which match and one is a subtype of the other, the more specific type will match.
scala> def implicitly[T](implicit t : T) = t implicitly: [T](implicit T)T scala> implicit val foo = "foo" foo: java.lang.String = foo scala> implicit val aUnit = () aUnit: Unit = () scala> implicitly[String] res2: String = foo scala> implicitly[Unit]
Type parameters can quite happily take part in the implicits mechanism.
So, we know how to use defined implicit arguments now. But how can we define them? We've seen one way:
implicit val foo = "foo"; scala> implicitly[String] res2: String = foo
If this was all we could do then it wouldn't be that powerful a feature. A nice to have, but ultimately not *that* exciting. Fortunately there are a few more things we can do. For starters, Scala has the uniform access principle, so any (wait, no. That would be too general. We can't have features without special cases. Sigh. Ok, let's say most) things you can do with a val you can do with a def
implicit def foo = "foo" scala> implicitly[String] res2: String = foo
This def will be invoked each time we want the implicit. Here's an example to demonstrate this
scala> implicit def aUnit : Unit = println("Hello world")
aUnit: Unit
scala> implicitly[Unit]
Hello world
scala> implicitly[Unit]
Hello world
scala> implicitly[Unit]
Hello world
In general, implicit defs shouldn't have side effects. It can lead to some really counterintuitive behaviour. This is just for demonstration purposes.
Now, the ability to use defs opens up a bunch of possibilities. For example, they can have type parameters:
scala> implicit def emptyList[T] : List[T] = Nil; emptyList: [T]List[T] scala> implicitly[List[String]] res9: List[String] = List(Hello world) // Oops, we still had an implicit List[String] left over from an earlier example. Note how that was used in preference to the parameterized version. Let's try again. scala> implicitly[List[Int]] res10: List[Int] = List()
Moreover, implicit defs used in this way can themselves have implicit parameters. For example:
scala> case class Foo[T](t : T); defined class Foo scala> implicit val hello = "Hello" hello: java.lang.String = Hello scala> implicit def foo[T](implicit t : T) = Foo[T](t) foo: [T](T)Foo[T] scala> implicitly[Foo[String]] res3: Foo[String] = Foo(Hello)
(Note: I originally tried to write this example with Option. It turns out there's a bug with how covariant types are handled which made it not work)
The basic idea is that anything marked as implicit which you could write as a single identifier (possibly with a type signature to handhold the type inference system) is valid to be passed as an implicit argument.
This should provide enough to get you started. Your next step should probably be to check out the documentation for Scalacheck and SBinary (the latter of which is... less than stellar at the moment. I'll fix that, I promise. :-)). If you're looking for some slightly more hardcore reading, Generics of a Higher Kind has some interesting examples. Other than that, the best thing to do is play with some code.
With 2.7 of Scala on the way, people are being exposed to Java wildcards more and more, which translate to Scala existential types. Unfortunately no one seems to understand these (including me at first!) and had previously let them go largely ignored, and now everyone is getting confused.
Here's a brief introduction.
scala> def foo(x : Array[Any]) = println(x.length);
foo: (Array[Any])Unit
scala> foo(Array[String]("foo", "bar", "baz"))
:6: error: type mismatch;
found : Array[String]
required: Array[Any]
foo(Array[String]("foo", "bar", "baz"))
This doesn't compile, because an Array[String] is not an Array[Any]. You can put 1 into an Array[Any], but not into an Array[String]. Nonetheless, it's completely typesafe - we've only used methods in foo which would work for any Array[T]. How do we fix this?
Here's one way:
scala> def foo[T](x : Array[T]) = println(x.length)
foo: [T](Array[T])Unit
scala>> foo(Array[String]("foo", "bar", "baz"))
3
We've parameterised the method by T in order to make it accept any T. But now we have a superfluous type parameter on our method. This may not seem like a big deal, and it's usually not, but it can add up if you're not careful (and can be particularly annoying when for some reason the type checker is no longer able to infer a single one of your type parameters and you have to supply all of them). It's also not really what we mean - we mean "I want an Array, and I don't care what type of things it contains"
This is exactly what existential types are for.
scala> def foo(x : Array[T] forSome { type T}) = println(x.length)
foo: (Array[T] forSome { type T })Unit
scala> foo(Array[String]("foo", "bar", "baz"))
3
This is quite verbose, I know. There's a shorthand, Array[_], but this has some unfortunate unintuitive behaviour. I'll explain this later.
Sometimes we want to act on a more specific type, but don't care exactly what type it is. For example suppose we wanted this to work on any CharSequence and do something more complicated to each argument. e.g.
scala> def foo(x : Array[T] forSome { type T <: CharSequence}) = x.foreach(y => println(y.length))
foo: (Array[T] forSome { type T <: java.lang.CharSequence })Unit
scala> foo(Array[String]("foo", "bar", "baz"))
3
3
3
The type arguments in an existential type can have upper and lower bounds like normal type declarations. They can't have view bounds, presumably due to technical limitations.
So we've seen how these are used, and that was relatively nonconfusing (I hope!). Let's pin down exactly what these mean.
Suppose we have a type constructor M. In other words, for a type T, M[T] is a type, but M is *not* itself a type. M could be List, Array, Class, etc. M[T] forSome { type T; } is the type of all things for which there is some T such that they are of type M[T]. So an Array[String] is of this type, because we can choose T = String, as is an Array[Int], etc.
If we add bounds, all we do is restrict the range that T can lie in. An Array[String] is not an Array[T] forSome { type T <: Number; } because the only possible choice of T (String) is not a subtype of Number
Now, all you need to do in order to understand a given existential type declaration is to apply this rule rigorously. But this can be hard, especially because precedence matters in subtle ways! I'll walk you through some examples.
T forSome { type T; }
This is the type of all things for which there exists some T such they are T. Wha?
Think about it for a second. It's the type of all things for which there exists a type such that they are of that type. i.e. it's a long winded way of writing the type of all things, Any. This is important, and it often trips people up when they write subtly the wrong thing. Considering the following two types:
Array[T] forSome { type T; }
Array[T forSome { type T; }]
They look almost identical, but they're in fact very different. The first is the type of all arrays, whatever their type parameter. The second is Array[Any]
Let's take another example, because this is the one which seems to come up a lot. We have a Map. We want it to map classes to something. Let's say Strings. What type do we use?
Here. Pick one:
Map[Class[T forSome { type T}], String]
Map[Class[T] forSome { type T}, String]
Map[Class[T], String] forSome { type T}
Which did you pick?
The correct answer is "Map[Class[T] forSome { type T}, String]", or to save you searching for ]s, "the middle one".
Why? Well, the first one is a Map[Class[Any], String]. Class is invariant in its type parameters. So the only Class[Any] is in fact classOf[Any] (this is basically the same as Object.class in Java). So that's not very useful. Similarily, the third one is the supertype of all map types such that there is some T such that they are a Map[Class[T], String]. So again, we've got some fixed class type for keys in the map - it's just that this time we don't know what type it is. The middle one however has keys of type Class[T] forSome { type T }. That is, its keys are classes which are allowed to have any value they want for their type parameter. So this is what we actually wanted.
Now, the final confusing point. As I mentioned, we have this shorthand use of _ for wildcards. So we can write Array[_] to mean Array[T] forSome { type T}. That's nice. So what happens if we try to use this in the above and write Map[Class[_], String]? It turns out, we get "Map[Class[T], String] forSome { type T}". The wildcards always bind to the outermost level of the type expression. This is, unfortunately, almost never what you want in cases where it affects anything. There's been some discussion about changing it. I don't know if it will go anywhere.
Anyway, hopefully this has made some sense of things for you. It's a really confusing subject when you first encounter it, but once you've got it straight in your head it's not too bad. It would be nice if it could be simpler, but I'm not really sure what the best way to do this actually would be.
I realised today that I actually have a fairly large number of open source projects published online (all on google code. Another thing I realised is that I should fix that).
I also realised that some of these are totally defunct.
I thought this would be a good time to do a quick breakdown of them, explaining what's there, what they do and what their current status is.
Code generator for creating immutable data structures in Java. The idea was to define simple datatypes with a record notation and get an immutable class from them with correct equality, hash code and toString implementations, getters for the properties and a builder class for generating instances.
Status: Very very dead. It was at best a weak idea, and I have no interest in pursuing it. Use Scala's case classes instead.
A brief foray into functional programming in Java. Lazy generators with functions like map, filter, etc.
Status: So dead. I didn't get very far before concluding that trying to do this in Java was unusably awful.
Experiments in efficient representation of String types, with the aim being to provide a drop in replacement for java.lang.String with a different set of performance characteristics. Started in Java, moved to Scala.
Status: Just resting its eyes. I'm not doing much with this at the moment, but I occasionally peek at it and will probably factor out some of the ideas and turn it into a more tightly focused library.
Very small Scala library for statically checked numeric ranges.
Status: Awaiting a use case. I occasionally think about picking it up again, but then I wonder why. It's a fun idea, but I don't actually have anything I need to use it for and as far as I can tell neither does anyone else.
A small library for binary serialization and deserialization of Scala data types, based on Haskell's Data.Binary.
Status: Very much alive. I've just released version 0.1 RC1, am using it as a dependency in other things and am continuing to tinker with it to improve its usability.
A wrapper around the Java preferences API designed to be nicer to use from within Scala and support a wider variety of preferences in a typesafe way. Uses SBinary to serialize Scala types to and from the preference backing store. It's been factored out of the code for Hector's Reminder Service.
Status: Fledgling. I've only just released it. It's very small, and I intend it to remain so, so I expect to push it towards a 1.0 fairly quickly and then have it enter maintenance mode where future updates are just to fix bugs and bring it into line with the latest versions of its dependencies.
Unlike the other ones, this one is an application. It's a small cross platform status bar application based on QT which gives you reminder messages on a semi-regular basis. Designed to be unobtrusive and simple and intended for the occasional casual reminder rather than of specific events. Uses "Prefer Scala" for persisting of state between application runs.
Status: Again, quite recent. I have a semi-official version released which works and more or less does what I want. I'm intending to polish that, add a very small number of new features (currently planned are a more expressive way of specifying message intervals, the ability to temporarily suppress a message group and possibly a simple API for other programs to interact with him) and then declare it to be feature complete. Once it's reached that point it will enter a similar state of "Updates are only to fix bugs and match new dependency versions".
Well, I spent a lot of today putting together the application I mentioned in my recent rant.
The program is called "Hector's Reminder Service". Basically it's a taskbar reminder app. You specify a random lists of messages and their approximate frequency. It gives a little notification message (not a popup window!) on the task bar showing one of those messages about that often. You can configure as many different groups of messages as you like and they'll be scheduled independently.
The code is available here. It's GPLed, mostly because it depends on QT and I couldn't be bothered to figure out the ramifications. Anything I consider reusable will be factored out into a library and released under a more moderate license. I have a few more things to sort out with it (mainly packaging) and will then release a version 0.1 of it.
Currently there's no packaging system set up. If you want to build this you'll need QT Jambi installed - both for the user interface file compiler and for the native libraries. It's currently untested on anything except windows, but now that I've given up on Swing and switched to QT I expect it should by and large work on OSX or any X-windows setup with a compliant toolbar. No doubt there will be problems, but they should be surmountable. Give me a shout if you do want to build it and discover it doesn't work on your platform. I'll do my best to help.
Edit: Actually, unless you're feeling brave you probably don't want to build this. It depends on having Scala 2.7 and jerbil installed as well as the QT Jambi libraries. You can download a prebuilt version from http://hectorreminder.googlecode.com/files/hector.zip , but you'll still need the QT Jambi libraries installed.
Edit 2: I can confirm that Hector does work properly under linux. You need to replace the qtjambi.jar in the lib directory with the one from your jambi install (turns out that's windows specific. Oops). Other than that he works perfectly.
So, I'm writing a tiny little application that sits in your status bar and pops up occasional reminder messages in the corner of your desktop. It uses the Java 6 desktop integration stuff. It's not very exciting - just fun and moderately useful.
I originally wrote this for Victoria with a set of hardcoded messages, so it only needed to run on windows. It worked really well, so I thought I'd turn it into a proper application - it should only be a few hours of coding to do so.
Especially, I thought, as it should run nicely cross platform.
Ha ha. Ha.
First off, Macs are ruled out - no functioning Java 6 yet. I took a brief look at using jdic instead, but it turns out that doesn't support macs either. Argh. So, we're stuck with windows and linux. Ho hum.
Yeah, linux? Not so much.
First off, Swing *never* works properly under linux. As far as writing cross platform GUIs, "Write once, run anywhere" is a blatant and utter lie. If you're using Gnome or KDE with their standard window managers, it will just about limp by. If you're using anything else, good luck.
Anyway, I use xmonad. However I use xmonad with gnome (at least on my laptop), and this is just a status bar feature, so given that I still have the gnome status bar I was optimistic.
Nope. Looks like Swing checks the window manager name, not the availability of the status bar. It's really great the way the Swing/AWT developers actually understand the environment they're developing for, isn't it? On a related note, the resulting toolbar icon looks *really bad* even when I run it under normal gnome. That's probably just a scaling and transparency issue though. I imagine I can sort it out if I try hard enough.
I have to say, despite this the desktop integration stuff is moderately nice. It's a great way of writing once and running anywhere that has windows and the latest version of Java installed.
Sigh.
Let's play a game. Take your favourite languages and tell us what you hate about them. Like I've said before - all languages suck. Don't pretend yours is perfect. If there aren't things about it which annoy you you're not trying hard enough.
Note: This should not be about bashing languages you don't use or like, or even languages which you use and don't like. Everyone does that already. I don't want to hear yet another rant about why Ruby sucks from a Java programmer, why Java sucks from a Factor programmer, why Lisp sucks from a Javascript programmer, etc. I'm not even interested in hearing about people who are forced to use Java|C|C++|Javascript|Whatever and hate it. Tired, been done to death. It specifically has to be a language you use at least semi-regularly and like. Feel free to post here, to your own blogs, to the inevitable reddit thread, etc. But wherever you post it, have fun. Go on, rant. Get it off your chest. You'll feel better for it. :-)
I've already started with Scala. Now I'll do Haskell. Hopefully someone more experienced with the language will follow up, as I'll freely admit it's not my best language.
So, things I hate about Haskell:
Let's start with the obvious. Monad tutorials. No, not monads. Specifically the tutorials. They're endless, overblown and dear god are they tedious. Further, I've never seen any convincing evidence that they actually help. Read the class definition, write some code, get over the scary name.
Now we've over the one sociological issue, let's get to some actual language ones.
Modularity. Haskell's module system is about the minimum possible you can get away with and still claim to have a module system. Anything less and you basically have C style header files. Now, this isn't a huge problem. To a certain degree type classes mitigate the need for anything functorial. But there are examples where it's an issue. Take ByteStrings for example. Lazy and strict ByteStrings have essentially the same module signatures, but there's no way to write code which is agnostic as to the type of ByteString it uses. They don't live in a type class because it was considered that the type class signature would be too long. This is probably fair, but life would be a lot nicer if you could write functors over the ByteString modules.
In general, Haskell has very poor namespacing. If two modules define something of the same name you have to redefine one of them or refer to one of them qualified. It's not a huge issue, especially as you *can* rename and Haskell modules are anyway not as long as Java packages. It's not like you have to refer to org.haskell.data.collections.map.mypetcat.lookup. M.lookup isn't too onerous. This mainly causes problems with symbolic operators, because things like Foo.++ look a lot worse as infix operators than Foo.bar does. I don't know of a good solution to this problem. Object oriented languages provide a solution to it by basically making the type of the first argument responsible for namespacing. I don't find this especially satisfactory, but it does work.
On a similar note, Haskell type classes basically live in global scope. There are good reasons for this, and it avoids you having to write a whole lot of painful sharing constraints (The example that is thrown at me every time I complain about this is what do you do if you want to unify two Data.Sets, each of which uses a different instance of the Ord type class)? It can be a real pain though.
There's another issue related to type classes. Let's pick an example. Data.List defines both sortBy and sort. The difference is purely that sort uses the standard ordering on the Ord type class while sortBy uses a provided ordering. And you need to define different functions every time you want to be able to use either a type class or something user configurable. Because you can't redefine type classes locally or pass them as first class instances there's basically no way around this. You can normally avoid the issue with selective use of newtype and careful choice of your types, but it's definitely there.
Which brings me to another point - type classes are not first class. The Scala encoding of type classes is clunky in many ways, but a very nice feature is that the things you're using as type classes are real first class members of the language, and you can bring to bear all the usual tools you have for manipulating stuff.
10,000 compiler extensions. GHC introduces so many compiler extensions, and everyone goes wild about them. This is understandable. Some of them are really useful. Some of them it's amazing to manage living without (No multiparameter type classes in H98? Yeargh. Getitaway!!). I really like most of these I've used, but it sometimes make it feel like you need to employ deep and profound type system voodoo to get anything done. It would be very nice to get these unified into something sane and consistent, but Haskell' seems to have stalled. Tuples and records. There's basically no good generic handling for either of these. Each of the different sizes of tuples is a totally different type (with no common type classes for abstracting over them), records are just a thin wrapper over normal datatypes with no extensibility or namespacing. Your record accessors will clash just as much as any other function name.
Something which has bitten me in the past is sharing code between monadic and non-monadic contexts. You essentially need to write pure and monadic versions of a number of functions. It would be nice if generic monadic code could automagically be specialised to the identity monad in a way that didn't uglify pure code using it, or pure code lifted to monadic versions (this is harder I think). This doesn't come up too much, but it means that there's often functions like foo and fooM for pure and monadic versions.
The prelude and standard type classes are a bit painful sometimes. Things which you'd expect to be overloaded into type classes aren't (Data.Monoid defines a function mappend for example, which in the List instance is ++. Why isn't ++ on Data.Monoid?) and sometimes the type classes which are there are poorly thought out (Num shouldn't extend Eq, and it would be nice if + was factored out in order to provide better support for things like vector spaces). I believe there is some work on alternate numeric preludes.
Plus lots of other little things which don't spring to mind at the moment.
Also, one final disclaimer. Please please please don't take this as a "Haskell sucks, don't use it!!!" post. If you do I'll... I don't know, give you a really devastating hurt puppy look or something.
I'd also appreciate it if you don't use this post to start a language war. Remember - you're only allowed to say bad things about languages you actually like. Otherwise you're cheating. :-)
I'm going to be prototyping some stuff in Scala at work in the coming week, and wanted a nice way of marshalling things to/from files and across the network. The BytePickle stuff in scala.io does nothing for me, and Java serialization gives me the screaming heebie jeebies, so this prompted me to get off my ass and do something I've been meaning to do for a while - port something akin to Haskell's Data.Binary to Scala using the encoding of type classes I've previously discussed. Well, it's done - it didn't take very long at all. The port is *extremely* loose - in particular I've just written it for imperative use rather than define custom monads for reading and writing in a pure manner (sorry). The project is hosted on google code at http://code.google.com/p/sbinary/
At its heart it's extremely simple:
trait Binary[T]{
/**
* Read a T from the DataInputStream, reading no more data than is neccessary.
*/
def reads(stream : DataInputStream) :T;
/**
* Write a T to the DataOutputStream.
*/
def writes(t : T)(stream : DataOutputStream) : Unit;
}
object Operations{
/**
* Use an implicit Binary[T] to read type T from the DataInputStream.
*/
def read[T](stream : DataInputStream)(implicit bin : Binary[T]) : T = bin.reads(stream);
/**
* Use an implicit Binary[T] to write type T to the DataOutputStream.
*/
def write[T](t : T)(stream : DataOutputStream)(implicit bin : Binary[T]) : Unit =
bin.writes(t)(stream);
}
Err. That's it. Did you want more? :-)
There's more to it than that of course, but most of the rest of the code I've written for this is just helper methods, instances and scalacheck tests.
Out of the box this will serialise tuples of any size (that Scala supports. i.e. of 22 elements or fewer), lists, arrays, immutable maps, options, Strings, all the AnyVal types and any combination thereof. Looking at the code should give you an idea of how to define your own Binary instances.
Using it is very simple. It works by knowing the type of thing you want to read or write from the stream and selecting the appropriate logic based on that type (but, unlike Java serialization, if you give it the wrong type it will attempt to read it as that type anyway and probably do crazy things - this is very explicitly using the type to define a compact encoding and doesn't select it based on dynamic information from the stream). e.g.
import binary.Operations._;
import binary.Instances._;
val foo = read[(Int, Option[String], List[Int])](inStream);
write(foo._2)(outStream);
The read and write methods on Operations take care of selecting an appropriate implicit instance of Binary and combining them to do the right thing.
Note that binary serialization logic is kept entirely external to the class, so it's almost as easy to define for classes from external libraries as it is for your own.
I'm not doing an official release yet - I want to have a play around with this and see how usable it is. Once I have, I might change the API around to improve it. On the other hand, the code works now and does enough (within its very simple objectives) that it's probably useful. I've written a bunch of scalacheck tests for it and am reasonably confident it gets all the current binary instances right. If you want to use it for something, go right ahead! Report back to me and let me know how it goes.
Edit: By the way, this only works properly on 2.6.1 or higher. There were some problems with the implicit arguments implementation prior to then that prevent the instances from working correctly.
synchronized(myMap){
doStuffTo(myMap);
}
Concurrency is not an afterthought. If you're going to be doing concurrent programming you should be using datastructures designed for concurrent use. java.util.concurrent has a number of good ones. Further, you should avoid explicitly synchronizing if at all possible and have your structures be internally threadsafe. If you try to ensure thread safety by synchronizing on the structures you're mutating you will
a) Make a mistake. Almost certainly. This will introduce bizarre bugs which you will have a serious headache tracking down.
b) Have worse concurrent performance than using a properly designed datastructure - e.g. a ConcurrentHashMap has finer grained locking, so it actually is possible for multiple threads to write to it in a safe manner.
c) Have really ugly code with synchronization logic spread all over the place. This is not a minor point - if your threading code is simple, it's much easier to determine if it's correct (although still not easy).
def swap(xs : Array[T forSome { type T; }]) = xs(0) = x(1);
The type is not completely unknown, and is persisted across method invocations, so for a given fixed instance you can make use of that to perform operations that would be impossible with wildcards. In particular the following can't work:
public void swap(List<?> xs){
xs.set(0, xs.get(1));
}
This can't work, because Java has no way of determining that the two wildcards in xs.set and xs.get are the same.
def foo = "Hello world"; println(foo()); def bar() = "Goodbye world"; println(bar);Pop quiz: Does this code compile? If not, which bit breaks? No cheating and running it through the compiler! Answer: No, it doesn't. Because foo was defined without an argument list, it can't be invoked as foo(). However, despite bar being defined with an (empty) argument list we can invoke it without one. I could keep going, but I won't. The short of it is that there are a lot of these little annoying edge cases. It seems to give beginners to the language a lot of grief.
This is just a quick introduction to one of the features of Scala's generics. I realised earlier on IRC that they're probably quite unfamiliar looking to people new to the language, so thought I'd do a quick writeup.
What does the following mean?
trait Function1[-T1, +R]
It's saying that the trait Function1 is contravariant in the parameter T1 and covariant in the parameter R.
Err. Eek. Scary words!
Lets try that again.
A Function1[Any, T] is safe to use as a Function1[String, T]. If I can apply f to anything I can certainly apply it to a String. This is contravariance. Similarly, a Function1[T, String] can be quite happily treated as a Function1[String, Any] - if it returns a String, it certainly returns an Any.
So, Foo[+T] means that if S <: T then Foo[S] <: Foo[T]. Foo[-T] means that if S <: T then Foo[T] <: Foo[S] (note the swapped the direction). The default Foo[T], called invariant, is that Foo[S] is not a subtype of Foo[T] unless S == T.
Examples of this sort of behaviour abound. Covariance is more common than contravariance, because immutable collections are almost always covariant in their type parameters. An immutable.List[String] can equally well be treated as an immutable.List[Any] - all the operations are concerned with what values you can get out of the list, so can easily be widened to some supertype.
However, a mutable.List is *not* covariant in its type parameter. You might be familiar with the problems that result from treating it as such from Java. Suppose I have a mutable.List[String], upcast it to a mutable.List[Any] and now do myList += 3. I've now added an integer to a list of Strings. Oops! For this reason, mutable objects tend to be invariant in their type parameters.
So, we have three types of type parameter: Covariant, contravariant, invariant. All three crop up and are quite useful.
But there are safe ways to treat mutable objects invariable. Suppose I want someone to pass me an array of Foos, and I have no intention of mutating it. It's perfectly safe for them to pass me an array of Bars where Bar extends Foo. Can I do this?
Well, this can indeed be done. We could start by doing this:
def doStuff[T <: Foo](arg : Array[T]) = stuff;
So we introduce a type parameter for the array. Because T will be inferred in most cases, this isn't too painful to use, but it can quickly cause the number of type parameters to explode (and you don't seem to be able to let some type parameters be inferred and some be explicitly provided). Further, we only care about the type parameter in one place. So, let's move it there.
def doStuff(arg : Array[T forSome { type T <: Foo }]) = stuff;
This uses Scala's existential types to specify that there's an unknown type for which this holds true. This is effectively equivalent to the previous code, but narrows the scope of the type parameter.
The equivalent using Java style wildcards would be:
def doStuff(arg : Array[? <: Foo ]) = stuff;
But this isn't legal Scala. This is unfortunately a case of Scala being more verbose than the Java equivalent. However, it's not all bad - because of the explicitly named type inside the forSome, you can express more complicated type relationships than wildcards allow for. For example the following:
def doStuff(arg : Array[T forSome { type T <: Comparable[T]}]) = stuff;
And that's about it for variance in Scala. Hope you found it useful.