Matt's Mind

Lisp take 2

2011-03-26T00:32:00.008+10:30

(This post has the dubious honour of being my first in 4 years. I had all sorts of grandiose ideas about moving this blog from blogger to something slightly more hip like posterous or my own hosting, but that would really have been such a waste of time. In the meantime, the planned move was a good excuse not to write anything. Further excuses like getting married and the birth of our daughter then presented themselves. What decided me to write again was re-reading some old articles and comments, and discovered some things I didn't even remember writing as well as some interesting comments - I realized I missed blogging.

Just for added nerd points, this article was written on an iPad. Using the touch keyboard. Yes, I also, apparently, did not get the memo that the iPad is only for consumption of content [link is to John Gruber's teardown of that strange meme].)

I recently re-read an article I wrote in 2007 about not being able to think in Lisp and worrying about what that meant to my development, or otherwise, as a programmer. Since then, I've resembled even more the developer's equivalent of a midlife crisis, and tried quite a few other languages, including Haskell, Scala, Groovy and, Brainfuck (just joking about that one but, frankly, I'm not convinced J or Forth, which I also had a look at, are that much different :) ).

Recently I've started a new commercial project using Clojure for the server backend and Objective-C on iOS for the (iPhone) client. And I'd like to report that I actually believe I can now, finally, think in Clojure, i.e. Lisp. Yay. It surprises and elates me to no end as it turns out.

What's more, writing significant software in Objective-C has given me an excellent basis of productivity comparison with Lisp and, I have to say, the Lisp code is so much more information dense,* it's a little scary. Partly so because I'm worried that means it's a write-only language from the point of view of anyone else. And also partly because I may not want to go back to OO Java land again.

That's not to say I don't like Obj-C, I do, and I really like the Cocoa/UIKit framework on iOS too: it's quite clearly the result of software API craftsmen working at the highest level. But things like the .h/.m file separation and, particularly, the lack of literal syntax for lists and maps are constant irritants, the latter so much so that I've started writing complex data structures in JSON and using a library to translate this into NSDictionary's/NSArray's at runtime, because it's just too verbose otherwise. And tracking retain/release bugs can be a huge black hole for productive time.

Did learning Lisp make me a Paul Graham-certified programming god? I rather suspect not, but I now I know what I was missing out on. And it was most certainly worth it in terms of mind-expansion and generally feeling more secure as a programmer nearing his 40's.

* The server, which implements a fairly scary data replication database, is less than 3,000 lines of Lisp. Whereas I suspect at least one of the Obj-C classes is probably that big by now :/

A close encounter of the spiny kind

2007-08-11T19:59:00.000+09:30

Yesterday I had an encounter with a very rare creature, an encounter which I suspect will be hard to top. It was effectively right in my backyard, for one of the advantages of living where I am in Adelaide is that getting out of the city into the (semi) wild is only a 20 drive up into the Adelaide Hills.

While the hills are certainly not an unspoilt wilderness, and have been changed quite a bit by introduced species and marauding ape-descendants, large areas are protected as National Park, and some of the more tolerant Australian wildlife still calls it come. This includes increasingly-ridiculous numbers of Koalas in some places: a group of us walkers once counted 21 Koalas in 1.5 hour walk.

Koalas are common as muck, and it's only interesting if you see one actually moving, which they seem to do about three times a day [1]. Kangaroos are more rare, but there are at least two groups that live in the areas run and walk in so it's interesting, but not a major event, to see them on the occasions that they are out, usually in the evening in open grassy areas.

But a real rarity and treat is seeing an Echidna [2], small spiny little buggers that are right up there with Platypuses in the weird stakes. Classifying them gave taxonomists some trouble, since they're basically mammals, except they lay eggs. They are small, secretive and roam widely, characteristics guaranteed to make sure you won't see one often.

I was out running starting from a place called Chamber's Gully up to a ridge, which is imaginatively named Long Ridge, cos it's quite long, you see? I went up the nice easy tourist trail to the ridge, and when I got to the top, with some spectacular 360º views, I found that the sun was setting and decided I needed to cut the run short.

So I watched the sun go down over the ocean and then headed down what is basically the opposite of the nice track I came up, basically part-time stream masquerading as a track, cutting vertically down the side of the hill. But it has the virtue of halving the time needed to get back to the bottom. [3]

I was on my way down this track in twilight, with an evening breeze from the plains pushing back on me, when I saw, shuffling up towards me on the same track, an Echidna. I stopped dead immediately; the wind direction had obviously worked in my favour since it showed no sign it had seen me and kept right on shuffling. So I squatted down and watched it come towards me. And keep on coming, until it nearly bumped into a running shoe, which it carefully sniffed with a pointy snout. It then sniffed the other shoe, looked around myopically with little black beady eyes, and then, with me trying not to laugh out loud, it started to work it's way between my feet and got stuck with me squatting on it.

Now this is not a situation I had anticipated being in that evening and, if there had been any reasonable prospect of someone coming down the trail and seeing me squatting on a protected native species, I would have been inclined to get out there immediately. But there wasn't anyone else, and the Echidna seemed to have decided that this wasn't a bad place for a break, even if it did smell slightly like ape-descendant. So I felt I could take my time working out the best way to deal with this.

It was when the spines started to make themselves felt that I started imagining what a suddenly-startled Echidna, wedged in what must be admitted is a fairly sensitive area, might do to my chances of producing any of my own personal ape-descendants. So, I quickly devised an executed a cunning plan, which involved suddenly leaping up and forwards, and thus simply dealing with a steep rocky path and gravity, rather than have an angry monotreme wedged in my crotch.

Amazingly, the plan worked and this turn of events didn't seem to faze the little bugger. What it had previously judged as a warm, dark hole to rest in for a bit had just leaped out of the way, but no worries, these things happen. It shuffled off in silence, while I continued to try not to wee myself laughing at the utter ridiculousness of what had just happened.

I once saw one jump between two trees and miss, which may also be hard to top.
See wikipedia entry. This article is also worth a read.
See long ridge between Greenhill and Waterfall Gully Road on this map. If you zoom in on the head of the ridge, you can see the vertical track heading off in the direction of 3 o'clock down to Chamber's Gully.

Followup to "A Quick Restorative"

2007-05-13T17:00:00.000+09:30

Just a quick followup to the previous entry. Since I made those comments about resource forks not being backed up I've had reason to rethink. The reason I thought they were not being backed up up was that MS Office 2004 stopped working with an error about a broken Carbon shared library. I naively assumed this was a resource fork issue, but it turned out that erasing and re-installing Office didn't fix the problem. Plus, other native Carbon apps stopped working with the same error. Oddly, Eclipse and other SWT apps, which use the Carbon API, were OK.

So, I had to bite the bullet and do what the Apple technical docs say, and do a system reinstall to fix Carbon. I was so not happy about doing this — in Windows this is just a world of pain, plus I was just about to go overseas on holiday and rebuilding the system just before a long trip sounded like an insane thing to do. But the "archive and reinstall" option that OS X has sounded straightforward, and I could always restore from backup, so off I went.

Despite my trepidation, it went unbelievably well. I took the opportunity to not install all languages and printer drivers this time, which gave me back a few GB of disk as a bonus. Then re-installing OS updates cost a couple of hundred MB of download, but I had a fixed, working system, including Carbon apps in about 2 hours. And everything else about the system, except the development tools and my UI customisation haxies migrated across.

A Quick Restorative

2007-03-13T17:25:00.000+10:30

A while ago I wrote a short HOWTO that describes how to back up a Mac OS X system to another drive using rsync. This was not for the pure academic enjoyment of it, it's what I use to back up my sidekick PowerBook, and I recently got to see how well it worked when The Worst Happens.

Actually, the worst would probably be the laptop being stolen. What actually happened is, without warning, the hard drive lapsed into brick mode. In the short interval between being the heart and soul of the PowerBook and becoming an interestingly-shaped piece of metal, it started making strange noises. Being a professional computer geek I knew what this meant and wasn't even that surprised—laptop hard drives are notoriously fail-prone and mine had been spun up and down several times a day for 2 1/2 years plus the usual knocks laptops get when travelling. And there's that warm glow called Extended Warranty.

What did surprise me was that even while I could hear the HDD dying, the SMART status readout in Disk Utility confidently continued to read "Verified". Some recent data from Google (PDF) and the blogs that popped up in response seems to indicate SMART just isn't very reliable for detecting when your drive is about to fail—here's another data point to back that finding up.

Anyway, when the drive started coughing, I tried to get some recent data off it, which just accelerated its demise. So I booted off the external backup drive and had my system back. I was just about to start feeling smug when I realised I hadn't backed up for nearly six weeks. Irritating, but I hadn't lost much and still had a working system to use in the meantime. A bootable backup is a must-have IMHO, you can limp along with this in the time between it being fixed. Commercial systems that create non-bootable backups are really hobbled in this regard.

A week in the AppleCare hospital later and it had a new HDD. Reverse rsyncing from backup took about 2 hours and, after some messing around with the "Startup Disk" preference pane (the "bless" command didn't seem to stick for some reason), the machine booted and I was back to December 27, 2006 in PowerBookLand.

So the backup and restore system worked. I really can't even begin to estimate how long it would have taken to reinstall Mac OS X, download 100's of MB's of updates and applications, reinstall Office, XCode, Eclipse, etc, tweak my settings, restore music, contacts, calendar, photos (assuming I had those), ...

The more I think of how many things most people don't back up which cost them so much time and energy to set up I find it astounding. Of course there were three things in my favour

I knew how much a disk failure and system rebuild would cost, having experienced this before,

I had the technical knowledge to make a real backup rather than copy random directories every now and then, and

I was using an OS that makes it easy to clone an entire system.

Because Mac OS X doesn't assume you're a pirate and require an activation scheme to prove you're not one (such as that used by some other companies which I won't mention but which sound like "Microsoft") it makes it possible to clone the system without the OS working against you. This is something I really appreciate about Mac OS X.

One thing that didn't work was resource forks. None of these got restored because, as it turned out, none were being backed up. I just didn't check—when the system booted of the backup and everything appeared to work I naively assumed resource fork backup must be working. However the reason the backup worked is that almost no software seems to rely on them anymore.

This was irritating, since the part of the point of the HOWTO was to add info on how to preserve these. But it certainly didn't have a large effect. What tipped me off is none of the Classic apps in my old "Applications (Mac OS 9)" folder which need the System 9 emulator work any more. No biggie, the System 9 apps were interesting for historical reasons, but I came into Mac land for Mac OS X.

More annoying, none of the Microsoft Office 2004 applications run. They're not even recognised as applications any more. Neither are Carbon ports of OS 9 applications such as Graphic Converter. Still, this should be fixable, and once I've fixed them I'll start investigating why resource forks aren't making it through rsync and update the HOWTO.

Vista

2007-03-05T20:35:00.000+10:30

I just saw this ad on an IT shopping site:

Holy crap. My jaw dropped: AU$1,600 for Vista* plus Office. You can buy an entire PC for less than that. Hell you can buy a Mac mini plus a screen for less than that. What is the world coming to when Apple is the value-for-money option?!

I don't understand why MS even bothers with the retail boxed version of Vista/Office when it's about appealing as a shite sandwich.

* An XP -> Vista upgrade is a far more reasonable** $380

** For a given value of "reasonable".

Error handling and elegance

2006-12-16T19:08:00.000+10:30

I was analing¹ some code today, and seriously reduced the size of some knotty configuration handling logic, which made me happy².

Here's the core Java logic as a snippet:


try
{
  ...
  config.setAll (propertiesFrom (fileStream (stringArg (args, ++i))));
  ...
  System.exit (0);
} catch (Exception ex)
{
  System.err.println ("Error starting app: " + ex.getMessage ());
  System.exit (1);
}

The code just assigns all configuration properties read from a file stream, where the file name is read from the next command line option. I suppose some might argue that it's too highly nested, but I'd say that if you read it from left to right you get close to reading out its meaning: "set all properties from file stream ...".

All the actual logic in the snippet is contained in functions, but the sheer number of failure modes in this code is huge. The options file could be missing, unreadable, unparseable. There could be unknown options, options in the wrong format, options out of range: setAll () has about a dozen option-related failures alone. The command line parameter could be missing. A malformed UTF-8 character could appear in the bowels of the config file. Anything.

This is not even considering all the usual runtime errors such as out of memory exceptions. And yet the code above handles all of these without distraction from its core purpose, which is after all what it actually does 99.99% of its life. And generates good error messages, if you'll take my word for it.

The reason the code is so simple is that all functions either return a valid result, or throw an exception indicating why they couldn't³. No function ever returns a value that could cause an error later on simply by being invalid.

Now the reason I was pondering this is that Joel Spolsky, someone who I have a great deal of respect for, doesn't like this exception-oriented approach at all. The two main reasons he cites are:

1. They are invisible in the source code.
2. They create too many possible exit points for a function.

These, he argues are bad things. Which puzzles me, because, to me, these are good things. I have to assume *any* line could cause an error and any line of code could except out.

This argument has been done to death by the way, not least of all on Joel's own forum. But I'm going to wade in anyway because I don' have anything better to do, and no one reads this blog anyway.

C is a language that follows Joel's prescription for error handling. It doesn't have exceptions, and instead usually uses the function return code to indicate success/failure (and the global errno variable, which is the work of the devil ... don't get me started). You can alternately pass a pointer to an error info variable which gets filled in if there's an error, which I've seen done on occasion. As Spolsky puts it, best practice in C is to "[...] have your functions return error values when things go wrong, and to deal with these explicitly, no matter how verbose it might be."

Both ways require error checks spattered throughout the code. Which, and here is where I wade into the debate swinging wildly, leads to ugly code that doesn't compose nicely - for instance you can't pass one function call as a parameter to another. However, in C this sort of chaining is often confounded anyway by the fact that allocated memory returned by a function must be deallocated.

Do we really like longer, more complex code, where the flow is broken by all the error checking? And this repetition leads programmers to be lazy: "why bother check the result of this malloc()? It's a tiny amount of memory, it'll never fail" ... SEGV.

Is this just my opinion? Quite possibly, but I decided to do my best to code the important logic above in C to see what it would be like as an alternative. The C version tries to keep the important properties of the Java code above, including trying to provide useful error messages (something that many C programs utterly fail at, leading to hilarious "Error: there was an error"-style messages).

Here's my best attempt:

/* better hope message doesn't overflow this */
char *message = char [255]; 
char *error = NULL;
properties_t *properties = NULL;
char *file_name;
FILE *file_stream = NULL;

if (!(file_name = string_arg (args, i)))
{
  sprintf (message, "Missing argument: %s", args [i]);

  goto error_handler;
}

if (!(file_stream = fopen (file, "r")))
{
  sprintf (message, "Failed to open file %s: %s",
           file_name, strerror (errno));

  goto errorHandler;
}

error = NULL;

if (!(properties = properties_from (file_stream, &error))
{
  sprintf (message, "Error reading properties file: %s", error);

  free (error);

  goto error_handler;
}

if (error = config->set_all (properties))
{
  sprintf (message, "Error setting properties: %s", error);

  free (error);

  goto error_handler;
}

/* done! */
exit (0);

error_handler:

fprintf (stderr, "Error starting app: %s", result);

if (properties)
  free (properties);

exit (1);

I should note that I cut my programming teeth on C, and got to what probably was a pretty reasonable level. But nowadays that may not be the case and the below might be considered bad style. If so, I'd be really interested to know how it could be improved.

Footnotes

1. A new verb I've invented for obsessively tweaking perfectly good code.

2. And simultaneously sad in the eyes of most people reading this. Irony.

3. The fact that GC handles cleanup for us is also a big reason why this code is so simple.

Another reason why I like static typing

2006-11-20T20:16:00.000+10:30

I just finished porting a fledgling project of mine from one version of a library API to a new one. The decision to convert was a no-brainer since the designers of the library had improved the API quite a bit over the previous one, and of course I wanted to be able to follow future updates.

But the authors hadn't actually bothered to describe the changes or anything, so I just basically dropped the new JAR over the old one and looked at the red error markers scattered over the code files that turned up in Eclipse. This is the key place where static typing made life much easier: I had a clear map of what needed to be updated and how much work it looked like it was going to be.

Things ranging from the obvious, like missing classes due to renaming, to more abstract errors such as unimplemented interface methods and exceptions that were no longer thrown were flagged. What was left was actual semantic changes and, after grokking some of their updated example code, a few search and replaces later: done.

All in all, surprisingly painless considering I was going in blind. With a dynamically-typed app it I suspect it would have been considerably less scientific. I suppose I'd start by running tests and build a list of missing method errors (assuming I have 100% test coverage).

But even when fixing these and having the tests pass I'd still be paranoid: what about "accidental" passes? Things like changed parameter lists, where extra parameters passed in but no longer used. And what happens when you use a library that uses dynamic tricks like handling missing methods in some clever way so what you're calling is no longer necessarily doing what you think it is?

Don't get me wrong, I think things like this are very cool and would probably use them myself. But I worry that there is a price, which is to make the code harder to understand and evolve.

The Real Alien 3

2006-02-26T14:22:00.000+10:30

At some point in my travels around blogland I made a note to have a look at William Gibson's unused screenplay for Alien 3. As I started to skim it I found myself getting sucked in and I'm now halfway through. And you know what? The pictures are better in text. There are sometimes only perfunctory references to special effects scenes "... ANGLE ON THE HULL A towering cliff of metal, Sulaco..", but pieces like

The ship slides past beneath us. A U.P.P. interceptor descends INTO FRAME, matching course and speed with Sulaco. The interceptor settles on Sulaco like a wasp.

gel into a perfect SFX version the head, no CGI required.

The other great thing is the screenplay format itself: it reads much faster than a novel. The characters are certainly more one-dimensional than in a novel, one of the key differences in novels is that you get direct access to the character's thoughts, which is not as feasible in film. So maybe I should be saying the screenplay reads much faster than watching the movie.

Reading this non-existent Alien 3 is like being in an alternate universe, it's kind of a parrallel-quel, to coin a word that will annoy grammar nazis. The actual Alien 3 wasn't totally bad — it had atmosphere and was OK as a stand-alone film — but it didn't feel like one of the series. The fact that it killed off three of the key characters from Aliens after their heroic struggle to survive in the previous movie didn't endear it much either. With its original characters, gloomy claustrophobic space setting, William Gibson's Alien 3 feels like the true sequel to Aliens.

Heatwave hacking

2006-01-22T19:34:00.000+10:30

Well, it's officially a heatwave here in Adelaide: four consecutive days over 40° celsius. And now finally a cool change is moving in, along with smoke from bushfires in various places around the state.

After the last few days a mere 30° feels nice and cool ;) 40° is at the level of heat where everything just feels hot. Put on a t-shirt and it feels like it's just been ironed. And even if you just sit still the shade you still sweat.

So, a good day for staying inside. And sorting out my photo collection. Yes, I definitely know how to get down and party.

The Problem

For historical reasons I currently have two sets of photos, one old set manually ordered into a "logical" set of folders and the new one in an iPhoto library. These overlap to a great extent, but I'm not totally confident that every photo in the older directory is now in iPhoto. Since there are about 2,500 photos I'm not too keen to match them all up myself. But I need disk space on my laptop, and removing 1.8GB of redundant photos would be handy.

The Solution

The solution is obvious: write a utility in language I've never used before to find files not in iPhoto and then plug it into a tool I've never used before to browse them. The recipe for a happy day of hacking :/

The Language

The chosen language: Ruby. For various reasons the other obvious candidate, Python, doesn't appeal and Ruby has a number of things to recommend it from my perspective. One, it seems to be Perl Done Right, and I'm a old-time Perl hacker who loves the power of it but hates the actual language. Ruby has proper OO, clean syntax, Perl regular expression goodness, and blocks up the wazoo.

The second reason is Rails – at some point I really want to try this out, and knowing Ruby would obviously be a good idea for that.

Ruby Experience

The Ruby experience was very positive. Apart from the usual thrashing needed to work out the nuts and bolts of a new language the script came together quite easily. Although I really missed the autocomplete and API browsing I get with Java in Eclipse. The ability to jump around the library without messing with a separate web browser is a godsend when learning a new API

I needed to build a table of JPEG files in the two directory trees and then essentially do a "diff" on them (script is here if you care). For this task, Ruby's libraries were quite sufficient, although somewhat inconsistent. Strange things popped up like the Array.nitems() method, which actually gives you the number of non-null elements vs Array.size() can confuse if you just browse the method names and don't read the spec fully.

Other people have noted that Ruby's libraries can be a little lacking in coherence too. Another example: the Dir class used to access directories is quite good, except the pattern used to scan directories is gratuitously different from a regex. But nothing's perfect, and on the whole I think Ruby is now my language of choice for these sorts of hacking tasks.

Automator

My script spat out a list of 600 files that appeared not to be in the iPhoto library. This was more than expected, and I needed a way to quickly inspect these and add any missing ones to iPhoto. The obvious route was to get them into Preview, from where I could then drag photos to iPhoto as needed. The obvious way to do this: connect my script and Preview with Apple's Automator.

Now, Automator is pretty cool and at first this looked like a no-brainer. I dropped a "Run Shell Script" action in, told it to run my Ruby utility, then connected an "Open Images in Preview" action. Automator seemed happy with the match. But hitting "Run" made Automator do nothing but fire up an empty Preview and then beep happily.

Of course hardened Mac heads are chuckling at this point since they know my mistake (the smug bastards). For some reason, even though the Mac has gone partly POSIX underneath and uses file/paths/with/slashes sometimes and eschews resource forks (in theory), it still seems to use HFS file:paths:with:colons for Cocoa (GUI) apps. My script was generating POSIX paths, and I needed to feed Preview HFS ones.

I won't bore you with the Googling that followed, but some time later I had worked out to plug the AppleScript below in between the script and the Preview task (screenshot here (160K)):


  on run {input, parameters}
    set output to {}
      repeat with i from 1 to length of input
        set x to item i of input
        set output to output & {POSIX file x}
     end repeat
     return output
  end run

Once I had grokked this, Automator hummed away and fairly quickly I had Preview showing 600 photos. It turned out most of the "missing" photos were ones I had rotated or otherwise edited in iPhoto. But I did catch a couple that had slipped through the net. Well worth the hours of hacking ;)

Mac: one year on

2005-12-29T20:29:00.000+10:30

I've now passed my one year anniversary of switching to a Mac PowerBook as my primary home computer, and I'm still loving it — this is still by far the best computer I've ever owned. But, as they say, familiarity breeds contempt, and although I certainly haven't developed contempt for the Mac, it does seem that one year of use has been enough to learn about the Mac's warts as well as its beauty.

One of the deeper problems that has been bothering me about the Mac interface is how much the look of the desktop is becoming muddled with differently-themed applications. It's worrying because it seems that the applications on the Mac may have started to move towards the visual hodgepodge that is the norm on Windows, where I spend most of my working time. And, worse, it's often the developers at Apple that are the worst offenders in violating their own Human Interface Guidelines.

Just a few trivial examples:

Safari in brushed metal, just wrong.

Finder, either brushed metal in one mode, or Aqua in another, more than wrong, just stupid ... "what were they thinking?" stupid.

I won't even comment on Mail's toolbar changes in Tiger.

Don't take it from me: this is not just my personal insight, many more experienced Mac-heads out there have been bagging Apple about this. The Woz himself (Steve Wozniak) commented in a recent interview that he believes much of the original Mac ethos is gone:

... the big companies, Microsoft, Apple and AOL, they tend to turn out the crappiest products, you know, software-wise. The ones that have the most bugs, the most items that are supposedly in there but don't work. The most things that are left out because they aren't finished. The most things that are inconsistent with the way they did their last program. I get the worst, worst software almost always from Apple.
... It's like, there was such a cleaner, good approach to it and they did this stupid thing. But remember, the people who wrote the OS X weren't the people who developed the Lisa and Macintosh. Those guys are gone.

The original Mac purists aren't at Apple any more: they've been replaced by Job's cohort of ex-NextStep developers.

John Gruber has also written about this in several articles, pointing out various questionable design decisions (e.g. "poofing") that are very much against the Mac ethos, and noting that Apple now have introduced at least four OS X application themes. This was before iTunes 6.0 introduced yet another one.

Bottom line: consistence, elegance and usability have always defined the Mac platform &mdash that's something that even recent switchers to the platform like myself understood was the reason for its existence. I'm not against Apple developing new user interface approaches, far from it, but I am against them randomly applying changes with no common underlying logic. These decisions, while they may seem superficial, mark a turning point if they indicate the platform's champions no longer care about the "ethos" behind the Mac. The Mac interface [1] seems to be slowly deteriorating.

I'm hoping Apple is aware of this, and have plans to renew the look and feel in OS X 10.5. OS X 10.0 loudly debuted with colourful, lickable, stripy Aqua which was visually striking and perfect for rapidly making a splash — it turned everyone's head. Since then Apple have gradually toned things down — e.g. they've all but removed the stripes and transparency — but kept the colourful buttons and added a muddle of visual styles.

Colour has never been used well in the OS X interface IMHO. For example the red "close window" button is overly distracting and the blue Aqua buttons also stand out too much over the important content. But Aqua has at least avoided the overload that many Windows apps inflict, bristling with colourful buttons on all window edges and floating palettes [2]. And Apple provides the "Graphite" option for those who find the colours distracting.

I would like to see Apple start to back off from colourful, transparent and gaudy and go for modern, minimal and sleek. At the same time in 2006 as Microsoft, in a bid to look modern, is stuffing Windows Vista with arrays of gaudy icons, gratuitous glassy transparency and animated everything, Apple could outflank them by simply redefining "modern" as it did in 2001 with OS X 10.0 making Windows XP look like a Fisher Price knockoff.

One way they could go is best embodied by a theme I saw being used by one of my favourite Mac bloggers, drunkenbatman, in screenshots of his desktop. What intrigued me was how minimal the theme was and how it erased Metal and tamed Aqua while looking elegant and letting the content in the windows take precedence over the frame decorations. I tracked this down to the "Milk Chrome" theme for ShapeShifter by Max Rudberg and installed it yesterday and have been absolutely loving it. It's erased a whole array of visual irrits that have been bugging me for a while, including banishing Metal for good. Unfortunately it can't fix Finder: only a bullet to the head could do that.

Here's a screenshot of my desktop with the new theme (click to enlarge):

Note how it erased Metal from Safari and the Calculator and even brought Mail into line somewhat. The white buttons and background get out of the way without being harder to access — the shading, shinyness and outlines of the controls are enough. The whole feel is actually a little like an iPod in fact: solid white rounded plastic controls.

There are some minor rough edges: the button press animation could use some work — it makes the button appear to recess into the background, but isn't quite noticeable enough; and the theme messes badly with iTunes 6, which I guess is not surprising since iTunes apparently has its own custom look and feel.

As far as I'm concerned, this one's a keeper until Apple can do better. It doesn't do anything to fix the various functional issues highlighted by Gruber et al, but cleans the visual appearance nicely. It does disturb me that I feel I need to go hack Apple's look and feel: after all they're supposed to be the masters. It will also concern me if Apple does not take the criticisms of well-respected pundits like Woz, drunkenbatman and John Gruber into account. Apple have been very quiet about OS X 10.5, so I can only hope that, disregarding the usual Apple fascination with secrecy, it's because they have something great in store.

Footnotes:

[1] That's "user experience", for any Microsoft marketdroids who might be reading this.

[2] What is it about Microsoft's button fetish anyway? Since they "innovated" the toolbar on Office some years ago they've gone utterly nuts with little nobs and buttons. When I first got the Mac and ran Entourage I could immediately tell it was a Microsoft app, with tiers of horizontal and vertical toolbar buttons stuck everywhere. This was one of the reasons I switched to Apple Mail: it appeared they'd done some work on making the interface easy rather than busy. It's like those remote controls that actually have fold-out flanges to hold extra arrays of obscure buttons.

AppleScript ... argh!

2005-12-28T18:55:00.000+10:30

I just spent more than an hour wrestling with AppleScript, a language that is supposedly easy to use. Well maybe it is easy read, but it's not necessarily easy to understand.

Quick quiz - what do you think the following prints?:


set ch to "."

if ch ≠ "." then
  log "not a dot"
else
  log "a dot"
end if

A cigar if you went with "a dot". Now, what will this print?


repeat with ch in characters of "a."
  if ch ≠ "." then
    log "not a dot"
  else
    log "a dot"
  end if
end repeat

You'd think that since I just wrapped the previous example inside a repeat statement over the characters of "a." that you'd get "not a dot" and "a dot". I'm sure that by now you will have guessed that this is not what you get, or I wouldn't be whinging. In fact you get "not a dot" twice. After reading the AppleScript docs regarding characters, strings, loops and boolean operators I'm still stumped.

The solution? Add this after the "repeat":


  set ch to first character of ch

This makes me think that this is some sort of variable type issue, but I really can't be assed spending any more time on this.

WMA is Scrod

2005-10-27T11:37:00.000+09:30

So, Apple has finally launched the iTMS music store in Oz. I've been thinking that Microsoft's WMA music format and associated Janus DRM are in trouble for a while, but kept quiet because of MS's famous talent for coming from behind (e.g. behind Netscape and Palm) and eating the competition. But I think the tipping point for this race went past some time ago, so I can now happily say in public that I cannot see any way in which MS's dreams of a media monopoly are not, in fact, totally scrod.

The numbers tell one part of the story: 600 million AAC/FairPlay songs are out there and iPod has 70-75% of the portable market share. And, of course, the cute iTunes interface that works on Mac and Windows, against Windows Media Player, which is an odd-looking beast and works only on Windows. The iPod integration in iTunes is perfect (funnily enough) and the iPod interface is a work of genius. And Apple are now busily rolling out video.

This is despite the fact that everything that's not an iPod, from the $700 HDD-based players to the shittiest little $20 flash ones, supports WMA [1]. I can only assume that MS gives away WMA licenses to anyone that looks like they might be building a player. I half expect my next credit card to include a WMA-compatible player on it.

But MS's spray-WMA-around-unti-it-sticks policy doesn't seem to be working. The plethora of bit-players [2] that support it may actually be hindering things by confusing the customers, and MS's "Plays For Sure" campaign seems downright insane, given that WMA does not in fact Play For Sure on three out of any four randomly-selected players on the street.

Napster's WMA-based subscription music store may be making some money but, when people realise they can't put the tracks on an iPod and that their music collection goes away if they ever stop paying their monthly fees, I suspect there will be an exodus. In Australia, the Big Pond music store doesn't even work unless you're running IE on Windows. An article on ZDnet lists the other reasons why Big Pond music is doomed.

I actually do feel a little hypocritical gloating here. After all, if MS, in a fit of unusual innovative fervor, created the mPod (or ActivePod or, more likely given their track record for exciting names, Windows Media Portable Device [3]), had created the mPod first, and tied Windows Media Player, the mPod and mTunes so tightly together, I'd be up in arms and buying only Ogg-compatible devices that require four Linux shell scripts to sync music.

The difference with Apple being in this position is that they're the underdog in everything else, so they have to actually keep delivering cool stuff to stay ahead, rather than doing very little cool stuff and maneuvering to make sure they keep squeezing more money out of old boring stuff.

Maybe in a few year's time I'll be bitching about Apple and its damned music monopoly. But for now I'm just happy to see yet another example of MS failing to parlay it's massive bank balance and Windows/Office hegemony into other areas.

[1] With the possible exception of Sony devices that probably still stubbornly do only mp3 (and that only recently) and ATRAC.

[2] I'm very happy with that term "bit players" It works in a number of ways in this context

[3] Someone once said "if MS made toilet paper, they'd call it MS Butt Wipe".

Open source getting down to business

2005-08-07T14:19:00.000+09:30

I had a revelation about the real reason for open source's huge acceptance for business use the other day courtesy of Linux kernel maintainer Andrew Morton (Linus's right-hand man). I was listening to Andrew's address to SD Forum where he made the blindingly obvious but brilliant observation that free market economics doesn't work for core infrastructure like operating systems and backend services.

The reason you can't have real competition (and thus a free market) for infrastructure technology is simply because the cost of swapping infrastructure massively outweighs any potential gains - you can't switch without re-tooling, re-training and possibly buying new hardware.

So we either have a convergence on a closed monopoly (Microsoft) or we agree on an open infrastructure and concentrate on making money in a free market in the things that we can mix and match (Linux and its software stack). And it's mostly in IT infrastructure that open source is making a killing right now, Linux hosting, Apache web serving, BIND DNS provision, Samba file sharing, etc.

We have little reason to worry, like we might have a decade ago, that Microsoft will be able parlay its monopoly in the desktop space into a monopoly in the server space - the emergence of Linux and the rallying of many of the key IT players behind it (IBM, Sun, etc) put an end to that possibility.

Andrew goes on to express the refreshingly sensible opinion that open source is just a sound method of doing business, not a religion, philosophy or way of life. Closed source development is just as "legitimate", both systems can work depending on the ecosystem they're working in.

Lisp

2005-07-03T14:37:00.000+09:30

I listened to a podcast of Guido Van Rossum, the architect of Python, talking at SD forum recently and, although it made me vaguely irritable the way dynamic typing people often condescendingly assume that "strong typing is for weak minds" (actual quote from an audience member), it did resurrect my desire to learn a new language.

Not Python - despite Pythonistas' claims I've done enough Python to convince me that it isn't different enough from other OO languages to change the way I think about programming. Indeed, since Python didn't start out OO, it can be slightly obtuse in that area - I think I'd like to avoid writing "self.foo" all the time to reference data members, and avoid, too, the subtle bugs that come with forgetting the "self" (I realise that only weak minds would make this mistake though).

I recently revisited Paul Graham's writings on Lisp, and his claim that all languages strive to become Lisp (Guido touched on that too in comparing Python to Lisp), so I decided to have another go at that disturbing language (I'll elaborate on the "disturbing" part next). Luckily Paul Graham has a very good book available online called "On Lisp".

What I immediately found from the first few chapters of Graham's book is that, although I can recognise deep down the elegance of Lisp, I find I can't "think" in it at all. I had the same experience at Uni looking at examples in Lisp and Miranda - after some forehead-wrinkling and mental contortion they finally clicked in my head and I was able to see the mathematical elegance of the solution. Although the elegance can be somewhat lost when "real" Lisp programmers try to avoid non-tail recursions for performance reasons and make the code that much less clean - see section 2.8 of the book if you care.

The problem may be that I had already learnt to program before I went to Uni. I learnt snippets of basic on my friend's Commodore 64. Then I moved to MS GW BASIC when my family finally got our own, IBM compatible, PC. From there I moved to QuickBasic, which lived up to it's name - I've never had such a speed improvement for "free" by just by upgrading compilers, it was almost like magic. QuickBasic had examples of programming with, gasp, procedures with no line numbers. Actually, this procedure thing wasn't totally alien, I knew about gosub/return, but named procedures with actual parameters, wow, what a revelation when I "got" that idea.

From QuickBasic I moved to QuickC. C actually wasn't too much of a leap from procedural basic, except of course pointers and complex data structures were new concepts. But basically C had the same core forms as basic (pointers and malloc/free excepted). And it was a "real" programmer's language. For the first time I was able to develop (what I thought were) real applications. I eventually got into writing low-level 8086 assember inside C to do things like write to the text mode screen buffer, execute the terminate and stay resident DOS call and access the then new MS mouse BIOS driver API's (and I can still remember that it was interrupt 33H).

My rambling back down memory lane is not intended to show that I can't stay on the point. It's intended to illustrate that I've had nearly 20 years (and I'm only 33) of procedural thinking. And while my current language of choice, Java, is a nice language, it wouldn't be at all alien to the me of 15 years ago.

So, I'm getting that same "hard of thinking" problem when reading the Lisp in Paul Graham's book. After some staring, I can get the examples, but I wouldn't have had the first clue how to write them. So it appears learning Lisp might be like my attempts to learn to touch-type, and fail for the same reason - while learning my productivity would be so painfully bad I'd give up.

It does vaguely worry me that the facility with which I can write code one way might be paid for by the inability to learn new, better, ways of doing things. After all, I somehow suspect that we haven't achieved perfection with current languages and tools.

In spite of this, or perhaps, because of it, my own belief is that language has very little overall effect on productivity. Things like GC, protected memory, sophisticated tools, copious good-quality libraries and a massive community support seem to be much more important factors than whether I can sort a list using a custom comparison function in two lines of code rather than 6. And the key factor? The quality of the developer obviously makes the most difference.

But of course I may be wrong. How can I really know whether I'd be a much better developer if I could think in Lisp? But I suspect the human mind at my age (i.e. post adolescence :/ ) has room for only one major language, programming or otherwise.

I'll keep at the Lisp though.

Swing's new text renderer

2005-06-18T17:08:00.000+09:30

It looks like Swing (or more correctly Java2D) is getting sub-pixel anti-aliased text (aka ClearType) in Java 1.6. This addresses one of the chronic problems with Swing's native look and feel emulation - it's hard to look native when you're the only jaggedy-ass app on the desktop.

Or it would address the issue if Swing's slightly retarded font renderer didn't bravely snatch defeat from the jaws of victory.

Look at this side-by-side example. Swing'osities abound. The Sans Serif font is particularly worth mentioning: bollixed up letters include "M", "W" and "y". It looks like it gave up anti-aliasing the "2" altogether. The "#" is sloped in the original, but Swing goes for the more upright look. And the letter spacing for "FOX" is quite original. In fact looking Swing's "FOX" rendering compared to the rendering on the Mac I'm writing this on forced an involuntary guffaw.

I don't want to come across as totally mean-spirited here. It seems that Sun is getting the message. It's just that it also seems they aren't competent to address it. The well-meaning JDIC project is producing components like the taskbar icon API, which developers have already built many times over (I know, I wrote one too), but this one at least has the virtue of trying to be cross-platform (e.g. task bar icons sort of work on Mac OS X too, for a given value of "work").

However, the fact that Swing menus attached to taskbar icons cannot popup over the taskbar means they do the same crazy dancing workaround as all other Swing popup windows and can thus appear several centimeters above the icon when you click on it.

This is kind of emblematic of the Swing attitude right now: let's add anti-aliasing before we fix the basic font renderer. Let's add taskbar icons before we fix the basic popup menu support. Let's add a new look and feel skinning engine before we fix the native look and feel on the world's largest desktop platform.

There's no reason to be surprised that many developers are happy to put up with the extra deployment hassles that come with using SWT - at least these are something we have some control over.

Whereis ...

2005-06-18T16:53:00.000+09:30

I generally use www.whereis.com.au reflexively when looking for maps online. Not sure why, given that they insist on using POST to generate map pages meaning you can't send a whereis link to anyone or bookmark it.

Looking at their URL while cursing their POST technique, I noticed they do put some info in the URL. There's a "brandId" field, which selects which brand logo appears in the page. Slightly amateurish but, even better, setting brandId to an invalid value (e.g. 4) generates a server error (what a 733t hax0r I am).

Even if I couldn't tell they're using Struts by the ".do" in the URL, the NullPointerException gives the game away. I wonder why such a big site can't afford to employ developers with some sort of clue?

Windows ... not

2005-05-06T13:55:00.000+09:30

Just as a followup to my whinge about Mac OS X previously, and to illustrate how Windows usually manages to bollix things up much more efficiently than OS X, here's a screenshot of Explorer on my XP machine - see if you can spot the mistake.

Yes, the tree on the left shows two directories under "work" while the list on the right shows, correctly, three. No amount of F5'ing would convince the bugger otherwise, even though this is a local NTFS filesystem where you'd think MS could get it right by now.

Later on, it got better. I decided to delete one of the three directories and got the dreaded "file in use" dialog. Windows users know this indicates that, no matter what, be they Administrator or even God, they will not be able to delete this file short of booting Knoppix and nuking it from orbit. The only other alternative is the "check every process to see if it might be using the file" mamba

In this case I knew damn well that no process was using the bugger - I had just created the thing. But, because I was desperate (I couldn't get any work done until I could reset that directory) I shut down all cmd.exe's, killed the virus scanner, closed emacs, everything. And still no dice. I went to get a coffee and simmer down ... and when I came back, without having done a damn thing, now I could delete the folder.

This sort of random crap: locked files, flashing icons on the desktop, endlessly scrolling Program menus and drivel-spouting pop-ups drive me mad when using Windows. Actually I shouldn't have mentioned the random pop-ups, for some reason they particularly elevate my blood pressure: Take a tour of Windows!? Hide inactive icons!? ... what I'd like is for you to f*ck off and let me get some work done, son of clippy.

XP came out in 2001, so for 4 damn years I've been putting up with this blizzard of shit every time I log into a new PC. Not mention the fact that turning off the fisher-price "task panes" and other crud has become an automatic process, no longer requiring higher-brain intervention.

Maybe Mac OS isn't so bad after all.

Spotlight ... not?

2005-05-06T13:05:00.000+09:30

OK, I got all excited about Spotlight when I upgraded to Mac OS X 10.4 recently. I've been doing some research into making DVD's from recorded TV and keeping the results (links, text snippts, etc) in a folder. But this folder has gotten very full, so I assumed a better way to find stuff now is to use Spotlight to search for topics of interest, rather than page through it all.

So, I'm looking to see if I've saved any info on a tool called "replex". I go Ctrl+Option+Space "replex" ... nada. Which is odd because I know I saved an email recently (as a text file) on that subject. So I manually find the email. And here's a screenshot of the text file with "replex" in it with Spotlight failing to, er, spotlight it.

I also discovered, while trying to post the screenshot via FTP, that Mac OS X will happily mount an FTP volume (with username + password) but won't write to it. WTF?

Much as I love my Mac, things aren't perfect in Mac land right now.

Update: further investigation showed that Spotlight was only working for files that aren't in my home directory. I'm using FileVault, and I suspect this is related. After logging out and then logging in again, Spotlight is working fully. Let's hope that's fixed in 10.4.1. I'd report this as a bug, but after spending a lot of time trying to find out where to report a previous bug in iCal and then getting zero feedback, I'm not too excited about the prospect.

High Fidelity

2005-04-25T19:37:00.000+09:30

Note: in this blog entry (actually a minor essay, sorry) I'll be using the term "HD DVD" to refer to both of the proposed next-generation high definition video standards: HD DVD and Blu Ray. And I'll be using the term "suits" as a friendly way of referring to MPAA and other music and movie industry middlemen.

So, it looks like HD DVD will have encryption up the wazoo, what a shock. Who would have thought the movie industry would react to the cracking of CSS with the cry of "if at first you don't succeed, just use more?"

They will of course have access to the finest minds money can buy, and indeed the likely encryption system for HD DVD looks pretty impressive - must have cost millions to develop (it's called AACS: see a short description of it on slashdot).

The AACS system guarantees that, if a key is cracked, instead of being forced to continue to produce discs that they know can be cracked (as is the case now in DVD-land), they can issue new keys, revoke player keys and distribute updates onto players via the HD DVD discs themselves. I'm told this sort of system is in use for satellite links, and hasn't been hacked yet.

Yes, the suits must shiver with delight when they consider that they'll finally be one step ahead of those pesky, copy-happy consumers.

Leaving aside the issue of why suits are so worried about small-time copying - the real pirates will continue to do what they've always done and use high-grade bit-for-bit copiers which defeat both CSS and the proposed systems - do they really expect this system to work in the presence of open platforms like Linux, where the last resort for copying is to simply hijack video and audio at the driver level?

Of course not. Encryption is only as strong as it's weakest link, and in order to make it impossible to copy on consumer-grade equipment they need to control your hardware and software from the top to the bottom. This even includes the cables that connect your components, like HDMI, which has anti-copy encryption built in. Because, otherwise you sneaky consumers will just hook your high definition HDMI connectors into a recording setup.

Open systems like Linux are impossible to "protect" from sneaky consumers. So don't expect HD DVD or Blu Ray on Linux any time soon. Or ever.

One of the things that made DVD one of the fastest-adopted consumer technologies ever was the synergy between PC's and DVD players. The massive market for DVD drives on computers helped make them extremely cheap very fast. Unfortunately for suitdom, the hosting of DVD players and their associated software on generic platforms made CSS open to attack from the inside, and it didn't take long to fall.

To be as successfully-adopted HD DVD also needs to ride the PC wave, but this time around, the suits think they can do it right and lock us thieving customers out properly.

But in order for cunning plan to work on the PC, they need a PC platform that is unhackable. Enter Palladium, or Next Generation Secure Computing Base (NGSCB), or whatever it gets called next when the populace figure out what the current obscure name actually entails. NGSCB, or its derivative, requires both "secure" PC software and hardware. Which means cryptographically protected from meddling by untrusted parties, i.e. you. You won't be able to write a driver or create any hardware for the system until you've handed over some serious money and signed away your first-born child to the consortium that controls the HD DVD standard. If you write a hacked driver that permits copying, you go to jail. If you leak a key, they revoke that key, and you go to jail. If they decide they don't like you for any reason at all, they revoke the key.

Nirvana for the suits is when your equipment will lock you out of doing anything but what they've prescribed you can do. Fair use goes out the window. So does competition: to publish titles and make hardware you'll need to sign up and pay a fee to get a key. And, if the current "forced" sections on some DVD's are any indication, once they've got more control we may well see ads on purchased HD-DVD's - all in high definition of course.

It may sound paranoid, and indeed I've been indulging in some worst-case speculation. Happily, reality tends to intrude even on the best-laid plans of mice and middlemen. There are two lights at the end of the tunnel, and they're not an oncoming car.

One roadblock is that the very existence of large numbers of Linux installations guarantees a market for non "secure" PC's. This means that suits, in collaboration with Microsoft and the various other usual robber barons of the PC world, cannot force the market by fiat to replace open platforms with "secured" ones. This still won't make HD DVD available on open platforms, but at least the rest of us will have a place to play and a platform for competitors to target.

The second roadblock is that HD DVD isn't quite the guaranteed winner that the suits are dreaming of. DVD was so wildly successful because it was simply a quantum leap from VHS - there is not a single way in which VHS is competitive with DVD. DVD's have a better picture and enormously better sound, are cheaper to produce, have smaller and more reliable media, allow easier navigation, can play in PC, sport nice friendly menus, and frequently ship with extra features and DVD-ROM goodies.

But when we go from DVD to HD DVD, the new format offers ... well, a better picture is about it. Given the current woeful penetration of HDTV's, this is not going to be a major selling point for some time. And even when (or if) the price of a decent HDTV drops below $1,500, the suits will need to wait until people retire their old TV's. DVD's didn't require retirement of old components to see benefits. To demonstrate an advantage, HD DVD's do.

The other problem is that people have consistently demonstrated that more is better: given the choice between higher quality and more, they'll take more - given the quality remains acceptable, and DVD certainly provides acceptable quality. So it's entirely possibly people will use HD DVD's as simply a larger DVD. Maybe two or three movies on a disc.

This more-is-better fact of human nature, plus the current high price of HDTV's and the inevitable initial high price of HD DVD units and movie titles, plus draconian DRM restrictions may doom HD DVD to stillbirth, or at best a minor niche in the cashed-up techno-geek market, similar laserdisc.

What's the alternative to next-generation uber DRM and pissed-off customers? Well, the suits could try something utterly crazy, like reduce the price of movie titles to the point where there's no point in anyone copying them. If HD DVD titles were, say, $10 I, for one, would never buy another blank disk again. When you take into account the amount of time and effort to make a copy plus the lack of cases and cover art, I'd pay $10 for the real thing over $1 for a bad copy any day.

The real problem is that the current usual DVD price of $25 is over the invisible line for which many people will make the effort to copy rather than buy. I buy maybe three DVD's a year simply because the ones I want to spend $25 on need to be guaranteed good, and the only ones in that category are ones I've already seen. Which means I only buy DVD's that are so good I'll see them 5+ times - otherwise it's just cheaper to hire them, which is my usual method of viewing. There are very few movies that I want to see 5 times, but there are quite a few that I'd like to see more than once, and which I'd pay $10 for. But, for most suits, I imagine a $10 standard price for a DVD is too scary to even consider.

Will HD DVD flop? Time will of course tell. In the meantime I, for one, won't be particularly worried if I have to stick with DVD's for while.

Further reading:

Next-Gen DVD Encryption: Better, but Won't Stop Filesharing

DVD Copy Protection: Take 2

MonoDevelop == Eclipse#? (plus why OSS pisses me off)

2005-03-20T16:28:00.000+10:30

Is it just me or does MonoDevelop look a hell of a lot like Eclipse?

Check out this screenshot of MonoDevelop in this article on its progress to date. Then compare this to this Eclipse screenshot (140K). Projects (sorry, "solutions" in Microsoft-speak) on the left, tasks at the bottom, source code on the right complete with closable tabs. In a few years time, they'll have written Eclipse#, probably complete with their own plugin system.

I realise Eclipse has borrowed a lot from other IDE's, but this is really too similar to credit, even down to copying the bad parts of Eclipse like the unscalable tabbed source viewer that becomes clumsy after about 6 tabs. I suppose they'll soon be "innovating" (Bill's favourite MS-speak) refactoring support.

This is getting off-topic, but Mono is also busy forking Java staples like Ant (NAnt) and JUnit (NUnit). Why exactly did the Mono/Novell guys decide to build on .NET when they could have taken on the years of work gone into OSS Java VM's like Kaffe and GNU Classpath?

The muti-language thing is a furphy - .NET runs any language that looks like C# (single-inheritance, single return value, statically-typed, etc) just like Java runs any Java-like language. They can both run other languages like Python, but they end up more as interpreters-on-a-VM, since neither natively supports Python-like dynamic types and multiple inheritance. Plus, regardless of language, you still get the same C# or Java-oriented system frameworks. You end up writing C# programs in Python (or VB.NET, or whatever) syntax.

The static compilation thing also holds no water: Java can be statically-compiled if you wish (gcj or Excelsior JET are two that I've used). Although, as I've argued before, this seems to be more to appease hard-core C addicts than to actually improve performance.

I'm going to veer even more off-topic and say that this is the thing that pisses me off about the OSS world at the moment: so much time is spent by tribes of geeks busy re-engineering existing stuff that very little that is truly innovative is appearing. Case in point: KDE and Gnome are both run-of-the-mill desktops, trying to match Windows. They're OK, but meanwhile Mac OS X still continues to blow me away.

The OSS world just seems to have zero big-picture vision at the moment, it appears to be just a bunch of tribes squabbling over which is The Way, KDE vs Gnome, Java vs .NET vs Python, RPM vs DEB, etc. They're not building new stuff, they're continually rebuilding old stuff, much of which is tools and frameworks for building things rather than the things themselves that would make enough difference to someone considering switching from boring old Windows. I can't think of a single thing about KDE, Gnome or Windows that would make a big difference if I was forced to use only one.

Each OSS project that essentially duplicates another is not progress, it's not healthy competition and it's not about helping choice: it's fracturing standards, duplicating effort and balkanising OSS.

Speed bump

2005-02-02T15:58:00.000+10:30

Shifting to a PowerBook for my home computing has underlined a something that I should have realised a while ago: it's time to rethink the obsessive focus on speed in computing. Case in point: I'm writing this on a computer with a "mere" 1.33 GHz CPU with what I recently heard someone call an "anemic" 167MHz bus connecting it to the rest of the system. This laptop has a relatively slow hard drive, memory and a "small" 1024x768 display. By the yardsticks of many hardware fetishists, this rates as a truly poky system speed-wise. And yet this is hands-down the smoothest and nicest-looking PC I've ever used. At work I have a 2.4GHz system with vastly faster components, and it somehow doesn't manage to feel any faster.

I realise I haven't been excited about computer hardware performance improvements in quite a while. This may be just a sign of getting jaded, but it wasn't always this way and I do believe a change is happening. My first PC was a 4.7MHz IBM XT clone. Then a 25 MHz 386 which felt like a supercomputer - this thing was at least 15 times the speed of the XT, try to remember when you last had that big an upgrade. Then a 66MHz 486 which was nice but not as huge as the previous jump.

After Uni when I started working, I got the chance to use one of the first 100MHz Pentium PC's on the site - once again I had a little supercomputer on my desk. Coupled with NT 4, which was an equally huge improvement over the squalid world of Windows 95, this just felt like a next generation system. But since then, speed has become less and less of a noticeable improvement (and newer versions of Windows have failed to improve much either, but that's another story).

This article points out a hard truth that hasn't been discussed much: CPU speed has hit a wall, one which is not likely to shift for a while. The article includes a graph of CPU speed over time, and shows that it was in 2003 that we hit the wall. In 2005 we should have 10GHz CPU's, in reality we may get a 3.7GHz one this year.

And what if we did have 10GHz CPU's? Maybe I'm just limited in imagination, but what would we actually do with them? Fancier graphics? This "anemic" Mac already has some serious graphics chrome thanks to the GPU. Faster compiles for us developers? It's been years since I had to wait for the compiler thanks to incremental compiles. Games? Get a PS2, for $250 it'll give you way more bang for buck than a tricked-out PC at ten times the price that draws more current than a toaster oven. There are always uses for more power in tasks requiring sheer calculating grunt e.g. audio compression, video/graphics editing etc, but these won't change the computing experience for most of us.

The good bit is, what gets me excited today (professionally anyway ;) is the software, not the hardware. Technologies built on platforms like Java, OS X, and next gen languages are evolving nicely. I just hope that the hardware speed wall won't impede the progress of these technologies.

Bad tools

2004-12-06T19:19:00.000+10:30

Not to seem like I'm down on Sun or anything, but I can only assume that they really want us to avoid using their command line tools for Java development. First it's the Java runtime complaining about a class not found when really the class is there, it's a class that it depends on that's missing. Then, while using "jarsigner" today I get this error:


[signjar] java.lang.InternalError: jzentry == 0,

[signjar] jzfile = 10414112,

[signjar] total = 1555,

[signjar] name = C:\development\invision.ui.web\clovis_view.jar,

[signjar] i = 1,

[signjar] message = couldn't read LOC header

[signjar] at java.util.zip.ZipFile$2.nextElement(ZipFile.java:321)

[signjar] at sun.security.tools.JarSigner.getManifestFile(JarSigner.java:985)

 ...

"Oh no, that looks serious, I have a corrupt JAR file" was my natural response. Rebuild JAR file. Same error. Turns out that, although jarsigner allows you to specifiy the same source and target file, it doesn't quite go as far as actually handling that and tries to read and write to the same JAR file - bad. A simple if (inputFile.equals (outputFile) <warn poor sod user I can't do that> would have saved me, and probably 5,000 other programmers 10 minutes of time.

Swing "Hi-Fi"

2004-12-02T10:29:00.000+10:30

This article from a Sun engineer working on Swing in JDK 1.6 ("Mustang") and the comments it inspired revisit some of the Swing issues I've banged on about before. Here's an excerpt:

In Tiger and Mantis, we have provided support for the Windows XP look and feel. The mechanism used to provide this support was based on reading the resources embedded in the XP styles files and using those icons and colors to render our Windows Look and Feel. Unfortunately, Microsoft has changed the Styles files in Longhorn so that the resources are no longer visible using the mechanism we used in Tiger and Mantis
[...]
We talked this over with engineers from Microsoft's graphics team, they suggested that the way to work with the Longhorn themes was to use the UxTheme API. So Leif Samuelsson (Swing Windows Look and Feel Author) and I prototyped a version of the Windows XP Look and Feel that uses the UxTheme engine to perform the rendering of Swing components.

And one of the responses:

I for one am not holding my breath. This very approach was suggested to Sun three years ago in bug #4587653, right down to the names of the API functions to use. It has even been submitted multiple times by independant posters. But it has consequently and repeatedly been turned down by reviewers in favor of the always-trailing-behind bitmap copycat approach.

Obviously he's been in the same situation I have: he's tracked problems, suggested solutions, voted for them, and then waited years for any acknowledgement that there's even a problem. And then the Sun guy actually seems to be expecting people to congratulate them on finally doing it the right way instead of the unsupported hack they're using right now. And we'll have to wait another JDK cycle (18 months) before we can use this.

This guy goes on to say:

Surely, no future builds of Mustang will look any more native than the current one before the font support is fixed, too. Don't bother suggesting any APIs for this, however. That should be considered redundant as the issue will probably be talked over with some Microsoft engineers already around the end of this decade.

Sounds bitter. I feel his pain ;)

Another interesting comment:

Natively themed drawing means that you're using the toolkit's functions to perform the painting, but without having any of its structures associated with the component. This still doesn't save you from the anomalies stemming from a mock user interface, but at least it looks the same. [...]

It is often portrayed as the only way to achieve platform-independant user interfaces (why anyone would want that is above me), but in reality it is more like writing your own filesystem on top of the disk driver.The real reason it is done is probably because the model of custom drawing fits better with the garbage collector in Java. You still have to close your files explicitly, but nobody is complaining about that, though.

The comparison with emulated Swing and wrapped native IO is telling. Anti SWT pundits often claim that disposing widgets is "non-OO" (whatever that means), but rarely realise there are quite a few real-world operations that unfortunately don't fit the GC model. If Sun had written it's own filesystem implementation to avoid having to do icky things like close () files, people would have agreed that was insane. When they decided to write an entire widget framework and graphics stack, the major part of a modern OS's interface with its user, somehow that didn't sound insane.

This article is particularly apt, coming as it does as the latest Swing Sightings article goes out. There are some truly excellent and sophisticated apps in that selection. And all the good examples in there contain workarounds for Swing's L&F issues. Either they use one of the extended L&F's, or they apply WinLAF-style tweakage. If the Sun guys hadn't painted Swing into such a corner, these sorts of uber-apps might have appeared 4 years ago and blown people away compared to the dull Windows apps of the time.

SWT and Swing: Clash of philosophies

2004-11-22T14:20:00.000+10:30

Most shoppers don't like to get in car that looks and smells terrible, even if it does have a nice engine.

-- From email discussing SWT vs Swing history

I’ve written about Swing vs SWT before, but for some reason I feel compelled to expand. This is probably because I needed to write a small Swing GUI last week and it once again prodded a sore point. The reason it’s a sore point is that, once upon a time, you wouldn’t have found a more ardent supporter of Swing than me. I started working with Swing in 1997, while it was in the development 0.x releases, and followed it as it went from com.sun.java.swing, tracked the arguments about whether it should be java.swing or javax.swing, and celebrated the final release as part of Java 2.

I know Swing inside and out. I’ve written big applications, and developed Swing application frameworks for esoteric things like integrating copy/paste/drag/drop between tree, table and list views, command integration for dynamically managing commands in menus, popups and toolbars. I know why you need to add components to JFrame.getContentPane (). I feel I can justly claim to know Swing.

And I went from being a Swing booster to a rebel.

Why? The conversion corresponded to a transition in my view on software, from focusing on technology to usability, from abstract to pragmatic when I first started to get feedback by people using my applications. I started to realize that excuses don’t cut it: if I want people to like and use my software then it needs to Just Work.

The software and frameworks I wrote back then were like Swing, big and intricate. Everything was abstracted to the nth degree. And they were so complex that, after a while, even I had trouble extending, debugging and using them. At the time I, arrogantly perhaps, assumed that the others having trouble with the software I wrote were just less smart and would “get it” eventually and realize the wisdom of my approach.

This is the reason why Swing failed to win the hearts and minds of developers: it was written by people with this mindset. It’s powerful and abstract. It has a complex inheritance hierarchy and an even more complex pluggable L&F system. And it’s hard to learn, hard to extend. It is consistent rather than intuitive. It obsessively uses models for everything, even buttons. And it tries to avoid any special cases so, for example, if you want a set of radio buttons to behave normally (ie have only one selected), you need to know to create a ButtonGroup and add the buttons to it, otherwise they'll just look like radio buttons. Because having radio buttons in the same panel automatically exclusive would be a special case and wouldn’t fit the clean Swing model.

And Swing often doesn't provide things that every GUI app is going to need: eg if you want to add a context menu then, up until Java 5.0, you had to create your own right-mouse-click listener and add the special right-click selection behavior expected in trees and tables yourself. This means that, until recently, every Swing programmer had to reinvent the same context menu logic, including having to ensure the menu doesn’t go off screen. Not to mention the lack of a font selection dialog box or spinner (introduced years late in 1.4).

Swing also rates platform integration as a very low priority. It even seems to go out of its way to make it harder to add platform-specific extensions: one of the things that finally convinced me to ditch Swing was when Sun gratuituously removed the last sane way to get a window handle on Windows in JDK 1.4.1. There was no technical reason, since I can see from the source code that it still uses the same way internally to get the handle. The alternative Sun provided, the JAWT library, was designed to allow native plugins and explicitly stops you getting a handle for anything but a Canvas, so I had to use a horrible hack and create an invisible Canvas in the frame and then walk up to the parent frame using native code. And after I got that working it turned out that the JRE didn’t actually put the JAWT DLL in the path, so I got linker errors when deploying: it was clear the Sun guys didn’t actually use any of this stuff themselves.

Many of the irritating things about Swing stem from the fact that it tries live in its own world: the lack of native anti-aliasing (ClearType or otherwise) stems from the fact that Java2D doesn’t integrate with the platform’s font renderer; the wrong default font probably stems from the fact that Java2D has it’s own TrueType renderer that doesn’t handle Tahoma properly; the sticky menus are due to the fact that it doesn’t support mouse capture; missing display effects like shadows, wrong borders because it’s just too hard to rewrite all this stuff. Collectively they induce a feeling of no confidence.

This seems to me to be a case of some overly zealous Win32 API programmers thinking they could do things a better way. Even though Eclipse looks responsive, IDEA has proven that you can do the same thing with Swing.

I just read that IDEA has anti-aliased fonts using Java2D layer, can you get that with Eclipse?

-- JavaLobby Swing-booster talking about SWT

Here’s the thing that differentiates the new me from the old me, and me from the guy quoted above: I now realize that the people using my software don’t care that I can do anti-aliased fonts. In fact, if my app is using anti-aliased fonts when none of the other apps are, then they may complain that it looks funny. They do care if they’re used to ClearType and the application doesn’t use it, and Swing apps are almost unique on Windows in that they end-run around ClearType: even my old NTEmacs, compiled 2 years before ClearType even existed, happily uses it. Before supporting anti-aliased fonts, Swing needs to get the basics right and do things like actually use the correct default system font on Windows.

Users care about things like menus not going away when they click outside them, or that tooltips appear in all sorts of odd places. They may care that the app uses several MB more of memory. They may not consciously notice that Swing widgets look subtly wrong (eg see WinLAF) and don't support the same shadows, animations, etc as the other apps, but they'll likely feel that your app looks a bit naff regardless. They definitely care that the File Open dialog doesn’t support their Favorites links, or their context menus, thumbnail image view or D&D from the desktop. They will also likely feel that the UI feels slow, even though Swing is way faster than it started out. I still don't why that is, but I experience it myself and I'm convinced it's real.

Programmers on the other hand care that laying out GUI’s with SWT's GridLayout is much easier by hand than GridbagLayout with its 8 or so parameters per layout info object.

Here’s a response further down:

> I don't know about the IDEA hacks, but the standard
> Swing in J2SE still doesn't use the font antialiasing
> feature of Java2D, after so many years..
>

Not true.

Try this to create an antialiased label:
_perspectiveHeaderLabel= new JLabel() {

 protected void paintComponent(Graphics g)

 {

   ((Graphics2D) g).setRenderingHint(

   RenderingHints.KEY_TEXT_ANTIALIASING,

   RenderingHints.VALUE_TEXT_ANTIALIAS_ON);



   super.paintComponent(g);

 }

};
-- From JavaLobby

Okay, so I just need to subclass every widget then I can enable anti-aliasing (not sure what you do if you use 3rd party components). So this isn’t useful, but that apparently isn’t what the author thinks is the point.

SWT takes a “make it as simple as possible, but no simpler” approach that I would once have derided. It is simpler than Swing. It is not simpler than it should be. It hits a sweet spot that Swing abandoned in its focus on software engineering geekery.

SWT doesn’t have pluggable table cell renderers or buttons you can put animated graphics on. It doesn’t have three ways to create a window, JFrame, JWindow, and JDialog: it has a Shell. Instead of 20-odd methods with 5+ arguments each on the strangely-named JOptionPane, it has MessageBox, which 95% of the time does what you need. MessageBox doesn't do HTML text, but instead it gets the basics right by making the correct alert sounds and wrapping the message text properly.

SWT has Table and Tree classes that look and feel native (because they are) down to the expansion animation and smooth scrolling. It doesn’t support pluggable renderers, but supports straightforward customization of text, icon and tooltip, which is all the majority apps usually need, without forcing you to deal with the horrors of Swing’s Component-based cell renderer approach.

In short, SWT tries to Get Out Of The Way (tm) and Just Work in the 95% case. And for that 5% case where you need animated graphics in an auto-scrolling TreeTable view: write your own or use one of the custom widgets that SWT already supports. Custom advanced widgets are just as straightforward in SWT as Swing (see some Azureus screenshots for an example). People who don't use SWT often don't realise that it's not limited to native widgets, and in fact has several "emulated" widgets in its default toolkit (eg CLabel and StyledText), it just defaults to native whenever possible rather than rebuilding the world.

OK, so now I feel I’ve got this off my chest. I bought into Swing in a big way and, after years of defending it and believing that Sun would live up to the expectations I had, it was a big deal to admit that I was wrong. Thankfully the designers of SWT seem to have the same philosophy as I do, and I don’t have to trust they’ll get it right eventually: SWT has already got it right enough, and now I can get on with writing the software.

First, do not suck

2004-10-26T15:47:00.000+09:30

Have to get this rant off my chest: I am so sick of software sucking. Today was an extreme example, but every day I use computers there's several examples of self-imposed software stupidity driving up the blood pressure.

So I propose a new maxim for software designers based on a maxim from medicine, "First, do no harm":

First, do not suck

So: do not tell me one thing is wrong when in fact something totally other is. Do not randomly do stupid things that I can't stop you doing. Do not assume anything that I can't correct you about.

The highlights of the random screwups so far for today:

Both my monitors randomly refresh occasionally, setting icons blinking, windows flashing
Saving a file in Acrobat sometimes burns 100% CPU for no apparent reason. Cursor still blinks in Save As dialog, but dialog is dead. Comes back after about a minute
Acrobat informed me that the disk for temporary storage was full (a mere 5GB free), when in fact the file name was too long.
Java app using XML parser works on one PC with JDK 1.4.2 but fails to find SAX parser on identical machine with identical JDK.
Letters take half a second to appears when typing a filename into the Save As dialog when pointing to a network drive (presumably due to the dialog trying to auto-complete the name)
Open an image by double clicking. Try to compare it with another image by double-clicking - replaces first image. No apparent way to turn this off.
Text an Excel spreadsheet that was displaying fine suddenly turns into a bunch of "######"'s. Checked that format of cell was "Text". Only by changing format to "General" did I manage to fix it
Word pops up irritating panel whenever I copy more than one piece of text - take a few minutes to work out how to blow that away. It then insists on selecting whole paragraph randomly when I want to select a piece. Wade through checkboxes in options to turn that off
While editing this post, discover that Ctrl+S publishes - I was trying for Ctrl-A to select all. Why do I need a shortcut for publishing? It's hardly something I do every few minutes.
An endless number of apparently temporary user account profiles are appearing in my "Documents and Settings" folder. They're using several gigabytes of disk, but can't be deleted because something is using the files. No idea what that something is though.

Mac'ified

2004-10-20T10:10:00.000+09:30

Now that I finally gave into temptation and bought a 12" PowerBook and have been using it for a week, I can happily say that I'm actually more happy with it than I though I would be (and I was pretty gung ho in the first place). And for someone who's in the past looked at Mac's as the Volvo of computers, that's a happy thing to be able to say.

So many people have waxed lyrical about their conversion to Mac-land, and I don't intend to add to that, not least of all because I'm still using XP as my primary desktop OS so haven't really "converted" at all. And there's that horrible taint of the really religious Mac zealots who assume anyone willingly using anything else must be retarded or unfit to live, or at the very least in need of some pitying condescension.

However, I do see why people have such regard for the platform, because every little thing on the Mac feels like someone spent weeks obsessing over just the right way to work, look, feel and act. Everything indeed seems to Just Work. And not only just work, but give you that nice fuzzy feeling of working so well that you're sure you'll discover even more stuff later.

Things to like:

Fonts. The Mac's fonts are truly a work of art. Even on a relatively low-res 1024 x 768 screen they look gorgeous even at 10pt. I thought I would miss XP's ClearType, and indeed the Mac's fonts do look a little blurred compared to XP's, but they are also far more elegantly shaped and don't have the vaguely irritating red tinge around the edges that ClearType generates. Even the Windows fonts that Office installs look better on the Mac than on Windows.
Exposé. Let's not mince words: Exposé. Is. Fucking. Fantastic. I used to think it was a neat trick, now it's my main method of navigating through windows. I can slam the mouse up to the left corner to show all, hit F11 to show the desktop, or F10 to show what windows an app has open atm. The dock has simply become a way to launch apps, not to navigate them.
Consistent key shortcuts. On a laptop these are a godsend. Close a window option-W, kill app option-Q, next app window, option-left, prefs option-,. Yes Windows apps often have equivalents, but not always, especially when flipping between app windows.
Security. When apps need to do something that needs root, the system pops up a dialog and asks for a password - you don't need to take the risk of running with admin privileges all the time. There is a protected system-wide "keychain" for storing passwords. You can FileVault-encrypt your home directory, making me much happier about letting the system store passwords and other info like email that I'd hate anyone to get if the thing gets stolen.
Safari. I thought I'd use Firefox, but Safari is cool enough that I haven't yet. The only thing I miss are extensions like Adblock.
It just looks cool :) Everything in the UI is antialiased and looks like it's made by skilled glassblowers :) Things slide around rather than snap about as they do on XP. You get the feeling that the platform is so powerful underneath that cools things are easier to do than boring ones.
It runs Eclipse very nicely.

Indigo: WS-TheEmperorHasNoClothes

2004-10-07T16:33:00.000+09:30

From MSDN Indigo FAQ (emphasis mine):

J2EE is deeply rooted in a classic distributed object architecture (based on EJBs)—an architecture that has proven to be complex and brittle for Internet and enterprise-scale business integration scenarios. Service-oriented development is a fundamentally better way for building connected systems. But with its distributed object architecture, J2EE is not designed for service-oriented development. Indigo builds on .NET's leading-edge support for Web services, and provides a complete service-oriented programming model and integrated communications infrastructure for building and running connected systems in a productive and reliable way.

From MSDN article on developing with Indigo:

With this [stateful web service] model, private variables can be persisted and properties can be employed. Essentially, true object oriented programming is brought to Web services. Those who have used client-activated objects in .NET Remoting will find this familiar; however, this is completely novel for those used to ASP.NET Web services.

Que? On the one hand OO systems like J2EE Enterprise Java are "brittle", and on the other "true" OO programming is a good thing for developers of web services. They seem to be implying developers can have their OO cake and still adhere to Indigo's avowed goals to separate service contracts and semantics from the underlying object model. I can't see how this can happen if they're encouraging the OO mentality to persist. I also find it a little bemusing that they're actively promoting one of the key things that WS advocates abhor: stateful services.

This sort of crap is liberally sprinkled throughout MS's descriptions of why Indigo and web services are so superior to the previous One True Way of building services. They also only use J2EE as a punching bag, neglecting to note that apparently they've realised DCOM now also sucks.

While I'm certainly not the only one wondering whether the emperor has no clothes, I have to wonder whether even some of the senior, non PHB people at MS are having trouble working out why exactly they're shifting from RPC via DCOM to RPC over XML.

Sigh. Swing still broken in Java 5.0

2004-10-06T18:18:00.000+09:30

I'm happy to say that this doesn't worry me much any more since I've switched to SWT for GUI development but, unbelievably, it appears Sun still haven't fixed any of the outstanding stupidities in Swing's look and feel emulation in Java 5.0. It's been seven years since the Swing project started.

For example, this image shows font glyphs that are still broken in 5.0 on Windows:

This image (via this blog) shows the glitches becoming visible in the UI when using JFileChooser. And, according to the entry, all the usual missing borders, spacing issues, etc that WinLAF fixes are still in 5.0. And this bug (Java developer site login reqd unfortunately) describes the issues in all their ugliness.

Now these issues individually are irritating, but when they're all combined they become totally frustrating to a programmer who's spent so much effort making a good GUI, only to have it suck in ways they can do nothing about. And it only confirms the sentiments of the anti-Swing bigots to see Swing apps still look as crappy in 2004 as they did years ago.

I have to wonder, as the poster notes, if the Swing guys actually ever use their own stuff on Windows. I have noticed in the past that a lot of screenshots in articles by the Swing team show the CDE (Motif) L&F, which I'm sure is emulated perfectly in all its ugliness.

I love (to chuckle at) Win32

2004-09-29T10:02:00.000+09:30

This is a work of beauty (taken from this article by Raymond Chen). The code below is handling popup menu selections in order to show help text in the status bar (actually the window title in the example).

void OnMenuSelect(HWND hwnd, HMENU hmenu,

                int item, HMENU hmenuPopup, UINT flags)

{

 if (g_pcm && item >= SCRATCH_QCM_FIRST &&

      item <= SCRATCH_QCM_LAST)

 {

   WCHAR szBuf[MAX_PATH];



   if (FAILED(IContextMenu_GetCommandString(g_pcm,

         item - SCRATCH_QCM_FIRST,

         GCS_HELPTEXTW, NULL, szBuf, MAX_PATH)))

   {

      lstrcpynW(szBuf, L"No help available.", MAX_PATH);

   }

   SetWindowTextW(hwnd, szBuf);

 }

}

Ah, so wonderfully readable and obvious (see Raymond's post for more crunchy C/Win32 goodness). No wonder people actually like VB given that this sort of madness is the alternative. And this is from article 6 in a series of how to manage popup menus in Windows. Eeeek!

Nice!

2004-09-28T11:27:00.000+09:30

This language called Nice is one of the few new languages I've seen recently that really looks like next step on from Java. It adds a lot of expressive, functional capabilities that I've wanted for a while, while not gratuitously inventing its own alien syntax.

One of the really neat things are multi-methods, which might sound dull, but actually address a real problem I've had in Java for a while: how to implement extensible "utility" methods that aren't logically part of a class. For exanple in Java you might do this (taken from this tutorial wiki):

package my.package;



import geometry.*;



class Tools

{

  static boolean overlap(Shape s1, Shape s2)

  {

    if (s1 instanceof Rectangle && s2 instanceof Rectangle)

    {

       ...

       // return the result in this case

    }

    throw new Error("Case not supported: " + s1.getClass() + " and " + s2,getClass());

  }

}

But in Nice you can do this:

package my.package;



import geometry;



boolean overlap(Shape s1, Shape s2);



overlap(Rectangle s1, Rectangle s2)

{

  // Here s1 and s2 are both rectangles. We don't need to use instanceof or cast.

  ...

  // return the result

}

Which is far neater and extensible: you can add new sorts of Shape and new versions of overlap (), which wouldn't be possible in the Java version.

Java generics

2004-09-28T10:21:00.000+09:30

This article by one of the sw engineers at Google about the reasons behind the decision to use the "erasure" approach for Java generics is very enlightening. The purist in me derides the limitations imposed by the approach, which are mainly due to the fact that the generic type parameters are not available at runtime, only at compile-time.

The realist in me understands that, although pure generics might be prettier, the massive Java codebase demands a way to migrate smoothly between non-generic and generic worlds, what he terms "Migration Compatibility". This allows a company to start shipping a generified version of their library that is fully forwards and backwards compatible with both environments, thus not forcing clients one step up the chain to go all generic or all non-generic - they can migrate as they write new code and leave old code intact.

Update: this article by Bruce Eckel responds to these arguments. Bear in mind that Bruce is from the dynamic typing camp.

More API craziness

2004-09-08T10:17:00.000+09:30

This is how you make a String from an array of bytes in .NET:

System.Text.Encoding encoding = System.Text.Encoding.UTF8; return encoding.GetString (buffer);

This is how you do it in Java:

return new String (buffer, "UTF-8");

If you have an array of bytes that you want to stringify, then you'll likely go straight to the String class to see what kinds of things it can do with byte arrays. In .NET you need to find your way to System.Text.Encoding and then work it out from there.

(I really, really dislike .NET's "always capitalise everything whether it's a package, class, constant or variable" naming convention. What a waste of an alphabet).

More on dynamically typed languages

2004-09-08T10:01:00.000+09:30

Check this Python API document for a mail-processing library:

class UnixMailbox( fp[, factory]) Access to a classic Unix-style mailbox, where all messages are contained in a single file and separated by "From "(a.k.a. "From_") lines. The file object fp points to the mailbox file. The optional factory parameter is a callable that should create new message objects. factory is called with one argument, fp by the next() method of the mailbox object. The default is the rfc822.Message class (see the rfc822 module - and the note below).

This is a classic example of why dynamic typing makes it harder to understand an API. So you need a "fp" to create a UnixMailbox instance. Que? "The file object fp points to the mailbox file" - oh, I see, that makes it much clearer. Now I can go and grep for "file" and see that class might actually do the job here. Is it a file, filename, something that implements open () and read (), any of the above?

With a statically typed language it would be "class UnixMailbox (File file)", with a link to the File class so I can work out how I can make one that fits.

Static vs Runtime Typing

2004-08-18T11:22:00.000+09:30

In this post Bruce Eckel, who's written a lot of wisdom on software development, describes how he went from advocating C++-style static typing to Python-style dynamic "implicit" typing. What he means by this is the removal of things like Java interfaces and classes when describing an object. For example, a Python method to calculate a factorial takes an argument "x", but doesn't specify what x is: integer, floating point, BankAccount?

  def factorial(x):

   if x == 0:

       return 1

   else:

       return x * factorial(x-1)

Bruce argues that static typing makes things harder for developers:

The extra verbiage that must be added in order to make the compiler happy doesn't really accomplish that much and has a severe impact on both productivity and code maintenance. We have to write unit tests anyway, so the illusion of safety comes at far too high a cost. Ned Batchelder put it another way: "Static typing prevents certain kinds of failures. Unfortunately, it also prevents certain kinds of successes."

The interesting thing here is that Bruce seems to be going further than many dynamic language advocates, who often say that dynamic programming is better because it lets you get small things done quicker without the syntactic boilerplate. He's claiming that since we're going to have 100% test coverage anyway, why pay the price of static typing in order to catch things like missing methods, etc? A number of responses to this come to mind from day-to-day development experiences.

Point 1: the testing requirement may be too onerous for a medium-size project, where 100% test coverage might be infeasible. For example, the 14KLOC GUI messaging application I'm developing would need a very large number of tests to cover all the code paths. Apart from the perennial problems involved in testing a GUI's and network communication, I simply don't wish to pay the price of 100% test coverage. I have tests for the core components, and test the rest by simply using the application day-to-day. Static typing ensures there are no trivial faults like typo's in method calls, the basic unit tests ensures there are no serious faults in the core components, and day-to-day usage exercises the rest.

Point 2: by not stating type info, you reduce the information available to readers of a program and to IDE's helping you develop the program. For example, say you come across some code that contains a "toolbar" object that seems to have an "add (item)" method. What sorts of objects can I add () and what behavioural contract do those objects need to satisfy? If there's type info like "Toolbar.add (ToolItem item)", then I (and an IDE) can immediately see what behaviour an item on the toolbar needs to provide and what classes provide it. I'd rate this as such important information that I'd argue that the lack of type information would have an impact on code maintenance, not the other way around (as Bruce claims in the article).

Point 3: dynamic typing excludes conventional optimization options. For my messaging application I can use a static compiler (Excelsior JET) that generates a highly optimized Windows .EXE from my Java code. A dynamic language does not currently have this option.

Point 4: when you need some some dynamic programming in Java, you can get it. Reflection, although it's messier than Python's elegant syntax, gives you the ability to ignore type when you need it. For example, a common GUI paradigm is the Command pattern: a command object packages the functionality of an application command (eg "Edit -> Cut") and is embedded into toolbars, main menu, context menu etc which invoke its execute () method when the user clicks the various items. A key issue with this pattern is that many commands, such as "Cut", are context-sensitive: the Cut command obviously needs to know what to cut.

This "object-to-operate-on" property of a command is very often bound to the selected item in a view such as a list or a tree, and it would be nice if we could have a simple way to just bind a command's property to the selected item in a view instead of writing endless tiny event handlers. So I have a class that listens to selection changes on any GUI viewer control and feeds the selected item(s) to a given property X of an object (usually a command), which is done by calling its setX () method. The type of the target object is irrelevant, as long as it has a setX () method. In fact this binding approach ends up being more flexible than the equivalent in Python because the binding object can find the type of X and automatically decide whether the currently selected item is compatible: if not it does not proceed with the binding and instead disables the command.

Funny insight on religion in Slashdot

2004-08-11T10:18:00.000+09:30

I just had to post this excerpt of a quote on Slashdot from yesterday which arose, of all places, in the middle of a Q&A session with Larry Wall (of Perl fame). He had a quite odd description of how he reconciled his Christian beliefs in god with actual religion (IMHO).

And then, as is standard practice on Slashdot, things went off on a tangent and resulted in the following quote that I think actually deserved its "Insightful" moderation:

(Slashdot 10/08/2004: http://interviews.slashdot.org/comments.pl?sid=39406)

Then again, my two favourite books in the Bible are Ecclesiastes and
Job. Both of which...

So have fun, be smart and if there is a God - let's hope this is some sort of practical joke. Making us in the form of giant hairless monkeys is rather amusing when you think on it.

...provide a sound Scriptural basis for your conclusion. So welcome to the True Faith, Brother :-)

On a more enlightening angle, allow me to quote Carl Sagan (atheist):
How is it that hardly any major religion has looked at science and concluded, 'This is better than we thought! The Universe is much bigger than our prophets said, grander, more subtle, more elegant'? Instead they say, 'No, no, no! My god is a little god, and I want him to stay that way.' A religion, old or new, that stressed the magnificence of the Universe as revealed by modern science might be able to draw forth reserves of reverence and awe hardly tapped by the conventional faiths."

- Carl Sagan
The God worshipped by Islamokazi nutbags and Creationist fundies is of little interest in me. He may have created us in His image - but some of his less-than-clued followers have unfortunately returned the favour :(

The Islamokazis (and to a lesser extent the Roman Catholic Church) re-created him in the form of a celestial slot machine ("insert [ 767 | money ], pull lever, get [ 72 virgins | indulgences ]"), and the Christian fundie hucksters re-created him in the form of a carnival barker ("I whipped it up in a week six thousand years ago and hid different ratios of potassium and argon isotopes in the dinosaur bones just to confuse their scientists 6000 years later! Suckerrrrz!").

But a God who can come up with an entire universe based on a few fundamental constants and some deep mathematics, such that out of that universe, a few bits of carbon compounds might emerge into sentient life capable of looking around at the universe and trying to unravel the math for themselves... That's the kind of God that might be worth getting to know more about.

Unsigned integer types

2004-08-05T10:15:00.000+09:30

OK, so I've been programming in Java now since the 1.0 days and never needed to worry about the lack of unsigned types. Java kind of sneakily works its way around the issue when they are needed by just using the "next size up": for example the call to read a byte to a stream, InputStream.read (), returns an int rather than a byte. I used to wonder why until I realised this is an unsigned byte, and using byte would break for values > 127. Another example is the java.util.zip.CRC32 class that outputs a long (64 bits), even though you might expect an int (32 bits).

A lot of the time this trick works, but a real problem occurs when you want to promote, say, an "unsigned" byte to an unsigned int. That is, a byte that you are treating as unsigned (eg one read from an input stream into a byte array) - it's only a bit pattern after all and most of the bitwise operators don't care about the sign. If you just do something like:

byte [] bytes = new byte [10];
input.read (bytes);
int uint = bytes [0];

Then you'll find that, for negative values, Java helpfully "extends" the negative bit on assignment, so for example if bytes [0] had the hex value FF (binary 11111111), then uint ends up as hex 807F (binary 1000000001111111). Which may cause some problems, especially since things still work for values less than 128.

The way to work around this? The code below promote an "unsigned" byte to an "unsigned" int (there may be a more elegant way to do this, but I know this works):

public static int promote (byte value)

{

 if ((value & (byte)0x80) != 0)

 {

   // create int without sign extension and then re-add high bit

   int uintValue = (value &amp;amp; (byte)0x7F);

   uintValue |= 0x80;



   return uintValue;

 } else

 {

   return value;

 }

}

Even though this is a PITA and a trap for new players, I can understand why the Java designers might have decided to leave unsigned types out. For a start, you immediately increase complexity a lot: you double the number of primitives and need to define how to handle things like assigning a signed to an unsigned safely. The current types entirely enclose the smaller types (a long can handle any int, an int can handle any short etc). But unsigned types overlap but don't enclose. Also, the JVM encodes the type of operand into bytecode instructions, and there are limited number of those (256 IIRC).

Windows bug o' the day

2004-07-30T18:40:00.000+09:30

Don't try this at home: select a bunch of folders in Explorer and select "Search...". Rather than bringing up a search window that searches the folders, it will slowly bring up a bunch of the buggers (one per folder) and then, in my case, crash hard.

BTW: one of the most irritating things about explorer.exe crashing is that many taskbar icons vanish when it restarts. Fascinatingly, although MS provides an API for apps to use to restore icons on a crash, MS's own Task Manager doesn't actually implement it.

Insanely bad Outlook 2003 search folders

2004-07-19T10:23:00.000+09:30

I just took the leap from Outlook 2000 to Outlook 2003. Gratifyingly, the upgrade was flawless (so far) and pulled all my settings across, even including the SpamBayes plugin which has continued doggedly making spam a dim memory.

However, one of the features that I thought I would like most about 2003, Search Folders, appears to have been implemented in an afternoon by an undergrad arts major. The default views such as "Unread Mail" are of course useless for anyone that filters mail, especially since a lot of unread mail is spam. And you can't customize the criteria for inbuilt views: the button is simply greyed out. It's also hard to restrict the folders you want to scan, since you can either select "recursive" mode for all folders or for none, and then select everything manually, a real problem for people like me that have about 20 folders.

To make it worse, once you've worked out that you need to create your own search folder you can only specify AND for combining criteria. I want to show unread OR flagged mail, which doesn't seem to be supported even in the advanced area (which doesn't actually have an "Unread" field to select anyway, even though other parts of the UI display it). There is a "Unread or For Followup" inbuilt view, but it shows nothing and I can't see what criteria it uses because, surprise surprise, the button is greyed out.

And then, in case I was still in the mood for self-flagellation, it turns out that Outlook 2003 has some insane number of different coloured followup flags you can use, and I cannot match any of them to the ones already in use in my inbox (there's no "faded red" flag icon in the menu, just red and orange).

Since I'd consider myself an expert Outlook user and in a good position to evaluate this product, I hereby award Microsoft -3 out of 10 for usability.

PS: on further investigation ... it gets worse. Trying to sort by unread and then by ascending date just gets ignored - the ascending date part is not there when you go back to "Customize View" no matter how hard you swear at the dialog box. Even better, selecting the "Unread Messages in This Folder" view while in a search folder produces a broken blank message list. Glad this product has shaded buttons, 20 flavours of followup flag and alpha-blended fading mail alerts, or I'd feel ripped off.

C# and virtual methods

2004-07-16T11:09:00.000+09:30

Continuing my ramblings on C#'s default to non-polymorphic (non-virtual) methods, I've been revisiting an excellent series of interviews at artima.com with Anders Hejlsberg, daddy of C#. One quote is of particular interest:

Bill Venners: In Java, instance methods are virtual by default—they can be overridden in subclasses unless they are explicitly declared final. In C#, by contrast, instance methods are non-virtual by default. To make a method virtual, the programmer must explicitly declare it virtual. Why is non-virtual the default in C#?

Anders Hejlsberg: There are several reasons. One is performance. We can observe that as people write code in Java, they forget to mark their methods final. Therefore, those methods are virtual. Because they're virtual, they don't perform as well. There's just performance overhead associated with being a virtual method. That's one issue.

This is a very telling statement because (a) it's wrong - it's not a performance issue in Java and (b) because it's not consistent with another key C# design decision. Java classes are JIT'ed at runtime when information about subclassing and method overrides is known, so that JIT can easily auto-"final" methods and classes. The only reason for the programmer to use final is to ensure safety API-wise.

The inconsistency is that, if methods are "sealed" by default to make API contracts safer, why aren't classes also sealed by default? I think the real reason is that C# required sealed methods in order to not suck when linking against precompiled classes (eg the entire .NET standard framework), but sealed classes wouldn't have made much difference. If Anders actually believed what he was saying about sealed methods in API contracts, then it would apply to classes too.

I think the real reason for these design decisions in C# is that this is the same default as C++, a language whose author unabashedly admits major design decisions were driven by performance, not safety.

Blogs adding to knowledge fragmentation?

2004-07-14T11:53:00.000+09:30

Continuing my rambles on RSS, I've been vaguely worried recently that RSS-based weblogs add yet another level of fragmentation to the information store that is the internet.

Once upon a time there was Usenet, which was one of my primary knowledge resources. The advent of Deja News (now Google Groups) was a big leap in being able to tap into other people's expertise. Now I monitor mailing lists, RSS feeds, web-based forums not to mention instant messaging logs. It worries me that I often search Google Groups and get invaluable information, yet rarely contribute because of the low signal-to-noise ratio. How do I search the information sources that I now contribute to?

RSS and webcasting

2004-07-14T10:38:00.000+09:30

Now I know why I started to feel a little deja vu coming on when I finally twigged onto RSS's rise to prominience. It's webcasting come back to haunt us.

Some of us remember the non-event that was webcasting back in the heady days of the Netscape/Internet Explorer Wars. Netscape had Netcaster and Microsoft had Active Channels (there was so many "Active" products from Microsoft in those days that I initially thought "ActiveX" was a joke based on their naming scheme). Everyone was very excited about who would win the war (they both had incompatible channel formats of course), and there was much serious discussion about how terrible it was that news sites had to support both.

And then it all just faded away, probably because it was a classic solution-looking-for-a-problem technology that no one actually wanted. And also because there wasn't much interesting content: it was designed around the idea that big news sites would use it rather than as a grassroots tech.

Now we have the grassroots rise of RSS, which provides 90% of webacasting and which people actually want and use.

Java vs .NET runtime optimization

2004-07-09T17:05:00.000+09:30

This article is yet another article showing Java beating C/C++ in several benchmarks. There are quite a few of these around, and whether you have problems with their methodology or not, I think they provide cumulative evidence using C++ for performance alone is pointless. I think it's likely that the only people remaining in the "Java must be slow" camp don't actually use it.

Interestingly it contains a discussion on why Java might be able to beat C++. There seem to be two main reasons why:

The Java JIT can make global optimizations at run time that aren't possible at compile time: this includes system classes.

The removal of arbitrary pointer manipulation allows not only safe GC, but also safe inferences about aliasing.

Some of real wins come when the JIT can de-virtualize and inline methods. For example, although any method on HashMap might be overridden by a subclass, 99% of the apps out there won't subclass it. For those apps, methods like HashMap.isEmpty () can be inlined and reduced from a virtual method call to three bytecode instructions (and probably two in machine code).

In fact I believe the main reason why C# defaults to non-virtual methods is because of the CLR's use of precompilation, which means that having every method virtual would be a big performance hit. So in the CLR, de-virtualization effectively needs to be done by hand. C#'s designer makes a virtue out of a necessity by claiming this is a better idea anyway: you should design for overridable methods. While I can agree with that, there's no reason not to use virtual by default and use "final" when you really don't want to have someone override a method.

MS actually recommends you do not use static compilation in this MSDN article, making the same claim that the JIT can produce much better performance. So why do they precompile the CLR framework classes and remove the opportunity for global optimizations?

Development Platform Franca

2004-07-02T08:53:00.000+09:30

All the discussion on Slashdot today about the release of Mono 1.0 (people variously asserting that Mono is the devil, Mono is crucial to Linux, Java is better, nyah, nyah, your father smells of elderberries) got me thinking again about what the real values of Java as a platform are. As a language it's fine, but other OO languages are good too. The API's are fine, plenty to choose from but that's not it it either.

It has struck me more than ever over the last couple of years or so while Java has really matured that its real value is as a lingua franca development platform, not a language or a VM. The platform developers agree on language (Java), API's (XML, net, logging, threading, etc), namespace management, API documentation (JavaDoc), build systems (Ant and Maven, standard build layout in src, lib, classes, etc) deployment (especially for web deployment) and even IDE's to some extent (a number of projects I received recently came with Eclipse project files). You can take big web app developed in SunOne and have a very good chance of running it in WebSphere or Tomcat/JBoss. I've developed my own web apps in Tomcat on Linux and then dropped them into WebSphere on Windows without issues (this ability to move apps around must give Bill Gates the shivers).

This standardisation allows me to grab some fairly complex libaries and frameworks and just use them, without worrying about what hokey build system, source code layout, language, memory manager, collections framework, homegrown logging system or dirty preprocessor tricks it uses.

The language part of the Java platform is the smallest part. It's a standard: I don't particularly care what the standard is, just that it's good enough and that everybody agrees with it. The common language is important across projects too: although I can use a library in .NET that's written in Eiffel# I'm likely to have some problems if I need to dive into the source to work out what I'm doing wrong. The language-geek dicussions about generics, value types, checked exceptions, etc are, while interesting, just not germane: none of these are features that would influence your choice of platform. The key choice is the environment: platform API's and runtime containers.

Java 5.0?

2004-06-29T17:11:00.000+09:30

Sun appear to be renaming what used to be JDK 1.5.0 to J2SE 5.0 (link). Boy is this getting messy. I think they should just separate the branding part from the version part and call it Java 3. This makes more sense since they're almost certainly going to have to leave the internal version number at 1.5.0 anyway for compatibility.

I wonder whether Microsoft will come up with .NET 6.0 next? ;)

Most convincing argument against DRM

2004-06-29T09:44:00.000+09:30

I've been against DRM for the reasons that many people probably are, mainly that it's not futureproof and I don't want to choose a closed format (for example all my music is in Vorbis format). But this article from Cory Doctorow (author of one of the best blogs I read regularly: Boing Boing) is an utterly convincing argument that it doesn't matter whether you're pro or anti DRM, it isn't a viable proposition in any case.

I've always held that anything you can play, you can copy. The author underscores this by pointing out that DRM has an intrinsic conceptual flaw: it hinges on encryption, which in order to work requires you to keep secret one or more of the following: (1) the encrypted content itself, (2) the encryption algorithm or (3) the encryption key. But in DRM systems like iTunes FairPlay the publisher must give you all 3: the encrypted music, algorithm and the key.

Most, if not all, of the DRM cracks I've seen (Hymn for iTunes, DeCSS for DVD) involve simply working out what the secret key must be, since it has to be in the system that's playing it (based on the computer's hardware signature for Hymn, embedded in the software of a DVD player for DeCSS).

There are a number of even better, non-technical, arguments against DRM in this well-written essay. Well worth a read.