Goodbye dmr

Like so many others in the world of software, I was greatly saddened yesterday by the news of Dennis Ritchie's death. We have lost a great contributor to our world and a really fine man. I first read K&R in 1980—a free copy included with my first Unix licence—and it changed my life. The book and the C language both shared many virtues which are so widely-known that I won't repeat that story now.

My first direct connection with dmr was an email he wrote to me back in the days of Usenet, something he avoided. But somebody had alerted him to a piece I had written and he thought it would be good to disabuse me of a misunderstanding. His email to me was as clear and spare as the book and his tone was perfect—polite, respectful, generous and kind. Naturally, I corrected my error in the forum where I had first written.

A couple of years later, after K&R2 had been published, I pontificated about the correct way to do something in C and was shouted down by people who were conversant with the new edition—a book I owned but had not read carefully at that time. I said K&R was wrong if they disagreed with me, expecting to find that those people had mis-read the book. Subsequent research revealed to me that K&R did put things differently from my understanding. So I wrote to Dennis and asked him to help out. Initially he said he thought the book was right. But when I wrote back with a detailed explanation for my belief, he promptly replied that I was right and the book was wrong. This resulted in an extra item in the still short errata for K&R2. What impressed me most in that exchange of email was that dmr never acted as if he had all the answers, but was ready to listen carefully to anybody who had something to say. I have tried to learn that from him.

Later I had the opportunity to meet him in person at a conference where he gave a keynote about an early release of Plan 9. He wore his hat with a propeller on top and gave a great talk. In person, dmr is just as nice as I had expected from my other interactions with him. We had occasion to correspond about C and its directions several times after that and I'm very grateful for his thoughtful and amusing emails.

Although this is really about Dennis, it seems wrong not to also mention his collaborator on K&R. I never met Brian in person, but have had email correspondence with him too. He showed the same wonderful manner of communication as did Dennis. I think of them both as mentors.

So, thank you, Dennis Ritchie. You have made a powerful and admirable mark upon our world.

Test Driven Freedom

Like most programmers, I've been writing bits of test code all my life; but I've never embraced Test-driven development (TDD). There are many reasons, but probably the principal ones were an expectation that TDD would slow me down and a belief that my existing methods worked fine—at least in the context in which I have worked for the past few decades.

Circumstances change and one of the changes facing me is that I appear to have spare cycles that could be applied to some free software development. Finding myself ready to begin creation of a new utility—for myself, but with wider usefulness likely—I thought I'd devote a day to experimenting with TDD. It has been an interesting experience.

Yes, it did slow me down, mainly because I was getting used to a completely new way of writing code and, to a lesser extent, because I was writing more code than I would have in the past. It will take me some time to get up to speed with TDD, but I can envisage a time when I could expect to travel at about the same speed as I'm used to. I take that as a positive lesson from today's experiment.

The more interesting discovery, for me, was something that I had not thought about at all, although it should have been obvious from the reading I had done. The process of small discrete steps (write a little test, watch it fail, write a little code to make it pass, refactor, repeat) keeps things in a state where the usual interruptions are much less of an issue. My usual way of working involves holding a substantial mass of code in mind at any time and interruptions are a real killer. I can usually arrange for blocks of uninterrupted time, but that's not always possible.

Today, while I was at my least amenable to interruptions because I was learning something new, it just so happened that my spouse needed to interrupt me several times. Each time I was able to turn away from what I was doing and resume fifteen or thirty or more minutes later without any stress—the small step that was interrupted turned out to be easy to resume without any difficulty. That alone seems like a reason to keep going with TDD for a few months to see if it can become part of my approach.

And, to return to the original issue, provision of a full unit test suite should make it much easier for other people to take the code and adapt it for their own purposes.

RapidXen Update

Almost 18 months ago I wrote about my unhappiness with RapidXen on my old blog. Contrary to my original intentions, I kept one of my RapidXen instances alive as a spare. I recently had a small issue with it and discovered that the RapidXen response was extremely prompt and effective. And I already know that their move away from Fremont had helped their reliability. So, prompted by lengthy failures of my Linode in Fremont, I thought I might put the RapidXen host back into an important role. And I moved my Linode to Dallas, hoping that the increased latency might be offset by better reliability. I'm hoping that this strategy, which now gives me hosts in Chicago, Dallas and Los Angeles, will be a success when I'm in Europe for four months next year.

I know the RapidXen people were unhappy with my previous post, so this post is intended to show them there are no hard feelings. I'm hoping to continue as one of their happy customers for a long time.

Should I Write My Own?

I've been making an effort in recent years to look first for something that exists already that I can use, rather than choosing by default to write my own solution for tasks that can be solved with software. Sometimes that's difficult because the thing I need is very particular to some internal requirements. At other times, it's difficult because the task is an obvious and common one and a gazillion people have solved it and the task of evaluating all those solutions seems bigger than writing a new tool from scratch.

My current project is backup. This seems like one that might have been solved a few times already. And it's a task that I've solved a few times already myself. In this case, I want something that does what Dropbox advertises, except that the data needs to be encrypted on my machines so that only opaque blobs are stored in the cloud. I had a look at Spideroak, which claims to do what I want—but, the first dozen times I ran the client software, it crashed. This does not fill me with confidence for that sad day in the future when I might want to access my data. And in any case it doesn't give me the other thing I want—the ability to select the places where the data will live. I want the data to be stored in redundant, widely separated data centres, all under different control.

I've done a lot of Googling, and a ton of reading announcements and documentation for various solutions. Nothing quite does what I want, at least among the things I've so far found. But I'm thinking that a little bit of scripting, some cron jobs and some tools like rsync and/or Duplicity might just achieve what I want. At least in this day of cheap disks the job of testing out the critical restore capability is just a matter of tedium. I might have more to say about this in the future.

Face Down Baby

I just watched a young mother get her priorities completely mixed up. She was crossing a minor road using a pedestrian crossing. I was driving the only moving vehicle on that road and was approaching the crossing at walking speed as I prepared to stop for her. She focused all her attention on me.

Her baby, about two, pitched forward out of the stroller she was pushing and landed face first on the road. Mother didn't notice, although I could clearly hear his screams inside my sealed car. She kept pushing, scraping his face along the ground. She didn't even look down for a second to see why she had to struggle to push.

It was not until I beeped my horn and waved frantically at her and the baby that she noticed the problem. Then she dashed around, grabbed him by the arm and dragged him and stroller off the road. Last I saw, the baby was making his feelings about this pretty clear. I just hope she gets a few clues from today's experience.

Who Cares About Free Speech?

It seems that everybody is allowed to write a book about themselves, unless our fine government dislikes them or their message. To quote from the Overland Blog, "On 20 July 2011, the Australian government served David Hicks with a notice of their intent to restrain any funds obtained from the sale of his book, Guantanamo: My Journey, under the Commonwealth Proceeds of Crime Act."

The Overland team have written more about this and have begun a Petition in Support of David Hicks which I have signed. I urge other people who believe in freedom of speech to sign it too—regardless of your feelings about David Hicks the man.

An Unexpected Insight

Way back when I was a newish programmer, I discovered something that probably every programmer has discovered: explaining the problem you're stuck on to somebody else will often cause the penny to drop and you'll see the answer quite clearly just because you have articulated the story for the other person. It turns out that this effect is not limited to solving programming puzzles.

Just the other day, I wrote about my decision to return to Python, specifically Python 3.2+, for my next software project and I articulated some reasons for that decision. Telling my small blog audience about that was a catalyst.

Yesterday, while working on something not connected with software, I became aware that I was unhappy about something. I let that awareness sit for a couple of hours while i completed my task and then sat down in a comfy chair to reflect.

What became clear to me was that I was unhappy about the Python decision. One of the goals of my recent research into programming languages was that I felt the urge to learn a new language. Not just any new language, but a language that would introduce me to some aspect of programming that was not familiar to me. While Python 3 is a new language for a Python 1.2 programmer, it's not really a new language and doesn't involve the kind of learning I had in mind. And so, if I just focus on Python, I'll certainly be able to write some software, but I won't get the mental stretch I was looking for.

Further thought has given me a new plan. I will still learn the latest Python incarnation properly and keep up to date with it, because it seems likely that Python's ease of use, wide applicability and the excellent local Python community are all things that will make it a useful language for me to employ for quite some time. But I will continue my study of some functional languages, excluding Erlang for the reasons I gave the other day, focusing on Haskell and Clojure for the time being. One new idea that came out of my latest thinking is that it might be smarter for me to devote smaller bits of time more regularly to my study rather than trying to read several pages of a dense textbook in one sitting.

I need to keep reminding myself that one of the principal goals is for this to be enjoyable. Hopefully, this two-pronged approach that allows me the time to get comfortable with two or three languages together will produce the result I'm looking for.

Erlang Misses The Boat

There is much to like about Erlang, especially its great support for concurrency, distributed processing, hot swapping code and the functional programming model. And it's a mature language, even perhaps a bit long in the tooth. As with many languages, the available books are a mixed bag and generally not very inspiring (although often a bit too full of fanboism).

Nevertheless, I persevered with my collection of books on and off over several weeks before starting to think that it was a toss up between Erlang really being too hard to use or me being too thick to get the magic. And then I found What Sucks About Erlang by Damien Katz, somebody who actually uses the language. And I instantly recognized all the gotchas he mentioned as the same things I had been bumping up against. There's no point in repeating Damien's story, just read the original for all you need to know.

So, rather reluctantly in view of the strengths of the language, I decided that for me, for now, Erlang is not the way forward. One of my prime goals in writing software is that it should be enjoyable, and with Erlang and the kind of stuff I have in mind at present, it wouldn't be fun at all.

Back to Normal

I've neglected this blog over the past seven months, due initially to escalating family dramas and then to inertia. The dramas are not material for the blog and seem to have settled sufficiently for me to turn back to what I call normal life.

My major project this year has been to tackle some new software development and, as part of that, to learn some new stuff. That took me on a tour of interesting programming languages in the hope that I could combine a new language with my development project. In no particular order, I've looked at some Lisp languages (Scheme, Common Lisp, Clojure) and some functional languages (Erlang, Haskell). All of them appeal in some ways and not in others. I'll try to cover the salient points in subsequent posts, but the take home lesson is that I've decided to go back to an old favourite, Python, for now.

I've been writing in Python for almost twenty years, and the fact that most of my code is written for Python-1.2 is a symptom of that. I've modified it over the years to accommodate what I call gratuitous changes in the language and libraries (which the Python community have of course all considered carefully and determined to be improvements). Being fully engaged in my work, I have not participated in the Python development process in any way for many years and that has left me in an awkward position when it comes to whining about the evolution of the language.

So I've decided on two steps for the immediate future. I'm going to sit down and learn Python-3.2+ thoroughly to get up to speed with the current state of the language and I'm going to pay attention to the ongoing development of the language in order to stay current. I may even manage to contribute in some way to Python. I'll probably return to some of those other languages in a year or so, to keep my brain open to useful ideas, but I think most of my work will be with Python and C for the present.

A Solaris Puzzle

I wrote about issues installing an operating system on some new hardware a few days ago. Since then I've done more exploring. A subsequent attempt to install Ubuntu 10.10 on the same box was a success, right up to the part where you reboot from the newly-installed system. That was a total failure and required power cycling to escape.

Then, several restarts later (due to my lack of speed on the DEL key), I managed to get at the BIOS. I decided to start from scratch, and loaded the safe defaults. Then I went through and tweaked it carefully (and very conservatively). After that, it booted fine. To be sure of this, I rebooted it (both warm and cold) about 30 times. All good. Then I left it to its own devices with a bunch of stuff running for a couple of days. Still good.

Then I tried again to install my Solaris-11 system, hoping that it would work on the new setup. No joy at all. The installer crashed, several times, in exactly the same place as before (i.e., long before getting close to actually installing anything). I really wanted to be able to use ZFS, which I think is a wonderful file system, but it seems impossible for me to install Solaris on any reasonably up to date hardware that I have available. I think I'm over this now and will wait for some other operating system that I can bear to use offering native ZFS.

Computers Can Make You Mad

I've just put together a new computer. It starts up fine. It reads a CD with an operating system I want to install just fine. The installer runs fine until we get close to actually doing something. Then it crashes. This is repeatable. I re-burned the CD in the drive on the new machine and tried again. Same deal.

I'm now running memtest86 over the 24G of memory in case the problem is there. So far, it has made seven complete passes without detecting anything. I will let it go until it says it has completed a full test sequence, but I'll be surprised if it finds anything now. If it's not memory, it's a bit hard to guess where the problem is. Since we never get as far as writing on a disk, it's not going to be there. Nor do we use the network.

The machine is certainly almost OK. Maybe even quite OK. It certainly seems to run Linux as I'd expect, although I haven't attempted exhaustive tests yet. Testing is so slow and so boring and gets in the way of everything else I'd like to be doing. And it's Friday.

My spouse has decided that we need a weekend away in a luxurious place far from the normal world. This seems like an excellent plan. Maybe the computer will reveal something useful next week. I'll be running out of time to claim DOA status for the bits by then, so that might affect my sleep tonight. On the other hand, losing sleep over a box of bits is probably unwise. Consider this as a muted scream.

There's Somebody at the Door

My spouse has had a hard start to the week and announced last night that she wanted a nice easy start today because she did not have any clients until 10:15. I was happy to go along with this plan and so the morning began peacefully.

At 7:30, just as I delivered the first cup of coffee to the bedroom, the doorbell rang. Spouse looked alarmed and asked questions that were hard to answer. However, I speculated that the caller might have been one of her 7:30 clients arriving on the wrong day. Much quiet cursing followed this thought and spouse dashed about the room grabbing the minimal clothing that she could wear to the door to face a client.

By the time she opened the door, the caller had backed away and was digging in her backpack. She found her diary, looked at it, turned to spouse and said, "Oops, I'm supposed to be here next week." And off she went.

Spouse returned looking like thunder, threw her clothes on the floor and flounced back into the bed. By then the coffee was cooling. That seemed like a good moment for me to go out to the kitchen to make fresh coffee.

I guess people who need psychotherapy because their life is in chaos can be expected to make this kind of mistake, but some days it really would be great if they could look in their diaries before ringing the bell.

Sandfire Roadhouse

Armed with one hugely costly spare tyre, I headed north. The next leg was a wee bit daunting: 560 km of dirt road with nothing at all along it, according to my new map. I was a bit anxious about the tyres, but felt I had to press on. It was hot and getting hotter. About half way along this barren stretch, one of the front tyres exploded. It's tricky to deal with this on dirt at 200 km/h and the wheel itself and parts of the brake were junk by the time I stopped.

It was the middle of the day. The road was so hot, it hurt to kneel down to undo the wheel nuts. Eventually, I got the front right corner of the car jacked up enough to remove the wheel. By then I was exhausted and mildly panicky. I threw a towel under the car and slid underneath to get some shade. The heat from the road below and the hot exhaust above was grim, but the shade still seemed better than being out in the sun. I might have been slightly irrational at this point.

Half an hour later, a truck going south stopped to see if I was dead. When the driver understood what I was doing, he said, "Come on, let's get your spare on the car and then you can stop about five miles back there for cool shade and cold drinks and people to chat with."

I found that hard to believe, but he was insistent that there was a new place for travellers to rest at. He did most of the work getting the wheel on and the tools packed away and headed off on his way south, and I drove gingerly north. And it was true, there was a place to stop. My memory says it was called Sandfire Lodge, but either my memory is wrong or they changed its name later. There is a place called Sandfire Roadhouse marked on current maps of Australia in the right place, so it's still there.

I was one of their first customers and they asked me to sign their visitors' book and their petition to be granted an alcohol licence, which I cheerfully did. I doubt if they still have those records from 1970 and my name has been changed many times since then, so tracking that down would be tricky now.

They had a spinifex shelter with water running down the walls to cool the wind blowing through. It was better than heaven for me that day. And they had cold soft drinks on ice which made the whole experience seem magical. A lot of my angst over the Dunlop Tyre Service at my previous stop evaporated at Sandfire Whatever. Although I was still aware that I once again had no spare tyres and a long way to go before stumbling on civilisation again, so I took things easy when I finished my rest with the kind people in the middle of nowhere.

Easy Does It

Back in the present today, for a look at some software that I was wrestling with yesterday. I often whine about issues with free software that is hard to use or configure or that crashes messily. But it's rare for me to complain about commercial software. I'd hate people to think that was because commercial software is better when the real reason is simply that I avoid using commercial software as far as possible. But yesterday I spent some time with three of Atlassian's products (for which I have the cheap Starter Licenses).

Why? There are times when friends tell me I should use the Atlassian software rather than wasting my time with less-capable or otherwise deficient free software and I finally thought I'd combine two things by contributing a few dollars to the charity that Atlassian supports and trying out their software.

I downloaded Confluence, Jira and GreenHopper and installed Confluence first on a bare Ubuntu box. Along the way, I carefully followed every step of the instructions, including installation of additional software packages that were required. I configured a brand new MySQL installation as instructed. Then I started Confluence and began to step through its config screens per the instructions. Somewhere along the way, I encountered an error with an insane diagnostic.

Always willing to assign blame to myself in the first instance, I blew away the entire installation and began again. Again, I was meticulous about following the instructions. Again I got the same error.

I mention my following the instructions because this is a big deal for me. I'm one of those sad people who reads the owner's manual for a new car or a new mouse from cover to cover before using it, even though there is virtually no chance that I'll learn anything from my reading. But I am confident in my ability to follow written instructions perfectly. When I can't get a piece of software to work after carefully following the instructions, the problem is with the software. The instructions are part of the software, which is why I say the problem is with the software.

I'm sure I could probably get somebody at Atlassian to step me through the setup for Confluence, but I'm not interested in doing that. I've learnt all I need to know. That means that I won't use any of the Atlassian products, nor will I suggest that other people try them. I'm really documenting this here so I can point to it any time somebody suggests Confluence or Jira to me in the future.

Service With a Smile

As I drove into Port Hedland on my way north, I spotted a Dunlop Tyre Service and thought about the four blown-out tyres I was carrying. Perhaps it would be smart to stop for some tyre service. The place was quiet, the workers all wearing nice clean white overalls, and I was pleased to see that I could expect swift service and a chance to be on my way with little delay.

My illusions were shattered when I finally got one of the men to acknowledge me. "Sure, mate, toss them there. Come back on Thursday." I don't really remember which day he said, but it was about three days in the future and I could spare an hour or two at most. So I pleaded with him. No dice. I grovelled. Eventually, he told me they'd sell me tyres and tubes but I'd have to fit them myself if I was in such a hurry. With my tools, not theirs.

This didn't suit me at all. I hate tyre irons and car wheels. And it was hot, seriously hot. So I sat on the ground, lit a smoke, and fantasised about nasty things that could happen to the fine folk at the Dunlop place. Before I gave in and submitted to my fate, a shiny new ute arrived with a flat truck tyre in the back. The driver was wearing a suit, which was something I didn't ever expect to see in that part of the world.

He went through the same routine with the helpful men in white. But his grovelling was more intense than mine because he had a team of men waiting out in the sticks for him to get back with the wheel. He had no more luck than me and was on the point of tears.

Seeing a possible win-win situation, I approached him and asked him if he knew how to fix a car tyre. He said yes. So I offered to fix the truck tyre if he'd do mine. This wasn't completely altruistic, as the only hard part with truck wheels is knowing what to do. But it's much easier than a car tyre. He was ecstatic and offered to pay my bill as well as do the work on my tyre. I said OK, because I expected to be paying a couple of dollars for a new tube.

I put a new tyre and tube on his wheel. He put a new tube inside my old tyre and mounted it. Then he went to pay. Luckily he had a business cheque book, because my tube cost over $100 (in 1970). I don't recall how much they gouged him for his truck tyre and tube, but I do remember being shocked.

I'd actually got to be 23 years old and driven almost all the way around Australia before I saw my first real proof that the rumours I'd gown up on telling of the kindness and generosity of the people in the bush were not to be relied upon. Although I was soon to learn that not everybody was like that Dunlop place.