Wednesday, December 4, 2013

The Virtual Clock Test Pattern

You can have a hard time unit-testing code that depends on the system clock. This article describes both the problem and a common, repeatable solution.

[Note: This is an article I wrote in 2006. I'll publish it here because people sometimes still quote it, and the original version fell off the Internet. (But the older version, with examples in Java instead of Ruby, can still be found on the Wayback Machine).]

 

The problem


To see what the Virtual Clock is about, we can dive into a simple example. We'll code this in Ruby, but you should be able to understand it even if you're not a rubyist – and you'll learn some Ruby along the way.

We're building a minimal scheduling system where you can create tasks and execute them later. Each task has a maximum age, after which it expires and cannot be executed any more. Here's the Task class:

class Task
  def initialize(max_age, &action)
    @max_age = max_age
    @action = action
    @time_of_birth = Time.now.to_i
  end

  def age
    Time.now.to_i - @time_of_birth
  end

  def execute
    @action.call if age <= @max_age
  end
end

The method initialize is Ruby's equivalent of a constructor – it gets called when you write Task.new to create a task. The first argument of initialize is the maximum age of the task, in seconds. The argument is stored into a variable named @max_age. The @ prefix means this is an instance variable (you might be used to call these “object fields”). Ruby being a dynamic language, you don't have to declare the variable anywhere else: you just assign something to it, and it springs into existence.

The second argument of initialize is not really an argument – it's a block. Ruby blocks might look foreign to you. They allow you to write things like:

t = Task.new(10) { print “Executing, sir!” }

The curly braced thing is the block. It contains the action that we want the task to execute. From inside the method, it looks like an argument prefixed by an ampersand. From outside, it's more like a snippet of code that's attached to the method call. It's not executed immediately. Instead, Ruby internally converts it to an object, and we store it into the @action instance variable to execute it later.

The last instance variable, @time_of_birth, is initialized with the current time. Time.now gets the time from the system clock, and the method to_i converts it to an offset in seconds from a conventional date. The method age uses the same instruction to find out how old the task is.

Finally, the method execute does the real work: it executes the stored block by invoking its call method. This is only done for tasks whose age is less than @max_age – cranky old tasks just ignore your attempts to execute them. It's idiomatic Ruby to put the if condition at the end when the body is a single statement.

This is a smart little class, but we should have written a test for it. We haven't been good Test-First coders, have we? Better late than never, so let's focus on that test. But now we have a problem: to test this code, we need to be sure that some tasks expire before we execute them, and some others don't. That's not easy. We could insert pauses into our test by calling Kernel.sleep, but that would slow down the tests. It would also make it very difficult to test edge conditions. Worse still, as the code gets more complicated, our tests might become non-deterministic. You probably know the problem: sometimes the test code runs slightly slower for any reason, and the tests fail randomly. There is nothing worse than a test that fails 10% of the times.*

Unfortunately, we cannot control system time. Or, can we?

 

A solution


We need to control time itself. We can do this by defining two separate clocks:

class RealTimeClock
  def time
    Time.now.to_i
  end
end

class VirtualClock
  attr_accessor :time

  def initialize
    @time = 0
  end
end

The RealTimeClock returns the current system time when you call its time method. The VirtualClock doesn't care about the system clock at all. Instead, it returns the value of its time attribute, which is initialized to zero and can be modified by the clock's clients (attr_accessor just tells Ruby that we want an object attribute named time, and its value is stored into an instance variable named @time).

Now we can modify the Task class to use one of our brand new clocks:

class Task
  def initialize(max_age, clock = RealTimeClock.new, &action)
    @max_age = max_age
    @clock = clock
    @action = action
    @time_of_birth = clock.time
   end

  def age
     @clock.time - @time_of_birth
  end

  def execute
    @action.call if age <= @max_age
  end
end

You can pass the clock in when you create the Task. If you ignore the argument, it will be assigned a RealTimeClock by default. Now it's easy to write a solid test:

require 'test/unit'

class SchedulerTest < Test::Unit::TestCase

  def test_active_tasks_do_something
    clock = VirtualClock.new
    executed = false
    task = Task.new(0.2, clock) { executed = true }
    clock.time += 0.2
    task.execute
    assert executed
  end
end

We created a task that uses the Virtual Clock, and instructed it to change the value of the executed flag. Then we used our own little time machine to set the current time and check that the action associated with the task is actually executed. The test for expired tasks is even simpler:

def test_expired_tasks_do_nothing
  clock = VirtualClock.new
  task = Task.new(0.2, clock) { flunk }
  clock.time += 0.21
  task.execute
end

The block associated with this task is a call to flunk, a test assertions which always fails. We're simply testing that this action is never called, and flunk is never executed. Behold the Green Bar!

 

It all boils down to...


To write good tests, we need lots of control over our test environment. We must be able to set it up exactly as we like. If a piece of code relies on non-deterministic behaviour, then we are in trouble.

The system clock is non-deterministic by nature. It's an important system property, but we cannot control it. The Virtual Clock pattern gets around this by replacing the system clock with something that we can actually control.

Therefore:

Don't access the system clock directly. Instead, wrap calls to the system clock into a Clock object, and replace it with a Virtual Clock for testing.

 

More ideas


Global clocks – If you use this pattern, you can end up passing clocks all around the place. Some people dislike this, and consider it a case of tests polluting production code. An alternate solution is a singleton clock with global access. You can make it a Real Time Clock by default, and switch to a Virtual Clock for testing. But be careful: it's safe to have a global Real Time Clock, since this is a read-only object – but the Virtual Clock isn't. I was burned by this approach when I forgot to reset the global Virtual Clock after a test, and was punished by a mysterious failure in the following test.

Not only for testing – The Virtual Clock decouples the concept of “time as an input” from “real time”. Time becomes a variable like any other. This can be useful for things other than testing. For example, you might want to simulate a process over a long time span. Or maybe you have a piece of code that processes historical data, and you want to trick it into working at a different time than “right now”.

Clock supertype - The VirtualClock and the RealTimeClock of this article don't need to share any special relationship. In dynamic languages such as Ruby and Python, it's enough that both classes implement a time method. Any piece of code that relies on time alone will gladly accept any object that implements this method (this is known as duck typing). In Java or C#, the clocks need to share the same explicit type to get this kind of polymorphic behaviour. You'd probably do this by defining a common Clock interface.

A virtual family – Instead of a Virtual Clock that just counts seconds, you might want to define a Virtual Date to abstract calendar dates. You can also adapt this pattern to deal with any non- deterministic entity, such as random number generators or external device drivers.

Related patterns – In pattern-speak, a Virtual Clock is an example of a Test Double - more specifically a Fake Object. To make objects aware of the Virtual Clock, you can use any kind of Dependency Injection. In this article, we used Constructor Injection to pass the Clock around.

 

Known uses


Martin Fowler mentioned that he always uses indirection on the system clock. He touches on the subject when he describes the Time Point pattern.

Prevalence systems such as Prevayler use a Virtual Clock to guarantee deterministic behaviour.

Real-time coders routinely simulate time. John Carmack used this technique to test its Quake 3 game engine.

There are many more examples of Virtual Clocks around. This is a common pattern. [2013 update: There are many more examples available today. The Timecop gem is one of the current popular implementations of this pattern in Ruby.]

 

Thanks to...


The following people helped me review this article, gave me comments and suggestions, or just pointed me to useful material: Kent Beck, Emmanuel Bernard, Roberto Bettazzoni, David Corbin, Chad Fowler, Martin Fowler, Patrick D. Logan, Dan Palanza, J. B. Rainsberger, Andrea Tomasini, Marco Trincardi, Andrew Wall.

* On second thought, a test that fails 5% of the times is probably worse than that.

Sunday, December 16, 2012

Sssmoke - When even Sinatra is too much

Sometimes you don't need a web framework, no matter how light - you just want to slap a Ruby script or two on a web server. Enter Sssmoke:
gem install sssmoke

Put your erb templates in a directory, then run them in a web server by typing:
sssmoke

A template named foo.erb will get the URL http://localhost:8888/foo.

Advanced options for powah usahs:
sssmoke directory_name      # sssmoke templates from another directory
sssmoke template_name.erb   # sssmoke a single template at http://localhost:8888/

That's all. (As to the reason why it's called "Sssmoke"- that story would be better told in person.)

Saturday, August 11, 2012

What Good Error Messages Look Like

Good error messages do three things:
  1. They tell me what's wrong and how to fix it.
  2. They stand out.
  3. They soothe my soul.
A teammate just sent me some output from Maven, a popular package manager/project manager/kitchen sink for Java. She spotted a warning, well-concealed near the top of hundreds of lines of infodump. I had to format it to make it readable.

[WARNING] Some problems were encountered while building the effective
          model for [our_project]
[WARNING] 'dependencyManagement.dependencies.dependency.(groupId:
          artifactId:type:classifier)' must be unique: org.jboss.resteasy
          :resteasy-jackson-provider:jar -> version 2.3.2.Final vs 2.3
          .0.GA @ com.[my_customer].[our_project]:[our_project]
          :0.17.Beta-SNAPSHOT, /Volumes/Workarea/trunk/pom.xml, line 272,
          column 16
[WARNING] 
[WARNING] It is highly recommended to fix these problems because they
          threaten the stability of your build.
[WARNING] 
[WARNING] For this reason, future Maven versions might no longer support
          building such malformed projects.

Gotta love the "please follow us, citizen" tone of those last few lines. I'll tell you what, Maven: either the stability of my build is really being threatened, and then you shouldn't hide this information so carefully - or it's not, and then you shouldn't be such a picky jerk. And by the way, your mother is way more malformed than my project. Sheesh.

Compare that to Homebrew, a package manager for OS X. After an operating system upgrade, I ask it whether there are any problems with my setup:

~$ brew doctor

Instead of dumping the Library of Congress to my terminal as Maven would, Homebrew gives me a clear screen with three well-formatted warnings. First one:

Warning: Some keg-only formula are linked into the Cellar.
Linking a keg-only formula, such as gettext, into the cellar with
`brew link f` will cause other formulae to detect them during the
`./configure` step. This may cause problems when compiling those
other formulae.

Binaries provided by keg-only formulae may override system binaries
with other strange results.

You may wish to `brew unlink` these brews:

    libxml2
    libxslt

I brew unlink the two libraries as instructed. Fixed. Second warning:

Warning: Some installed formula are missing dependencies.
You should `brew install` the missing dependencies:

    brew install autoconf automake libtool

Run `brew missing` for more details.

Command copy-pasted, libraries installed, problem fixed. Third warning:

Warning: /usr/bin occurs before /usr/local/bin
This means that system-provided programs will be used instead of those
provided by Homebrew. The following tools exist at both paths:

    bashbug
    clusterdb
    [more stuff]
    
Consider amending your PATH so that /usr/local/bin
occurs before /usr/bin in your PATH.

I spend a few minutes fixing the PATH. Let's try again:

~$ brew doctor
Your system is raring to brew.

That's why Maven fills my soul with darkness and desperation, while Homebrew overflows my heart with love and unicorns.

Tuesday, October 11, 2011

Windows 8: solving the wrong problem

In the 80s and the 90s, we had a Holy Grail: cross-platform compatibility. We wanted to write our software once, and then run it on different platforms. It felt like a good dream to share: it wasn't pretty to rewrite the same stuff over and over. If your company wanted to support Windows and Apple, then you needed two separate teams writing the same application for two separate OSs. That felt like a huge waste.

We tried hard. We looked at every C compiler flaunting cross-compilation, every database driver promising vendor independency, every high-level approach touting push-button code generation. The more these solutions became sophisticated, the less they seemed to work.

We blamed Microsoft, Oracle and other corporate lockers-in for that sorry state of affairs. We were wrong. As it turned out, we were just trying to solve the wrong problem.

The real problem was not a technological issue: it was a usability issue, a culture issue, and a marketing issue. Different platforms approach the same domains differently, and their relative value lies in those differences, not in the common denominator. At one point, Java managed to solve the technological problem for good, and that was the point where we realized the awful truth: cross-platform compatibility was not important. It never had been.

So we quit trying to solve that problem. Instead, we left it behind by moving up a level and inventing a new, shared platform on the Web. (I still see companies pursuing push-button tools that generate or translate code for the CLR and JVM alike. That saddens me: somebody is still working on the wrong problem.)

Now, as it happens in IT, we're running another iteration of facing the same issues and making the same mistakes. We have multiple devices (PCs, smartphones, tablets), so we'd like to use the same software all over the spectrum. That's where Windows 8 seems to be going: you have the same OS on your tablet and your PC, so you can leverage the same technologies on both. And once again, this isn't going to work, because a tablet and a PC are different, and all those subtle and not-so-subtle differences pile up to require different approaches. Convergence is not important, interoperability is. Broad commonalities are not important, tiny details are. And please, Microsoft, get over it: the OS is not important, the user experience is.

That's why I think that Windows' current approach to tablets and smartphones is fundamentally, tragically, so-fucking-broken-it-cannot-be-fixed wrong.

Tuesday, August 9, 2011

Get Your Ruby Project on Travis and Have a Martini in 15 Minutes

Almost overnight, every Ruby project out there seem to be moving to Travis. Travis is a dead-simple, community-owned build system. I'm usually too lazy to put all my projects on automated build. Travis took away my excuses by getting me from zero to the first build in a matter of minutes.

Here are step-by-step instructions to get your project on Travis and have a delicious Martini Cocktail in about 15 minutes. Please note that preparing the Martini will take about 5 minutes, so the Travis part should take just 10 minutes of your life.

Check That Your Project Has What It Takes (2 minutes)

Your project needs three prerequisites to get on Travis:
  1. It's a Ruby project on GitHub.
  2. It uses Bundler to manage its gems. (Actually, that’s not strictly necessary, but it will make it easier to set up Travis.)
  3. You can run the project's tests with a single command. A Rake task is typical, but other commands (like, say, bundle exec rspec spec) are also fine.
Ultimately, you should be able to test your project on a new machine by just doing a bundle install followed by the test command. If your setup is more complicated, then you'll need extra work to put the project on Travis. It's probably a good idea to make your project very easy to setup, whether or not you want to use Travis.

I’ll assume that your project meets the three prerequisites, and that you can run your test with bundle exec rake test.

Create a Travis Configuration File (3 minutes)

Commit a new file named .travis.yml in your project root. Here is what mine looks like:


All the entries have sensible defaults, so your configuration could be even simpler. For example, if you skip the script property, then Travis will try bundle exec rake, or just rake if you're not using Bundler. You can find more details on the Travis configuration page.

Activate Your Project on Travis (2 minutes)

Go to http://travis-ci.org and sign in with your GitHub account. Grant Travis read/write access to your GitHub. You should see your private Travis build token on your profile page:


You don't really need to care about the token now - but while you’re on the profile page, flip the switch for the project that you want to build with Travis.

Run Your First Build (3 minutes)

Push to your git repository (make sure that you committed the .travis.yml file), then go to the Travis home page and sit back as Travis adds your project to its build queue, installs the bundle and runs the tests. When the build is done, check your email to find a little love message from Travis - and congratulations for getting it green (or red)!

If You're Curious...

Why did Travis require write access to your GitHub account? That's because Travis automagically configures GitHub to be notified when you push to the project. Go check it if you like: on your project's admin page on GitHub, follow Service Hooks, and click on the Travis hook. The configuration should look like this:


If you click Test Hook, Travis should schedule a build right now.

Prepare the Martini (5 minutes)

Fill a frozen cocktail glass with cold gin, add a touch of vermouth and stir. Garnish with an olive.

(If you’re in a hurry, you can merge this step with the previous one, thus sparing 3 minutes and keeping yourself busy as Travis is building your project.)

Drink the Martini (extra quality time)

I don’t think you need my help here. Just find good company and enjoy. Drink responsibly!

Wednesday, June 1, 2011

Euruko 2011 Thoughts

Back from the European Ruby Conference. Great people, good talks overall. Here are my quick thoughts on what was good, and what could be improved.

(But first, a message to the people who praised my speech: I'd have to spam Twitter to thank each one of you, so please accept one big collective "thank you". I was stoked by your feedback.)

Why Euruko Was Brilliant

Flawless organization. I don't mean to reinforce cultural stereotypes ("The Germans are well-organized"), but this was one of the smoothest conference experiences I've ever been through. The volunteer organizers did a better job than most people who do this for a living. I heard there were a few people lost in the bushes or eaten by wild animals while hunting for the Saturday night party, but you can argue that was part of the fun. ;)

In particular, the location was great. Berlin is already one of my favourite cities, but this topped my best expectations: Karl-Marx Allee, near Alexanderplatz, one of the most reachable landmarks in town. That, and the most impressive conference screen I've seen so far.

Internet connection is usually a sore spot at conferences, but these guys astounded everyone by installing a parabolic dish on a nearby building just to provide us with perfect wireless connections. There were a few minor hiccups on the fist day, but they were quickly fixed. I was as impressed as everyone else. Standing ovation!

Oh, and most conferences should get a clue from these people when it comes to classy, non-dorky-looking t-shirts.

Why Euruko Should Get Bigger

I've read comments that the conference was too big, and previous Eurukos felt cozier. I do agree that small conferences make it easier to socialize. Nonetheless, I think that Euruko should be bigger, not smaller.

Matter of fact, only a selected lucky few made it to the conference. I'm talking out of experience here: if I hadn't been a speaker, I would also have been left out. Getting a ticket proved impossible. Like so many other people, I was hitting "Refresh" constantly, and tickets went straight from "Not yet available" to "Sold out" in the space of one HTTP call. Some of the people who were left out organized their own parallel conference. Most likely, hundreds more people just gave up and stayed home.

The organizers told me that they were taken by storm by the number of requests, and I understand that. Still, it's frustrating to think that next year I might have to either submit a speech or snipe the tickets if I hope to enter the conference (especially since next year's Euruko will be in Amsterdam, my other favourite northern city).

Apparently, only a small fraction of the submitted speeches made the cut. I gather that the organizers intentionally kept the conference single-track, to keep people closer together - but I think the idea backfired. I'm more likely to socialize if I have multiple tracks, smaller rooms, and plenty of corridors to hang around. Many talks received lukewarm feedback, and I think that was because most talks were very specific, so they couldn't possibly appeal to everyone. I'd rather let people select the talks that they find interesting across multiple separate tracks, than have the organizers do the selection for them.

One of the organizers told me that he thinks the European conference should strive to stay small and single-track. Apparently there is this meme that small conferences are more about people, and big conferences are more about money and business. I guess this fits with Berlin's young-and-smart, community-oriented, mildly anarchist mindset. Again, I sympathize with those feelings, but I disagree with the conclusions. Small conferences may be more about the people, but matter of fact, most of that people are being left out in the cold now. If it's small conferences we want, we already get plenty. Ruby is growing big in Europe, and the main European conference should be as big as the number of attendees - not the other way round.

So, to the folks organizing next year's Euruko: maybe it's time to go bigger?

Tuesday, March 29, 2011

GitHub is a game changer

Update: here is another recent blog post with excellent detailed instructions on forking and contributing to a GitHub project.

I used GitHub for a while before I could really appreciate what this site is doing to our culture as software developers. GitHub isn't just a neat tool: it's a revolution in the making.

GitHub is a social platform for developing software, built around git. The use cases for git have always been interesting, but not necessarily compelling. GitHub builds on the strenghts of git, and the result is a brand new approach to writing and sharing code. While git takes care of the technical issues like forks and merges, GitHub takes care of the social issues like pull requests and tracking forks.

Just think of a typical development problem: patches. Your project uses a third-party library, and that library has a bug. Here is how you usually deal with the problem if the library is proprietary:
  1. You contact the producer's customer service and report the problem.
  2. You sit on your hands until the company fixes the problem. If ever.
Here is what you tipically do with an open source library:
  1. You go to the library project's forum and report the problem.
  2. You sit on your hands until the authors fix the problem. Eventually.
As an alternative, you could fix the problem yourself. To do that, you have to download the library's code, set it up, make sense of it, patch it, build it and test it. It could take you weeks. Then you submit a patch, and you hope that your patch will end up in the original codebase. Until then, you have to merge subsequent changes from the original codebase into your patched local codebase. Merges are usually painful enough that you'll probably stick with steps 1 and 2 in most cases. Bottom line, you'll probably end up doing like you would for a proprietary project: you find a workaround, and keep hoping.

Now, here is what you do with GitHub:
  1. You fork the library.
  2. You fix the problem.
  3. You use your own forked library in your project.
  4. You send a pull request to the original library.
GitHub makes step 1 as easy as you could reasonably hope. If you use good tools like Bundler, step 3 becomes a one-liner as well. And step 4 is both trivial and not necessarily critical. You still have to merge future changes from the project into your fork, but git makes it easy for you to merge, and for the original library to incorporate your patch. All in all, GitHub turns patching from a bottomless pit of pain into a geeky form of pleasure.

Now look at step 2, the remaining difficult step. To make it worth the trouble, you need a project that is easy to setup, easy to understand, and well covered by tests. So GitHub is a perfect fit for languages that promote clear, concise code and for communities who take pride in unit testing and easy project setup.

Suddenly, complicated open source projects feel so last century. By removing all accidental difficulties of patching, GitHub tips the balance towards projects that are trivial to set up and test, and languages that promote easy change. If I need to pick between two projects, I'll probably pick the one that's easier to patch over the one that has the most bells and whistles. Month by month, GitHub is shifting our collective coders' hivemind towards simplicity.

After all, wasn't this one of the original promises of open source? "It's your code. You'll never be stuck."

Wednesday, December 1, 2010

Bundler: How You Can Start Using It Today

I took some time before I could understand and use Bundler in my Ruby projects. There was something about it that just didn't click with me.

In hindsight, I should have learned Bundler earlier. Once you "get" the workflow, Bundler is very simple and very powerful at the same time. It's one of those well-made tools that makes it dead easy to do simple stuff, and entirely possible to do complex stuff when the time comes.

I wrote a Bundler primer for the latest PragPub, the Pragmatic Programmer's monthly magazine. If you're not using Bundler yet, I hope that this article will turn you into a Bundler user before dinner.

(Even if you don't care about Bundler, the rest of the magazine is likely to have something for you. PragPub is my favourite programming mag ever, and I'm proud to write for them when it happens.)

Thursday, October 7, 2010

The method_missing() Chainsaw

I have a guest post about method_missing() on RubyLearning. It's intended for Ruby beginners, but it's attracting interesting discussions in the comments and on Hacker News - mainly from people dissing Dynamic Methods in favor of Ghost Methods.

Sunday, August 29, 2010

The Ruby Metaprogramming Spell Book

Here are all the spells from Metaprogramming Ruby, as link-friendly Gists.

March 2014 update: I updated this list to match the second edition of the book. I added three spells that are new to Ruby 2.x: Refinement, Refinement Wrapper and Prepended Wrapper. I also dropped a few spells that don't seem to be as relevant today as they used to be, at least in the context of this book: Argument Array, Named Arguments, Pattern Dispatch and Class Extension Mixin.

































Saturday, August 28, 2010

RubyKaigi 2010

The scarily efficient organizers of RubyKaigi 2010 only took a few hours to publish the video of my talk on metaprogramming and the Ruby object model (slides and audio only).

Yesterday I also lived the Andy Warhol moment of my life when I signed the first hundred or so copies of Metaprogramming Ruby Japanese Edition together with translator Kado Masanori and Matz. I looked at the long line of people happily waiting half an hour to have their copy signed, and suddenly spending three years to write a book seemed like a perfectly rational thing to do.

Friday, August 27, 2010

Metaprogramming Ruby in Japanese is Out

This is selling great at RubyKaigi in Tsukuba. I'm a happy little author.


Big thanks to my translator, Kado Masanori, and my Japanese editor-in-Chief, Kahei Suzuki.

Wednesday, August 18, 2010

Why Inheritance Sucks (in Ruby, at least)

[Update: Some readers went up in arms over this post, probably because the original title didn't specify which language I'm talking about. To clarify, I'm talking about inheritance in Ruby, compared to using modules, also in Ruby. My point here is that inheritance is an essential feature in C++/Java/C#, but not as much in Ruby. No, I'm not saying that Java should drop inheritance anytime soon.]

I came to Ruby from a static language background (C++, Java), and I had a hard time leaving my hold habits behind. In particular, as a Ruby beginner I tended to overuse inheritance. These days, I rarely use inheritance at all. Instead, I use modules. Let's look at the difference.

When you use inheritance, the superclass becomes an ancestor of the subclass. When you call a method, Ruby walks up the chain of ancestors until it finds the method. So, objects of the subclass also get the methods defined in the superclass.


When you use modules, the module also becomes an ancestor of the class, just like the superclass does:


When you call a method, Ruby still walks up the ancestors chain until it finds the method. The net effect is exactly the same as the picture above, except that Bird is now a module instead of a class. So, having a method in a superclass or having the same method in a module doesn't make much difference in practice.

However, modules are generally more flexible than superclasses. Modules can be managed at runtime, because include is just a regular method call, while superclasses are set in stone as you write your class definitions. Modules are much easier to use and test in isolation than tightly coupled hierarchy of classes. You can include as many modules as you like, while you can only have one superclass per class. And finally, when you get into advanced Ruby, modules give you much more flexibility than classes, so you can use modules to cast magic metaprogramming spells like Singleton Methods and Class Extensions.

If inheritance is so much worse than modules in Ruby, then why do languages like Java and C# rely on inheritance so much? There are two reasons why you use inheritance in these languages. The first reason is that you want to manage your methods - for example, re-use the same method in different subclasses. The second reason is because you want to upcast the type of a reference from a subclass to a superclass - that's the only way to get polimorphism in Java. The first reason is not as valid in Ruby, because you can just as well use modules to manage your methods. However, upcasting is more interesting.

Java is both compiled and statically typed, so the compiler can analyze your code and spot type-related mistakes. In particular, it can spot upcasting mistakes: if you have a method that takes Minerals, and you pass a Dog to the method, then the compiler will complain that a Dog is an Animal, not a Mineral, so you cannot upcast a Dog reference to a Mineral reference. In Ruby you don't declare your types, so you don't have upcasting at all. Even if you did have upcasting, you wouldn't have a compiler double-checking it. So you don't get the same advantages out of inheritance in Ruby compared to Java.

"Wait a minute," I hear you say. "Some of the limitations of inheritance are actually a good thing! Including multiple modules in Ruby is just like having multiple inheritance in C++, and multiple inheritance is a big mess. That's why Java and C# force you to inherit from a single class". This is the "diamond" problem that my original C++ mentor used to warn me about: if your class has two superclasses, and they both inherit from yet another superclass, then you get a diamond-shaped inheritance chain that can potentially be confusing. Wouldn't modules be a throwback to this kind of headaches?

In practice, however, Ruby modules tend to be more manageable than multiple superclasses. In Ruby, the chain of ancestors always follows a single path, where each module or class can only appear once - so you can't have diamond-shaped inheritance. If you understand how Ruby builds the chain of ancestors, you're never going to find yourself in an ambiguous situation where you don't know which method is called: simply enough, Ruby always calls the version of the method that's lower on the ancestors chain. (You can still get a clash if two separate modules reference instance variables with the same name, but that rarely happens in practice.) More crucially, the way you write code in Ruby is different from the way you write code in a static language. If you get used to crazy stuff like replacing methods with Monkeypatches, then there is no reason why you shouldn't get used to managing methods with modules.

I took literally years to get rid of my tendency to think in inheritance. Now I finally understand why large Ruby projects such as Rails barely use inheritance at all, and rely almost exclusively on modules.

Use inheritance sparingly.

Friday, June 25, 2010

The New Ruby Ecosystem

Last week I finally caught up with the state of Ruby - the language, the community and the tools in fashion. I'm impressed. These guys seem to be doing a lot of things right.

I'd taken a break from Ruby a few months ago, after Metaprogramming Ruby had gone out of beta. On my comeback, I was a tad disappointed at first. I couldn't see much progress on things like standard language specs, Windows support or in-process concurrency. I didn't even see an increase in Ruby's popularity. In fact, even as the language is going strong in the marketplace, the burst of adrenaline that used to accompany its rise seems to have faded.

So, where did the once-vocal Ruby community go? As it turns out, they're hard at work building a new environment. It's an ecosystem composed of many small moving pieces, mostly interchangeable and in continuous flux, yet integrating nicely with each other. It's young and rough around the edges, but it's working well enough, and it pushes the envelope far enough, that it deserves its own name: I'll call it the New Ruby Ecosystem.

Here are a few examples:

- It seems yesterday that the Rails team announced their merging with Merb, their primary competitor for web developer love. In less than one and a half years, these guys merged two large frameworks into one better, cleaner framework that some people are already using in production. Most companies I worked with would take multiple years to accomplish such a feat - if ever.

- Any Ruby web framework now jives great with any Ruby web server. I can scale down from a fully load-balanced production system, to a local Apache, to a quick in-process web server without even thinking about it. I can also add components to the HTTP chain in a snap. Thanks to Rack's amazing simplicity, it took less than one year to move from Rack 1.0 to widespread Rack compliance. As Sam Ruby put it: I love it when a plan comes together.

- In what seems like one year, pretty much all Ruby-related development migrated to git and GitHub. GitHub is now a one-stop shop to do whatever you wish to whatever piece of open code in the Ruby world. Forking and contributing has never been so easy. The technology behind the library repositories took the time to split, experiment and then merge again, so that publishing a project to the community is now boringly easy. The barrier to contribute, join efforts or part efforts in the Ruby community is as low as they get.

- No other technology offers a comparable number of options to connect to just about any cool recent thingie out there, from noSQL databases to clouds. Yesterday I was looking for libraries to access MongoDB, and I was totally overwhelmed. These libraries also tend to be simple enough that switching from one to the other is as easy as I could hope. I just picked up the default Mongo driver for my next project, and I'm confident I'll be able to switch quickly to another driver or even a relational DB if I need to.

- There are now about ten flavours of Ruby interpreters, ranging from experimental, to cutting-edge, to enterprise-stable. I can have all of those interpreters, or even multiple versions of the same interpreter, on one computer. I can install a new Ruby, select a default Ruby, switch Ruby on the fly - each of those with a single command. I can switch Ruby automatically for a specific project directory, and I don't even have to notice: I just type ruby or irb, and bang - I'm using the “right” interpreter. I can even have multiple sets of gems (libraries) for each Ruby, and switch gemsets with one command like I switch Rubies. I cannot imagine a more flexible language environment.

- I can declare all the libraries that my project needs, install them all at once, instruct the project to ignore any unlisted library, and package all libraries inside the project itself. I can even use Bundler and rvm together to isolate each project within its own little bubble of libraries. The result comes close to copy-and-run installation from development to production. Complex library management tools suddenly feel so last year.

- The popular Ruby libraries, especially the Rails add-ons and the testing tools, evolve at a scary pace. Projects like Cucumber release production updates faster than you'll probably care to get them. Still, the tools are remarkably stable (unless you choose to ride beta versions) and play together nicely. There is something to be learned from a project that releases stable production versions every few days, even taking the time to go through a few large refactorings, and still manages not to break its thousands of clients.

- I can mix-and-match libraries to get my own ideal environment. Libraries are built to work together by default. I'm now switching a small testing project from Cucumber-RSpec+Webrat+Mechanize to Cucumber+RSpec+Capybara+Culerity+Mechanize, so that I can test simpler browser interactions in-process and switch to a real browser when I test JavaScript. The community is relentlessly looking for ways to make my life easier.

- I can deploy my Rails application to the cloud with a single command. The server will update all its libraries, tweak my app's configuration to run on its own backend, and generally do everything so that my application just works©.

Nothing of the above is revolutionary. Taken all together, though, these changes are a great display of the power of simplicity, testing, openness and relentless experimentation. In fact, I think that the development community at large should look hard at the Ruby community for inspiration. Sure, you can wait for the better stuff to make its way to your environment of choice - but no matter how good those efforts, you'll be lagging a few steps behind. If you fancy to be in the place where things happen, there is no current replacement for the Ruby community.

In a world where a standard can take years to be discussed, approved, implemented and supported, Ruby standards such as Rack skip from conception to widespread support at blazing speed. Every time I take a peek, the New Ruby Ecosystem seems to have reinvented itself in many different ways. This is inspect-and-adapt taken to new heights, and a wonderful showcase of the power of emergent design over big up-front design.

Final disclaimer: As usual, cutting-edge Ruby is not for the faint of heart. In particular, using Ruby on Windows still gives me more headaches than I'd expect, so I'd hesitate to suggest some of the above tools to customers still deeply mired in Windows. If you use the beta versions of all tools, expect your own share of bumps in the road and frustrating error messages. It will take a few more months until Ruby 1.9, Rails 3, Bundler, RSpec, Cucumber, Heroku and all the related technologies work together without a hitch.

I know I'll be waiting. With a big smile on my face.

Wednesday, March 10, 2010

Big in Japan

I was stunned and delighted to see Metaprogramming Ruby topping the Amazon.co.jp chart for English-language computer books. At one point, Amazon.co.jp even listed Metaprogramming Ruby amongst the 50 best selling English books overall! I could get a good sight of Dan Brown and Harry Potter from there (not to mention a lot of Michael Jackson biographies). It didn't last long, but it was nice to hang around with the big boys for a few hours.

As a first-time author, I couldn't have asked for more. To all the people in Japan who bought the book: thank you! Stay tuned, because there might be more news for you in a few months...

Saturday, January 9, 2010

Bill is back, in article form!

Those of you who read the beta version of Metaprogramming Ruby already know Bill, the grumpy old Ruby mentor. If you didn't get enough of him yet, here's a treat for you.

The January 2010 issue of PragPub magazine contains a short article where Bill lectures you about the ins and outs of Ruby's nil value. In this article, I also introduce a couple of spells that fell on the cutting room floor as I struggled to make the book shorter: the Null Object spell and the Black Hole spell.

You can download PragPub for free (in formats suited for Kindles, iPhones and plain old-fashioned computers). Enjoy!