For a long time javascript was something I avoided like the plague, usually only ever handling with the rubber gloves that rails provides. On a few occasions I had written some raw javascript myself and quite frankly I felt dirty afterwards and started hitting the bottle rather hard. I was writing clumsy procedural code, verbose, without unit test and working round all sorts of quirks in the language. Sometimes it was the only choice but whenever I could avoid it, I did.

This year I’ve been doing quite a lot of work in javascript, as part of a supercool project at work. However I am not an alcoholic nor a gibbering wreck. So what happened? In a single word, prototype (note that a lot of what I’m saying probably applies to similar libraries like jQuery, dojo etc… The important stuff isn’t really in the details).

Getting Prototypical

Now I’d heard of prototype before. It was that thing with the $ thing for getting dom elements by id that you had to include for the whizz-bang scriptaculous magic or for using ajax. Thanks to my blind raging hatred of javascript I’d never looked any closer. What a mistake! There is so much more to prototype, something which I discovered thanks to the Bungee Book which is a lovely piece of work (Seriously, stop reading this and go and buy it. It’s great (apart from the index)).

Fun in the Browser

Prototype brings two unrelated (at least at the conceptual level) things together. The first is a whole pile of things that make make the browser environment more palatable. The $ function is but the tip of the iceberg here. Some of my favourites include

• $$ (and Selector class that powers it): find all the elements in the document matching a CSS rule. The great thing here is that it knows a whole pile of tricks, even if you’re browser doesn’t, so you can go from the basic $$(‘p.alert’) (all paragraphs with class alert) to things that are a little more fun:

1

$$('.sources .source:not(.disabled)')

( all elements inside something with class ‘sources’ that have the class ‘source’ but not the class disabled).

• down, next, up etc… Navigating the dom by hand is the biggest pain in the arse since Vlad the Impaler, but these functions make it oh so easy. The up function gives you the parent element, down gives you the first descendant and so on (and of course you can say ‘give me the 3rd such element’). That in itself is a modest improvement in comfort. But the killer blow is that you can pass those lovely css selectors and say things like

1

foo.down('.source:not(.disabled)')

which does exactly what it should: keep going down until you find something that matches that selector. Now think about how you’d do that by hand!

• Event handling: prototype smoothes over all the differences between browsers and makes it really easy to have your own custom events which is great for having page elements react to changes without coupling things too tightly

• Style manipulation: getStyle (gets computed styles), setStyle, hasClassName, removeClassName, addClassName are just terrific. Before

1
2
3

foo.style.left = '8px';
foo.style.right = '8px';
foo.style.top = '8px';

After

1

foo.setStyle({left: '8px', right: '8px', top: '8px'})

There’s loads more of this sort of stuff, the list above is just what sprung to mind. All the way prototype hides the differences between browsers.

Inner Beauty

Great stuff, but it doesn’t address the issues of Javascript’s builtin classes and the language itself. In a way I think this is where prototype really shines, because after a while I couldn’t feel that feeling of self-loathing anymore. I was actually enjoying writing javascript. Part of it is down to the fact that beneath the horribleness of the 90s there’s actually some cool stuff in javascript: closures, functions are objects and so on. The trouble is I’d never noticed because I was so busy vomiting profusely at the rest of it. Prototype made me aware of the nice stuff and tidies up the dog turds (And if you ignore everything I’ve written but remember that javascript has these functional hiding inside then that’s probably the most important thing).

Enough waffling. What does prototype give you in this deparment? - Enumerable. A beast of a module. Mixed in to Array by default, it lets you use each, inject, map almost as if you were writing ruby (there’s some punctuation clumsiness, but that’s not prototype’s fault). If that sounds awfully similar to ruby’s Enumerable that’s because it is - Class. Write classes, subclass them = win - method binding - Loads of little utilities like Object.isFunction that really should have been there all along - Your sanity back.

It’s really a joy the first time you write javascript using all this. Seriously it brought tears to my eyes.

Feeling Testy?

The one bit missing from my workflow was unit testing. I started of playing with jsunit and while it worked fine I wasn’t really enjoying it. There just seemed to be too much machinery involved, and some things seemed needlessly complicated. For example if you want to run a test suite in the browser you have to open the testRunner page, then click on the choosefile button on that page, navigate to your test file and hit run. And because of that you can’t just set a firebug breakpoint. I ended up loading the test file itself in the browser and calling the setup, test and teardown functions from the firebug console. Yuck.

I know a lot of this is built for flexibility, but it was just completely overkill for what I needed. It turns out that prototype itself has a handy little unit testing framework that was much closer to what I was looking for. There’s even the javascript_test plugin which makes all that fit nicely into your rails app (although it would seem that javascript_test hasn’t seen much love - I’ve ended up ripping out most of its innards and replacing them with the lastest stuff from prototype). If you want to run a test suite in your browser, just open the test file! A rake task lets you run them all in one go

Started tests in Firefox
.......................
Finished in 23 seconds.
116 tests, 545 assertions, 0 failures, 0 errors

Finally, peace. Maintainable, structured, nonverbose javascript code, unit tested and running off a CI server. Who would have expected that!

If you ever do this

1

  render :some_partial, :locals => {'foo' => 'bar'}

don’t. String keys in :locals have been deprecated for a while and are no longer supported in edge (and thus in the not too distant 2.1), so go ahead and change them now and you’ll save yourself some grief when you migrate to 2.1.

When things break it’s always nice when they break in an obvious way, so in this case foo not being defined at all in the partial makes it immediately obvious that something has change from :locals. You check the api docs, maybe google around a bit and see that string keys aren’t accepted any more. If you do use a string key like this the foo will still be defined it will just be nil. You probably won’t suspect an api change and spend some time hunting up and down as to why the variable you’re using in :locals is nil (at least that’s what I did - hopefully you won’t have to do the same now!).

I have a confession to make: I really like hospital based tv series: House, ER, Scrubs, Green Wing. It’s all good. Of the lot, House is the one I’ve been watching most recently and in many ways a typical episode is basically a medical murder mystery. Understanding a bug or working out some unexplained behaviour is basically the same thing, it’s just that this time you’re the hardened police officer on the case!

This episode opens up with a strange scene:

1
2
3
4
5
6
7
8
9

>> s = Shift.find :first
=> #<Shift id: 1, start: "2008-04-30 22:19:31", created_at: "2008-04-30 22:19:32", 
updated_at: "2008-04-30 22:19:32">
>> s.start
=> true
>> s.created_at
=> Wed Apr 30 22:19:32 BST 2008
>> s.start
=> Wed Apr 30 22:19:31 BST 2008

Something is clearly wrong, and as the title sequence starts up the viewer knows there’s funny business afoot. In the grand tradition of TV series the next scene doesn’t seem hugely related.

You can’t be serious

A controller with a ‘start’ action was claiming that there was no such action, systematically. This controller had always had such an action and had not been changed in months, I was merely passing it while working on another issue. It was getting late so I called it a day. The next morning, with no code changes, it was fine so I moved on to other things, pushing that niggling feeling you get when you don’t understand a bug to the back of my mind. Then at lunch it stopped working, again without apparent reason.

Time to tackle the problem head on. I fired up my trusty debugger and went stepping through ActionController. When a request comes in and the controller and action have been identified, ActionController (broadly speaking) will try to find a method of that name, a template of the same name or hit method missing. Now there’s a bunch of methods that shouldn’t be exposed as actions so what AC does is subtract from your controller’s public methods all those public methods defined in ActionController::Base (which by definition pretty much can’t be actions). For some reason ActionController::Base seemed to have a method called start and so ‘start’ was on the list of method names that could not be actions. Searching through the source revealed no definition of a start method, but interrogation of Kernel reveals that it defined a start method. I fired up script/console to do some more poking, and interestingly enough, no start method in Kernel. It should be running the same code as the app on the same machine, same everything and yet there’s a difference.

There’s definitely some methodicalness to tracking down a problem. At some point though you’ve go to switch off the guidance computer and just use the Force. If you’re short on Jedi powers (as I am) then you’ve got to settle for second best which is to have an idea. It suddenly occurred to me that the difference was the debugger. When I’d been stepping through the ActionController code it was obviously on but when I’d been pissing about in script/console it hadn’t. So I had a look at the source for ruby debug and sure enough it defines Kernel#start. So the seemingly random factor that was causing the action to fade in and out of existance was whether or not I was debugging something else!

Dénoument

At this point the problem I started with is sort of solved, a quick experiment shows that under the debugger it exhibits the anomalous behaviour and without the debugger it’s fine. But why does the second call to start work?

When you get an ActiveRecord class it doesn’t have your attribute accessors defined. They come into existance the first time you try and use them (via method_missing). So the first time we call start, we’re calling the start method from Kernel.start that ruby-debug created. When we call created_at, method_missing gets hit which goes and generates all the accessors, overwriting the start method from Kernel.start with one that reads the start attribute. Then we call start again and this time we get the freshly generated one method

So there, mystery solved. The credits start rolling. I think this was a pretty good episode, who would have thought it was your trusted lieutenant who’d been stirring up trouble all along!

Epilogue: I now stick

1

Kernel.send :undef_method, :start if Kernel.respond_to? :start

In a file in config/initializers and order is restored to the galaxy.

Post-Epilogue: ruby-debug has since been updated to not define a start method. Still an interesting story if you ask me!

We all love :include, it cuts down on the number of queries made to the database (especially when iterating over a collection) and generally makes the hamsters running the servers happier. It’s not all green pastures though and then eager loading code in Rails up to 2.0.x had some shortcomings (not least that some of it was mind bendingly complicated), so Rails 2.1 has a new eager loading scheme which works in a completely different way.

Consider the following snippet:

1

  Post.find :all, :include => [:author, :comments, :tags]

In Rails 2.0.2, this generates one big query that selects from posts and joins authors, comments and tags. Because of the way the joins work, if a post has 10 comments and 4 tags then that particular post will create 40 rows in the result set (for 16 actual distinct objects) that rails has to filter down. The more has_many associations you have at any single level, the worse it gets (and to top it off the processing of that large result set has an O(n²) element to it).

Build it up, break it down

In Rails 2.1, this is handled completely differently. Instead of building one big query, we can break it down into some easy ones. First a regular Post.find(:all) query is run, which yields a array of posts. Then Rails basically does

1
2
3

  Author.find :all, :conditions => ['id in (?)', posts.collect(&:author_id)]
  Comment.find :all, :conditions => ['post_id in (?)', posts.collect(&:id)]
  Tag.find :all, :conditions => ['post_id in (?)', posts.collect(&:id)]

and hooks up all of those objects to their respective owners. So you execute more queries, but those queries are easy (both for the db and for rails) and the number of queries relates to the number of associations you are getting, not the number of rows. They also play nice with things like Active Record Context. You could eager load 10 top level associations and it would be just 10 simple queries, rather than a complicated join on 10 tables. This approach also makes it a lot easier to handle associations with conditions, polymorphic belongs_to, associations with orders and so on.

Conditional confusion

So a lot of rambling and nothing very relevant to the title! Here it comes. Suppose comments have a moderated attribute, and we wish to defeat the spam bots by only showing moderated comments. With the :include in Rails 2.0.2 you can do something like this

1

  Post.find :all, :include => [:comments], :conditions => ['moderated = ?', true]

Easy, get all the posts but only get moderated comments. Except this doesn’t work. Sure it only gives you back moderated comments, but it also only returns posts with at least one moderated comment so that article you just posted isn’t ever going to get read because it has no comments. Oh noes! But why? Recall the giant sql statement, returning many rows. If we have some articles, one with a moderated comment and one with none, our result set (ignoring for a second the conditions) will look something like

So when our conditions are applied, they’ll knock out the rows for our first and third blog post. See, I wasn’t even making that shit up!

So mixing :conditions and :include has dubious semantics at best. It should be immediately apparent that this cannot work as is with the new :include scheme since there is no single query where both the posts and comments tables are loaded. It would however be possible to massage :conditions into only applying to the query we wanted (in this case the load from comments). For now, we can solve the original problem by adding a moderated_comments association (with a condition) to posts, and :including it. With the new eager loading scheme that will accomplish our original goal.

But I liked the old way!

If you do rely on the old behaviour, fret not: ActiveRecord will fall back to the old implementation if it notices that you are referencing tables other than the ‘main’ table. You’ve got to give ActiveRecord a hand though, by disambiguating your column names. In the example given previously, ActiveRecord wouldn’t pick up on it, and you’d get a nasty error back from your database.

1
2

  Post.find :all, :include => [:comments],
                  :conditions => ['comments.moderated = ?', true]

will make it clear to AR that it needs to fallback to the old code.

So there you have it. You now know everything about eager loading in ActiveRecord, both old and new! Use it wisely.