New in elasticsearch 0.19 is the ability to batch searches together via the multi search api. As far as I know this doesn’t result in elasticsearch doing any less work (unlike sphinx’s multi query stuff which is able to work out when your batch contains common sub expressions and stuff like that) but it does cut out a lot of latency.

I was motivated to poke at this by some work I was doing at dressipi: a particular page required nearly 20 (similar but distinct) search queries to be run. The individual queries were quick, around 10-15ms each but repeat 20 times and this was adding up to around 250ms (and this was on my dev machine so very low latency between my app on elasticsearch). Once I reimplemented it to use multisearch it was down to about 30-40ms for the same batch of 20 queries.

Tire doesn’t directly support multi search, but it’s pretty easy to implement it. To execute a multi search you need the json representation of each individual query, along with a json object that represents query options (which index to search on, routing etc.) that would have be in the url in a normal search query. Join these altogether with “\n” in between (and at the end) and post it to the _msearch endpoint.

Elasticsearch will return a json array with the responses in the same order as your queries were. Tire’s search object already know how to serialise themselves as json, so all that is needed is to string them together. We’ve been using something like this gist. It adds a new multi search object that you initialize with a collection of Tire::Search::Search objects. For example

1
2
3
4
5
6
7
8
9
10
11
12

searches = []
searches << (Tire::Search::Search.new('garments', :type => 'garment') do
  query do
    text :_all, 'red dresses'
  end
end)
searches << (Tire::Search::Search.new('garments', :type => 'garment') do
  query do
    text :_all, 'blue dresses'
  end
end)
results = Tire::Search::MultiSearch.new(searches).results

Will execute a single http query that will do two searches. results[0] is a tire results collection of garments matching “red dress” and results[1] is a collection of blue dresses. You can set whether to load the corresponding model objects either when you create the Search objects or when you create the multi search:

1

results = Tire::Search::MultiSearch.new(searches, :load => true).results

You can’t currently set routing, preference or search_type - those have to be set in the per-search header. Hopefully I’ll get some time over the next few days to contribute this back to tire with tests etc. but you should just be able to drop this in your app and go.