Type Slowly

Testing Sunspot with Test::Unit

So, after singing the praises of Cucumber at LRUG last year, I’ve actually become a bit more ascetic with my testing practices of late, and am using Test::Unit for everything, including acceptance testing (albeit with a layer of contest and stories sugar on top). I could write about why I chose to do this in detail, but it really comes down to one thing, which is ‘speed’. I’ve been programming in Ruby for years, and to be honest, I can spec out a bit of behaviour faster in Ruby than I can in cucumber’s DSL. However, since most of the Rail community seem to be using RSpec and Cucumber, this can make some things a bit of a chore, as with some fairly common or general tasks, you have to go it alone, rather than just grabbing someone else’s solution.

A case in point: I’ve been working on getting full-text search (powered by Sunspot and Solr) working with a site I’m working on. I’ve been really impressed by how easy this is - except for when it came to getting my tests working nicely with it. The problem here is that the Solr service needs to be running in order for the search functionality to work,  but if it’s running for all your tests it slows everything down massively. Clearly, I need a way to only run Solr for those tests that need it, leaving the others to run without it. Since all my test cases are just plain ruby classes (this is Test::Unit), the idiomatic way of doing that seems like a mixin to me, and here it is: 

 module TestSunspot
  class << self
    attr_accessor :pid, :original_session, :stub_session, :server
    
    def setup
      TestSunspot.original_session = Sunspot.session
      Sunspot.session = TestSunspot.stub_session = Sunspot::Rails::StubSessionProxy.new(Sunspot.session) 
    end

  end
  def self.included(klass)
    klass.instance_eval do
      def startup
        Sunspot.session = TestSunspot.original_session
        rd, wr = IO.pipe
        pid = fork do
          STDOUT.reopen(wr)
          STDERR.reopen(wr)
          TestSunspot.server ||= Sunspot::Rails::Server.new
          begin
            TestSunspot.server.run
          ensure
            wr.close
          end
        end
        TestSunspot.pid = pid
        ready = false
        until ready do
          ready = true if rd.gets =~ /Started\ SocketConnector/ 
          sleep 0.5
        end
        rd.close
      end

      def shutdown 
        Sunspot.remove_all!
        Process.kill("HUP",TestSunspot.pid)
        Process.wait
        Sunspot.session = TestSunspot.stub_session
      end
    end
    def teardown
      Sunspot.remove_all!
    end
  end
end

We use this by calling TestSunspot.setup at the time our testsuite starts running, then including TestSunspot into any test cases that require Sunspot’s functionality.

Let’s have a closer look at what’s going on.

TestSunspot.setup saves the original sunspot session, and replaces it with a new stubbed session - this ensures that when CRUD operations on models that are indexed are called, they don’t fail as the server isn’t running. We could of course run the server for every test case, but this’ll slow down our test suite due to all the unnecessary updating of the index that’s going on. It’s not terrifically good practice either. The whole point of Unit tests is that they test Units of your codebase in isolation, and having all these tests depend on this massive cross-cutting concerns is both inelegant and almost certain to cause you headaches.

With that done, we don’t need to worry about sunspot interfering in our other tests, but we still haven’t solved the problem of how to get our tests that are actually concerned with searching to run Sunspot properly.

This is where the self.included method definition comes in. This allows us to execute an arbitrary block of code when the module is mixed into a class, and in our case, we’re using instance_eval to define two new class methods, startup and shutdown, which, as the Test::Unit documentation will tell you, are run once at the beginning of the set of tests defined in that class, and once at the end, respectively. This gives us our hooks to startup and shutdown the sunspot server, which is where things get a bit tricky.

First of all, we need to replace the current sunspot session with the ‘real’ one we stubbed out earlier. In addition, we need to start Sunspot in a different thread or process, as we obviously don’t want it to block our tests from executing. However, it takes a while to start up, and we need to delay executing our tests until it’s ready. So, we fork a new process to run Sunspot (storing the PID so that we can shut it down later), but we redirect it’s STDOUT and STDERR into a pipe. Then, the other end of the process polls the other end of this pipe, waiting for the message “Started SocketConnector” (which is helpfully  output by Sunspot when it’s ready) to appear, at which point it stops blocking and we can run the tests. In our shutdown callback we clear the index, terminate the sunspot process (waiting for it to close), and switch back to our stubbed session, so that any successive tests will run as normal.

This is a bit of a hack (in particular the STDERR redirecting business), but it’s working reliably for me, and offers a clean simple interface to testing search functionality in my application.