why spy?

I did learn the concept of spy from Ash, (sorry that I am
too lazy to find the exact tweet right now) after realizing
what it is trying to accomplish, I did implement something
similar in muack, as I feel it might be useful in some cases.

At first I didn't use it personally, because mostly I am
using bacon to test my codes, which I could define nested
context and shared context, making setup quite easy.

*

Before jumping into details, let me clarify what I think
spies could bring us:

* We could always stub the stuffs and then only examine
it later if we're interested. We would not be forced to
put assertion right into setup method/block.

* We could put all assertions in the end of operations,
not the begin of operations. So all the assertions
could be put in one place.

In the first point, that means we don't have to care what
we're going to test, we can simply put all stubs into the
setup block.

The advantage of this is that we don't have to think too
much whenever we're writing the setup block, and we could
share the setup block between tests, as we don't make
assertions there.

If we're using mocks instead, then we're putting assertions
right into setup block, so that this block can't really be
shared between tests.

The disadvantage is, slower tests, as we're running some
codes which we might never be interested.

This is more an implementation advantage than design advantage.

*

The second point is, we would be very clear about what
we're really testing as we put all assertions in one place
instead of scattering around due to the way we define mocks.

This would be more a design (or readability) advantage.

*

In bacon, since we have nested and shared context, the first
advantage does not really matter because bacon is flexible
enough to do whatever we want to setup.

But in an test/unit based testing framework, or to be more
precise, a testing framework without nested and shared context,
it would be a good advantage to workaround this shortness.

*

ok, let's see what I meant by nested context in bacon:

require 'bacon'
Bacon.summary_on_exit

describe Bacon do
  before do
    @a = 0
  end

  should 'have @a' do
    @a.should == 0
  end

  describe '@b' do
    before do
      @b = 0
    end

    should 'have @a and @b' do
      @a.should == 0
      @b.should == 0
    end
  end

  should 'have no @b' do
    instance_variable_defined?(:@b).should == false
  end
end

By running this, bacon would print:

Bacon
  - should have @a
  @b
    - should have @a and @b
  - should have no @b

3 specifications (4 requirements), 0 failures, 0 errors

That means, I could easily reuse the before block to setup
the stuffs I *exactly* need. This is somehow hard in a test/unit
based testing framework, e.g. the one Rails uses by default.

If we want to do that kind of stuffs in a testing framework
without nested context, we might need to extract those setup
methods into normal methods, and call them accordingly in the
setup method (in pure test/unit) or in a setup block (with Rails
extension), splitting one test case into multiple test cases.

Something like this:

require 'test/unit'

class TestA < Test::Unit::TestCase
  def setup
    @a = 0
  end

  def test_have_a
    assert_equal 0, @a
  end
end

class TestB < TestA
  def setup
    super
    @b = 0
  end

  def test_have_a_and_b
    assert_equal 0, @a
    assert_equal 0, @b
  end
end

By running this in Ruby 2.0, the result is:

Loaded suite -
Started
..

Finished in 0.001353 seconds.

2 tests, 3 assertions, 0 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications
100% passed

1478.20 tests/s, 2217.29 assertions/s

You might argue that this is not actually harder to do,
and it's even more clear what's really happening, as
we don't even have to show that there's no @b defined
in TestA since it's obvious.

However this has some disadvantages. First we're forced to
split related tests from a class to another class. We can't
easily tell from this example, but I think we tend to put
tests together because of functionality similarity instead
how we setup the stubs.

Of course they are related, so sometimes they are the same
thing. But suppose we're testing a function with correct
and incorrect fake input, we might really need different
setup/before method/block for testing the same function.

And, this doesn't work in Rails' controller tests. I am
not too surprised for this... Rails always has their weird
hacks so that normal Ruby can't really work.

On the other hand, if we want to do something like this in
bacon, we have shared context. Here's an example:

require 'bacon'
Bacon.summary_on_exit

shared :a do
  before do
    @a = 0
  end
end

shared :b do
  before do
    @b = 0
  end
end

describe '@a' do
  behaves_like :a

  should 'have @a' do
    @a.should == 0
  end
end

describe '@b' do
  behaves_like :a, :b

  should 'have @a and @b' do
    @a.should == 0
    @b.should == 0
  end
end

It's more like mixin instead of inheritance, making reuse
much more easier.

What My points are...

Since bacon is much more flexible (and also much faster) than
test/unit, some advantages of using spies are gone. i.e. The
first point I was referring about, that we could simply stub
everything without thinking too much.

If we have flexible setup methods, we could still manage to
only setup what we want in each test cases without splitting
them into different places.

*

I only realized this whenever I was writing a bunch of controller
tests in Rails. Now I wonder if I should try to run bacon in
Rails, then I don't have to suffer from this pain.

However I might not want to do this by myself, since integrating
with Rails is really a PITA as we would definitely touch the
internal details of eternal weirdness of Rails...



In the last, let me show you my current muack spies in Rails
controller tests: (slightly edited though)

# I have no interested in adding test directory into $LOAD_PATH
require_relative '../test_helper'

class UsersControllerTest < ActionController::TestCase
  setup do
    any_instance_of(User) do |u|
      # we don't care if the user processes or not
      stub(u).process.proxy
    end
  end

  def spy_user_process
    any_instance_of(User) do |u|
      # now we care if the user has processed
      spy(u).process
    end
  end

  def test_process
    sign_in(user)

    post :process, :id => user.to_param
    assert_redirected_to dashboard_path

    spy_user_process
  end

  # other tests for process which should call User#process
  # they need to insert `spy_user_process` in the end

  def test_show
    sign_in(user)

    get :show, :id => user.to_param
    assert_response :success

    # as we don't care if the user has processed or not,
    # we don't spy in the user stub.
  end
end

If I were writing in bacon, maybe I'll write it this way:

require_relative '../test_helper'

describe UserController do
  should 'show' do
    sign_in(user)

    get :show, :id => user.to_param
    response.status.should == 200
  end

  describe 'process' do
    before do
      any_instance_of(User) do |u|
        # process should be called once
        mock(u).process.proxy
      end
    end

    should 'process successfully' do
      sign_in(user)

      post :process, :id => user.to_param
      response.redirected_to.should == dashboard_path

      # we don't have to spy here because mock would
      # check for us. however we could still do it the
      # spy way if we want.
    end

    # other tests for process which should call User#process
  end
end

Actually even if we have spies, the bacon one is still
better, as we don't have to duplicate the line we're
dispatching spies in other tests! (i.e. spy_user_process)

Or you might think it's still better to explicitly say
what we're testing for... well, at least we have choices
in bacon :P