Single transaction approach performance in tests

When your project is big enough and amount of tests becoming bigger and bigger the speed of test execution becomes very low.

I had 100+ tests in my project. In my local machine it takes about 20 minutes to run all tests on amazon EC2 small instance it was even slower.

So I started to think wooow why it’s so SLOOOOOW. Actually I knew the answer and I think you also should have a hunch that fixture loading is the bottleneck here.

There are a few approaches to speed up fixture loading, that I know

  • use memory tables
  • single transaction approach

As for memory tables. So the idea is use memory tables Well I see some big disadvantages. First it’s not a copy of you database, i.e. you don’t have foreign key checks, you don’t have some abilities that your tables could have. Secondly this is useful to only some specific databases as mysql.

So we will talk about second approach. The idea is to begin transaction before each test and rollback it after each test. This should be much faster than reloading data each time.

Looking ahead, I will say that after the implementation of this approach speed execution became 40 second versus 20 minutes for 102 tests.

But what is bottlenecks here and what should you do to use it. For me it were 2 suddenness.

The main problem for me was following. I had functional and unit tests. For unit everything was perfect but for functional a half of tests were broken. What happened? Answer is easy. Functional tests tests real http request and the client don’t know anything about you transaction. So we need to run single transaction too on clients. Well ok for tests it’s quite easy to do. For example yii even uses specific entry script for test index-test.php, or you can pass some secret header.

if (APPLICATION_ENV != 'production' && $anyAditionalCheckThatYouNeedHere) {
    $config = require $pathToMyConfigFile;
    $config['onBeginRequest'] = function() {
        // use single transaction approach for tests. Begin but do not commit.
        \Yii::app()->getDb()->beginTransaction();
    };
}
This is not the end such test also won’t work 🙂
    $testUser = $this->users('serg');
    $result = $this->put("/my/api/{$testUser->id}", $data);

    $this->assertEquals($expectedId, $result['id']);

    // oh no my dear 😀
    $testPatient->refresh();

This means that in functional tests you must test only interface do not check that DB changes, check your response what it has, what it should have. Check all DB changes in unit tests.

Implemented this approach I’m really happy to see how fast tests execute.

Leave a Reply

Your email address will not be published. Required fields are marked *