Testing

General mechanisms used by git-ubuntu tests

We use pytest. On Ubuntu, with dependencies installed, it should be sufficient to just run py.test-3 at the top level to run the tests locally. In CI, we additionally use pylint.

We ship the test suite and this is available from the snap as git-ubuntu.self-test. This is done by CI prior to publication of the snap.

You can use git-ubuntu.self-test from the snap to test a local development tree with dependencies from the snap. TBC: how.

Many tests are parameterized using pytest’s parametrize mechanism.

General approach to testing

We try to test what is most likely to fail.

For example, testing that we call an API in the expected way when the code that does it is straightforward is of limited value if the risk is that we’ve misunderstood the API and are calling it wrong.

Most code should come with tests. Some old code doesn’t have tests; we are trying to improve this incrementally by writing tests in areas of code that we touch. Don’t feel that you have to go down the rabbit hole. Landing a simple fix doesn’t require that you raise the quality of everything you touch, but some incremental improvement is appreciated such that it’s possible to say that the quality is higher than it was previously.

git-ubuntu operations often operate on inputs that are git repositories or source packages, and then outputs the same. These inputs and outputs are fundamentally data structures just as a simple list is a data structure. We might test a function that operates on a list by providing sample lists as inputs and making assertions about the output. Similarly, most git-ubuntu tests provide sample repositories and source packages as inputs, and make assertions about output repositories.

Providing an input repository or input source package is a tedious and repetitive task, so it is automated. Input source repositories can be specified as data structures by using the gitubuntu.source_builder module. Likewise, the gitubuntu.repo_builder module allows git repositories to be specified as data structures. Where an input git repository needs a source package in a tree object, gitubuntu.repo_builder.SourceTree can be used to construct it.

Please avoid checking in already constructed git repositories or source trees as test inputs. These will inevitably contains binary blobs and/or boilerplate. These are opaque and hide the important part of data structure that actually matter to the tests, making such tests harder to work with. Instead, the “builder” style described above eliminates binary blobs and, with default arguments, the boilerplate. Where edge case inputs are required that are not already supported by the builder modules, the builder modules themselves should be enhanced instead. And if the way the builder modules are called become obtuse, then they should be refactored together with the tests.

Commonly used fixtures

git-ubuntu tests make extensive use of pytest’s fixture functionality.

You should make yourself familiar with the pytest built-in tmpdir fixture. Note that it supplies a py.path object, not a string. In some cases, this needs explicit conversion with str().

git-ubuntu uses the pygit2 library extensively. However git-ubuntu repositories have a particular structure for which we have many helper methods so we wrap the pygit2.Repository in a GitUbuntuRepository. The underlying pygit2.Repository object is available via the gitubuntu.git_repository.GitUbuntuRepository.raw_repo attribute on a GitUbuntuRepository instance.

The gitubuntu.test_fixtures.repo() fixture provides a gitubuntu.git_repository.GitUbuntuRepository instance in a temporary directory. Alternatively you can use the lower level gitubuntu.test_fixtures.pygit2_repo() fixture to get an unwrapped pygit2.Repository instance in a temporary directory.

gitubuntu.repo_builder operates on pygit2.Repository objects directly, so tests can either use a gitubuntu.test_fixtures.pygit2_repo() fixture directly, or they can use a gitubuntu.test_fixtures.repo() fixture and access its gitubuntu.git_repository.GitUbuntuRepository.raw_repo attribute to use with gitubuntu.repo_builder.Repo.

Example

Let’s say you want to test a function that looks up the package version string from debian/changelog in a git tree object:

def test_changelog_version_lookup(repo):
    # repo is our test fixture. When pytest runs this test, it will supply
    # it with an empty GitUbuntuRepository instance in a temporary
    # directory.

    # First we describe a git repository in a data structure and ask it to
    # write itself into repo.raw_repo.
    repo_builder.Repo(
        commits=[
            # We give the commit a name here, purely for the internal
            # reference that comes later
            repo_builder.Commit(name='root', tree=repo_builder.SourceTree())
        ],
        # The Placeholder() instance is used to internally reference the
        # commit we created earlier
        tags={'root': repo_builder.Placeholder('root')},
    ).write(repo.raw_repo)
    # In the line above, raw_repo is the underlying pygit2.Repository
    # instance

    # Now our git repository contains a commit that contains a source
    # package tree and a tag that refers to the commit.

    # We can fetch the tree object using normal pygit2 operations
    tree = (
        repo
        .raw_repo.lookup_reference('refs/tags/root')
        .peel(pygit2.Tree)
    )

    # Now we can test the our code in the normal way.
    # repo_builder.SourceTree() above defaults to a version of '1-1' and we
    # didn't override it, so that's the version string that our
    # function-under-test should return.

    actual_result = repo.get_changelog_versions_from_treeish(str(tree.id))
    assert actual_result == '1-1'