Precisely: better assertions for Python tests

Precisely allows you to write precise assertions so you only test the behaviour you're really interested in. This makes it clearer to the reader what the expected behaviour is, and makes tests less brittle. This also allows better error messages to be generated when assertions fail. Inspired by Hamcrest.

For instance, suppose we want to make sure that a unique function removes duplicates from a list. We might write a test like so:

from precisely import assert_that, contains_exactly

def test_unique_removes_duplicates():
    result = unique(["a", "a", "b", "a", "b"])
    assert_that(result, contains_exactly("a", "b"))

The assertion will pass so long as result contains "a" and "b" in any order, but no other items. Unlike, say, assert result == ["a", "b"], our assertion ignores the ordering of elements. This is useful when:

• the ordering of the result is non-determistic, such as the results of SQL SELECT queries without an ORDER BY clause.
• the ordering isn't specified in the contract of unique. If we assert a particular ordering, then we'd be testing the implementation rather than the contract.
• the ordering is specified in the contract of unique, but the ordering is tested in a separate test case.

When the assertion fails, rather than just stating the two values weren't equal, the error message will describe the failure in more detail. For instance, if result has the value ["a", "a", "b"], we'd get the failure message:

Expected: iterable containing in any order:
  * 'a'
  * 'b'
but: had extra elements:
  * 'a'

Installation

pip install precisely

API

Use assert_that(value, matcher) to assert that a value satisfies a matcher.

Many matchers are composed of other matchers. If they are given a value instead of a matcher, then that value is wrapped in equal_to(). For instance, has_attrs(name="bob") is equivalent to has_attrs(name=equal_to("bob")).

• equal_to(value): matches a value if it is equal to value using ==.

• has_attrs(**kwargs): matches a value if it has the specified attributes. For instance:

  assert_that(result, has_attrs(id=is_instance(int), name="bob"))

• has_attr(attribute_name, matcher): matches a value if it has the specified attribute. Using has_attrs is generally considered more idiomatic when the attribute name is constant. For instance, instead of:

  assert_that(result, has_attr("id", is_instance(int)))

  use:

  assert_that(result, has_attrs(id=is_instance(int)))

• contains_exactly(*args): matches an iterable if it has the same elements in any order. For instance:

  assert_that(result, contains_exactly("a", "b"))
  # Matches ["a", "b"] and ["b", "a"],
  # but not ["a", "a", "b"] nor ["a"] nor ["a", "b", "c"]

• is_sequence(*args): matches an iterable if it has the same elements in the same order. For instance:

  assert_that(result, is_sequence("a", "b"))
  # Matches ["a", "b"]
  # but not ["b", "a"] nor ["a", "b", "c"] nor ["c", "a", "b"]

• includes(*args): matches an iterable if it includes all of the elements. For instance:

  assert_that(result, includes("a", "b"))
  # Matches ["a", "b"], ["b", "a"] and ["a", "c", "b"]
  # but not ["a", "c"] nor ["a"]
  assert_that(result, includes("a", "a"))
  # Matches ["a", "a"] and ["a", "a", "a"]
  # but not ["a"]

• all_elements(matcher): matches an iterable if every element matches matcher. For instance:

  assert_that(result, all_elements(equal_to(42)))
  # Matches [42], [42, 42, 42] and []
  # but not [42, 43]

• is_mapping(matchers): matches a mapping, such as a dict, if it has the same keys with matching values. An error will be raised if the mapping is missing any keys, or has any extra keys. For instance:

  assert_that(result, is_mapping({
      "a": equal_to(1),
      "b": equal_to(4),
  }))

• mapping_includes(matchers): matches a mapping, such as a dict, if it has the same keys with matching values. An error will be raised if the mapping is missing any keys, but extra keys are allowed. For instance:

  assert_that(result, mapping_includes({
      "a": equal_to(1),
      "b": equal_to(4),
  }))
  # Matches {"a": 1, "b": 4} and {"a": 1, "b": 4, "c": 5}
  # but not {"a": 1} nor {"a": 1, "b": 5}

• anything: matches all values.

• is_instance(type): matches any value where isinstance(value, type).

• all_of(*matchers): matches a value if all sub-matchers match. For instance:

  assert_that(result, all_of(
      is_instance(User),
      has_attrs(name="bob"),
  ))

• any_of(*matchers): matches a value if any sub-matcher matches. For instance:

  assert_that(result, any_of(
      equal_to("x=1, y=2"),
      equal_to("y=2, x=1"),
  ))

• not_(matcher): negates a matcher. For instance:

  assert_that(result, not_(equal_to("hello")))

• starts_with(prefix): matches a string if it starts with prefix.

• contains_string(substring): matches a string if it contains substring.

• greater_than(value): matches values greater than value.

• greater_than_or_equal_to(value): matches values greater than or equal to value.

• less_than(value): matches values less than value.

• less_than_or_equal_to(value): matches values less than or equal to value.

• close_to(value, delta): matches values close to value within a tolerance of +/- delta.

• has_feature(name, extract, matcher): matches value if extract(value) matches matcher. For instance:

  assert_that(result, has_feature("len", len, equal_to(2)))

  For clarity, it often helps to extract the use of has_feature into its own function:

  def has_len(matcher):
      return has_feature("len", len, matcher)
  
  assert_that(result, has_len(equal_to(2)))

• raises(matcher): matches value if value() raises an exception matched by matcher. For instance:

  assert_that(lambda: func("arg"), raises(is_instance(ValueError)))

Alternatives

PyHamcrest is another Python implemention of matchers. I prefer the error messages that this project produces, but feel free to judge for yourself:

# Precisely
from precisely import assert_that, is_sequence, has_attrs

assert_that(
    [
        User("bob", "[email protected]"),
        User("jim", "[email protected]"),
    ],
    is_sequence(
        has_attrs(username="bob", email_address="[email protected]"),
        has_attrs(username="jim", email_address="[email protected]"),
    )
)

# Expected: iterable containing in order:
#   0: attributes:
#     * username: 'bob'
#     * email_address: '[email protected]'
#   1: attributes:
#     * username: 'jim'
#     * email_address: '[email protected]'
# but: element at index 0 mismatched:
#   * attribute email_address: was '[email protected]'

# Hamcrest
from hamcrest import assert_that, contains, has_properties

assert_that(
    [
        User("bob", "[email protected]"),
        User("jim", "[email protected]"),
    ],
    contains(
        has_properties(username="bob", email_address="[email protected]"),
        has_properties(username="jim", email_address="[email protected]"),
    )
)

# Hamcrest error:
# Expected: a sequence containing [(an object with a property 'username' matching 'bob' and an object with a property 'email_address' matching '[email protected]'), (an object with a property 'username' matching 'jim' and an object with a property 'email_address' matching '[email protected]')]
#      but: item 0: an object with a property 'email_address' matching '[email protected]' property 'email_address' was '[email protected]'