Tests
=====

Efficient and thorough testing is key to ensure bpfilter is stable and reliable. As manual testing is time-consuming and error-prone, bpfilter relies on automated tests to validate its features. Different type of tests are implemented to cover all the components of the project:

- Unit tests to test every function part of the public API of ``libbpfilter``.
- End-to-end tests to validate ``bfcli``'s language, and the behaviour of the BPF programs generated by ``bpfilter``.
- Integration tests to ensure ``libbpfilter`` and the core module can be integrated into other projects without issues.
- Check tests to validate code style and quality.

bpfilter uses CMake and CTest to manage and run its tests, a test suite is defined for each type of test. You can run the tests using:

.. code-block:: shell

    # Run all the tests
    make -C $BUILD_DIR test_bin test

    # Run a specific test suite (unit, e2e, integration, check)
    make -C $BUILD_DIR $TEST_SUITE

    # Run a single test: use the test's path under tests/, replacing / with .
    # e.g. tests/e2e/matchers/ip4_daddr.cpp becomes e2e.matchers.ip4_daddr
    ctest --test-dir $BUILD_DIR -R $TEST_NAME --output-on-failure

Each test suite has a corresponding ``make`` target to build its binaries (e.g. ``make -C $BUILD_DIR test_bin`` builds all test binaries). By default, CTest doesn't build the test binaries when running tests.

Test coverage is disabled by default, to enable it, configure CMake with the ``-DWITH_COVERAGE=1`` flag. Then, you can generate a coverage report using: ``make -C $BUILD_DIR coverage doc``.


Unit tests
----------

Unit tests are limited to the libbpfilter library. They are designed to validate the different components of the library in isolation, using mocks and fakes when necessary. We expect every public function of the library to be covered by at least one unit test.

**Adding a new unit test**

1. Create a new source file to contain the unit test under ``tests/unit``. You can use existing unit tests as examples. Usually, a source file in ``tests/unit`` will contain tests for a single source file in ``libbpfilter``.
2. Add the new source file to the ``tests/unit/CMakeLists.txt`` file using the ``bf_add_c_test`` function.
3. Implement the unit tests using the ``cmocka`` testing library. You can use existing unit tests as examples.

CMocka supports test fixtures to setup and teardown common test data. You can use them to avoid code duplication when multiple tests require the same setup.

**Example**

The example below is used to test the ``bf_version()`` function from ``libbpfilter``. All the tests source file must include the ``test.h`` header, define a main function, and register the tests to be run.

.. code-block:: c

    /* SPDX-License-Identifier: GPL-2.0-only */
    /*
    * Copyright (c) 2023 Meta Platforms, Inc. and affiliates.
    */

    #include <bpfilter/version.h>

    #include "test.h"

    static void get_version(void **state)
    {
        (void)state;

        assert_non_null(bf_version());
    }

    int main(void)
    {
        const struct CMUnitTest tests[] = {
            cmocka_unit_test(get_version),
        };

        return cmocka_run_group_tests(tests, NULL, NULL);
    }


End-to-end tests
----------------

End-to-end tests are designed to validate bpfilter's behaviour as seen by the user. They cover the entire stack, from ``bfcli`` to the BPF programs generated by ``bpfilter``. End-to-end tests ensure that the filtering rules defined using ``bfcli`` are correctly translated into BPF programs, and that these programs behave as expected when executed.

**Adding a new end-to-end test**

1. Create a new source file to contain the end-to-end test under ``tests/e2e``.
2. Call ``bf_add_e2e_shell_test()`` in ``tests/e2e/CMakeLists.txt`` to add the new source file to the end-to-end test suite.
3. In your test, use the functions provided by ``e2e_test_util.sh`` to setup the test environment. Any shell command failure will stop the test immediately and return a failure.

**Example**

``e2e_test_util.sh`` provides functions to create a sandboxed environment. Here is an example of a simple end-to-end test that creates a sandbox:

.. code-block:: shell

    #!/usr/bin/env bash

    set -eux
    set -o pipefail

    . "$(dirname "$0")"/../e2e_test_util.sh

    make_sandbox

    # Ping the sandbox's IPv4 address from the sandboxed namespace
    ${FROM_NS} ping -c 1 -W 0.1 ${NS_IP_ADDR}


Matcher tests
~~~~~~~~~~~~~

Matcher tests (``e2e.matchers.*``) validate that the BPF programs generated by ``bpfilter`` correctly match packets. They use ``BPF_PROG_TEST_RUN`` to run the generated programs against crafted packet buffers in kernel space, without requiring network namespaces or real traffic.

Each matcher test is a C++ binary located under ``tests/e2e/matchers/`` that builds chains with specific matchers, crafts packets using the ``bft::Packet`` builder, and asserts the BPF program's verdict using ``bft_assert_prog_run()``. Tests are automatically parameterized across all hooks that support the matcher under test using the ``MatcherTestsSuite`` class.

**Adding a new matcher test**

1. Create a new C++ source file under ``tests/e2e/matchers/`` (e.g. ``tests/e2e/matchers/my_matcher.cpp``).
2. Add a call to ``bf_add_e2e_c_test()`` in ``tests/e2e/CMakeLists.txt`` with your new test.
3. Use ``bf::Chain``, ``bf::Rule``, ``bf::Matcher`` from ``tests/harness/`` to build chains, and ``bft::Ethernet``, ``bft::IPv4``, ``bft::TCP``, etc. from ``tests/harness/Packet.hpp`` to craft packets.
4. Use ``MatcherTestsSuite`` from ``tests/harness/test.hpp`` to automatically run the test across all supported hooks.

**Example**

.. code-block:: cpp

    #include "Chain.hpp"
    #include "Matcher.hpp"
    #include "Rule.hpp"
    #include "test.hpp"

    extern "C" {
    #include <bpfilter/bpfilter.h>
    }

    static void my_matcher_eq(void **state)
    {
        auto *test = static_cast<MatcherTest *>(*state);

        BFT_CHAIN_SET(
            bf::Chain("test_chain", test->hook(), BF_VERDICT_ACCEPT)
            << bf::Rule(BF_VERDICT_DROP, bf_counter(), {},
                        {bf::Matcher(BF_MATCHER_MY_TYPE, BF_MATCHER_EQ, {42})}));

        // Packet matching the rule -> DROP
        bft_assert_prog_run(
            "test_chain", test->hook(),
            bft::Ethernet() /
                bft::IPv4 {.saddr = "127.0.0.1", .daddr = "127.0.0.2"} /
                bft::TCP {.dport = 42},
            test->verdictDrop());

        // Packet not matching -> ACCEPT (policy)
        bft_assert_prog_run(
            "test_chain", test->hook(),
            bft::Ethernet() /
                bft::IPv4 {.saddr = "127.0.0.1", .daddr = "127.0.0.2"} /
                bft::TCP {.dport = 80},
            test->verdictAccept());
    }

    int main()
    {
        auto suite = MatcherTestsSuite(BF_MATCHER_MY_TYPE);

        suite << MatcherTest(BF_MATCHER_MY_TYPE, BF_MATCHER_EQ, my_matcher_eq);

        return suite.run();
    }


Integration tests
-----------------

Integration tests are designed to ensure that ``libbpfilter`` and the core ``bpfilter`` module can be integrated into other projects without issues. They validate that the public API of ``libbpfilter`` is stable and behaves as expected when used in different contexts.


Check tests
-----------

Check tests leverage the ``clang-tidy`` and ``clang-format`` tools to ensure code quality and style consistency across the codebase. They help identify potential issues early in the development process. Any warning or error reported by these tools will cause the check tests to fail.


Harness
-------

The test harness is a set of convenience functions used to ease testing of bpfilter.

Test
~~~~
.. doxygenfile:: test.h

Mocks
~~~~~
.. doxygenfile:: mock.h

Fake
~~~~
.. doxygenfile:: fake.h