launch_pytest

This is a framework for launch integration testing. For example:

• The exit codes of all processes are available to the tests.
• Tests can check that all processes shut down normally, or with specific exit codes.
• Tests can fail when a process dies unexpectedly.
• The stdout and stderr of all processes are available to the tests.
• The command-line used to launch the processes are available to the tests.
• Some tests run concurrently with the launch and can interact with the running processes.

Differences with launch_testing

launch_testing is an standalone testing tool, which lacks many features:

• It’s impossible to filter test cases by name and run only some.
• It’s impossible to mark a test as skipped or xfail.
• The error reporting of the tool was custom, and the output wasn’t as nice as the output generated by other testing frameworks such as unittest and pytest.

launch_pytest is a really simple pytest plugin leveraging pytest fixtures to manage a launch service lifetime easily.

Quick start example

Start with the pytest_hello_world.py example.

Run the example by doing:

python3 -m pytest test/launch_pytest/examples/pytest_hello_world.py

The launch_pytest plugin will launch the nodes found in the launch_description fixture, run the tests from the test_read_stdout() class, shut down the launched nodes, and then run the statements after the yield statement in test_read_stdout().

launch_pytest fixtures

@launch_pytest.fixture
def launch_description(hello_world_proc):
    """Launch a simple process to print 'hello_world'."""
    return launch.LaunchDescription([
        hello_world_proc,
        # Tell launch when to start the test
        # If no ReadyToTest action is added, one will be appended automatically.
        launch_pytest.actions.ReadyToTest()
    ])

A @launch_pytest.fixture function should return a launch.LaunchDescription object, or a sequence of objects whose first item is a launch.LaunchDescription. This launch description will be used in all tests with a mark @pytest.mark.launch(fixture=<your_fixture_name>), in this case <your_fixture_name>=launch_description.

The launch description can include a ReadyToTest action to signal to the test framework that it’s safe to start the active tests. If one isn’t included, a ReadyToTest action will be appended at the end.

launch_pytest fixtures can have module, class or function scope. The default is function. For example:

@launch_pytest.fixture(scope=my_scope)
def my_fixture():
    return LaunchDescription(...)

@pytest.mark.launch(fixture=my_fixture)
def test_case_1():
    pass

@pytest.mark.launch(fixture=my_fixture)
def test_case_2():
    pass

If my_scope=function, the following happens:

• A launch service using the LaunchDescription returned by my_fixture() is started.
• test_case_1() is run.
• The launch service is shutdown.
• Another launch service using the LaunchDescription returned by my_fixture() is started, my_fixture() is called again.
• test_case_2() is run.
• The launch service is shutdown.

Whereas when my_scope=module, test case_2() will run immediately after test case_1(), concurrently with the same launch service.

It’s not recommended to mix fixtures with module scope with fixtures of class/function scope in the same file. It’s not recommended to use fixtures with scope larger than module. A test shouldn’t depend on more than one launch_pytest fixture. Neither of the three things above automatically generates an error in the current launch_pytest implementation, but future versions might.

Active Tests and shutdown tests

Test cases marked with @pytest.mark.launch will be run concurrently with the launch service or after launch shutdown, depending on the object being marked and the mark arguments.

• functions: Functions marked with @pytest.mark.launch will run concurrently with the launch service, except when shutdown=True is passed as an argument to the decorator.

@pytest.mark.launch(fixture=my_ld_fixture)
def normal_test_case():
    pass

@pytest.mark.launch(fixture=my_ld_fixture, shutdown=True)
def shutdown_test_case():
    pass

• coroutine functions: The same rules as normal functions apply. Coroutines will be run in the same event loop as the launch description, whereas normal functions run concurrently in another thread.

@pytest.mark.launch(fixture=my_ld_fixture)
async def normal_test_case():
    pass

@pytest.mark.launch(fixture=my_ld_fixture, shutdown=True)
async def shutdown_test_case():
    pass

• generators: The first time the generator is called it runs concurrently with the launch service. The generator will be resumed after the launch service is shutdown. i.e. This allows to write a test that has a step that runs concurrently with the service and one that runs after shutdown easily. The yielded value is ignored. If the generator doesn’t stop iteration after being resumed for a second time, the test will fail. Passing a shutdown argument to the decorator is not allowed in this case.

@pytest.mark.launch(fixture=my_ld_fixture)
def normal_test_case():
    assert True
    yield
    assert True

• async generators: The same rules as for generators apply here as well. The only difference between the two is that async generator will run in the same event loop as the launch service, whereas a generator will run concurrently in another thread.

@pytest.mark.launch(fixture=my_ld_fixture)
async def normal_test_case():
    assert True
    yield
    assert True

Fixtures

The launch_pytest plugin will provide the following fixtures.

• launch_service: The launch service being used to run the tests. It will have the same scope as the launch_pytest fixture with wider scope in the module.
• launch_context: The launch context being used to run the tests. It will have the same scope as the launch_pytest fixture with wider scope in the module.
• event_loop: The event loop being used to run the launch service and to run async tests. It will have the same scope as the launch_pytest fixture with wider scope in the module.

Changelog for package launch_pytest

3.7.0 (2024-11-20)

• Add test_xmllint to all of the ament_python packages. (#804)
• Contributors: Chris Lalancette

3.6.1 (2024-07-29)

• Switch to using an rclpy context manager. (#787)
• Contributors: Chris Lalancette

3.6.0 (2024-06-25)

3.5.1 (2024-06-17)

3.5.0 (2024-04-26)

3.4.2 (2024-04-16)

3.4.1 (2024-03-28)

• Switch tryfirst/trylast to hookimpl.
• Contributors: Chris Lalancette

3.4.0 (2024-02-07)

3.3.0 (2024-01-24)

3.2.1 (2023-12-26)

3.2.0 (2023-10-04)

3.1.0 (2023-09-08)

3.0.1 (2023-09-07)

3.0.0 (2023-08-21)

2.2.1 (2023-07-11)

2.2.0 (2023-06-07)

2.1.0 (2023-04-27)

2.0.1 (2023-04-12)

2.0.0 (2023-04-11)

1.4.1 (2023-02-24)

• Fixed typos (#692)
• Contributors: Alejandro Hernández Cordero

1.4.0 (2023-02-14)

• Drop unused data_files entry for example_processes (#680)
• Spelling correction
• [rolling] Update maintainers - 2022-11-07 (#671)
• Contributors: Audrow Nash, Geoffrey Biggs, Scott K Logan

1.3.0 (2022-11-02)

1.2.0 (2022-09-13)

1.1.0 (2022-04-29)

1.0.1 (2022-04-13)

1.0.0 (2022-04-12)

0.23.1 (2022-04-08)

0.23.0 (2022-03-30)

0.22.0 (2022-03-28)

0.21.1 (2022-03-01)

0.21.0 (2022-01-14)

0.20.0 (2021-11-29)

• Update maintainers to Aditya Pande and Michel Hidalgo (#559)
• [launch_pytest] Modify how wait_for_output()/wait_for_stderr() work, add assert_*() alternatives (#553)
• Updated maintainers (#555)
• • [launch_pytest] Fix issue when colcon --retest-until-fail flag is used (#552)
• First prototype of native pytest plugin for launch based tests (#528)
• Contributors: Aditya Pande, Audrow Nash, Ivan Santiago Paunovic

0.19.0 (2021-07-15)

0.18.0 (2021-06-18)

0.17.0 (2021-04-06)

0.16.0 (2021-03-19)

0.15.0 (2021-01-25)

0.14.0 (2020-12-08)

0.13.0 (2020-11-04)

0.12.0 (2020-08-18)

0.11.1 (2020-08-14)

0.11.0 (2020-08-04)

0.10.2 (2020-05-26)

0.10.1 (2020-05-08)

0.10.0 (2020-04-24)

0.9.5 (2019-11-13)

0.9.4 (2019-11-08)

0.9.3 (2019-10-23 22:36)

0.9.2 (2019-10-23 17:49)

0.9.1 (2019-09-18)

0.8.3 (2019-05-29)

0.8.2 (2019-05-20)

0.8.1 (2019-05-08)

0.8.0 (2019-04-13)

0.7.3 (2018-12-12)

0.7.0 (2018-11-16)

0.6.0 (2018-08-20)

0.5.2 (2018-07-17)

0.5.1 (2018-06-27)

0.5.0 (2018-06-19)

0.4.0 (2017-12-08)