iceoryx - true zero-copy inter-process-communication

Introduction

Great that you’ve made it to this neat Eclipse project! Let’s get you started by providing a quick background tour, introducing the project scope and all you need for installation and a first running example.

So first off: What is iceoryx?

iceoryx is an inter-process-communication (IPC) middleware for various operating systems (currently we support Linux, macOS, QNX, FreeBSD and Windows 10). It has its origins in the automotive industry, where large amounts of data have to be transferred between different processes when it comes to driver assistance or automated driving systems. However, the efficient communication mechanisms can also be applied to a wider range of use cases, e.g. in the field of robotics or game development.

iceoryx uses a true zero-copy, shared memory approach that allows to transfer data from publishers to subscribers without a single copy. This ensures data transmissions with constant latency, regardless of the size of the payload. For more information have a look at the 1000 words iceoryx introduction.

You’re right, middleware is a cluttered term and can somehow be all or nothing. To get a better impression what this means for iceoryx, please have a loot at our goals and non-goals.

Don’t get too frightened of the API when strolling through the examples. Think of the untyped C++ and the C API as a “plumbing” one (“plumbing” as defined in Git, which means low-level). We’re not using the “plumbing” APIs ourselves, but instead the typed C++ API. The normal use case is that iceoryx is integrated as high-performance IPC transport layer in a bigger framework with additional API layers. An example for such a “porcelain” API would be ROS 2. Others are listed in the next section.

You can find the full API documentation on 🌐 https://iceoryx.io.

Supported Platforms

In general unix platforms should work with iceoryx but we only test FreeBSD on our CI.

Where is Eclipse iceoryx used?

Build and install

You can find the build and installation guidelines here.

Examples

After you’ve built all the necessary things, you can continue playing around with the examples.

Build and run in a Docker environment

If you want to avoid installing anything on your host machine but you have Docker installed, it is possible to use it to build and run iceoryx applications.

Please see the dedicated README.md for information on how to do this.

Documentation

• Getting Started
• Installation Guide
• iceoryx Hoofs Hacker Guide

Quality levels & platforms

Quality level are 5 to 1+, where 1+ is highest level.

Please see the Quality Declaration for details of the quality measures according to ROS 2 guidelines.

Is something missing or you’ve got ideas for other nifty examples? Jump right away to the next section!

Contribute

Please refer to the CONTRIBUTING.md for a quick read-up about what to consider if you want to contribute.

Planned features

Get to know the upcoming features and the project scope in PLANNED_FEATURES.md.

Innovations enabled by iceoryx

Frequently Asked Questions (FAQ)

FAQ.md

Governance & maintainers

Please have a look at the GOVERNANCE.md.

Contributing to Eclipse iceoryx

Thanks for your interest in this project.

Project description

In domains like automotive, robotics or gaming, a huge amount of data must be transferred between different parts of the system. If these parts are actually different processes on a POSIX based operating system like Linux, this huge amount of data has to be transferred via an inter-process-communication (IPC) mechanism. Find more infos on the Eclipse site.

Eclipse Contributor Agreement

Before your contribution can be accepted by the project team, contributors must electronically sign the Eclipse Contributor Agreement (ECA).

Commits that are provided by non-committers must have a Signed-off-by field in the footer indicating that the author is aware of the terms by which the contribution has been provided to the project. The non-committer must additionally have an Eclipse Foundation account and must have a signed Eclipse Contributor Agreement (ECA) on file.

For more information, please see the Eclipse Committer Handbook.

Contact

Contact the project developers via the project’s “dev” list.

• iceoryx-dev@eclipse.org

Feature request and bugs

We love pull requests! The next sections try to cover most of the relevant questions. For larger contributions or architectural changes, we’d kindly ask you to either:

• Raise the proposed changes during a developer meetup

or

• Create a design document and raise it in a separate pull request beforehand

If you would like to report a bug or propose a new feature, please raise an issue before raising a pull request. Please have a quick search upfront if a similar issue already exists. A release board is used to prioritize the issues for a specific release. This makes it easier to track the work-in-progress. If you have troubles getting an issue assigned to you please contact the maintainers via Gitter.

Please make sure you have:

1. Signed the Eclipse Contributor Agreement
2. Created an issue before creating a branch, e.g. Super duper feature with issue number 123
3. All branches have the following naming format: iox-#[issue]-branch-name e.g. iox-#123-super-duper-feature
4. All commits have the following naming format: iox-#[issue] commit message e.g. iox-#123 implemented super-duper feature
5. All commits have been signed with git commit -s
6. The iceoryx-unreleased.md in doc/website/release-notes is updated with the GitHub issue that is closed by the Pull-Request
7. You open your pull request towards the base branch master
8. Link the pull request to the according GitHub issue and set the label accordingly

NOTE: For support while developing you can use little helper scripts, see git-hooks.

Branching strategy

master

• Main development branch
• Open for external contributions

release_x.y

• Branch for stabilising a certain release
• Write access limited to maintainers
• Fine-tuning of external contribution e.g. running Axivion SCA
• Finish any missing implementations regarding the quality levels

As depicted below, after the release branch has been created the stabilization phase will begin. After finishing the release, a git tag will be created to point to HEAD of the release branch. Follow-up releases will be branched off from the git tag.

o---o---o---o---o  master
     \
      \      v1.0.0      v1.0.1
       \        |           |
        o---o---o---o---o---o---o  release_1.0
                         \
                          \      v1.1.0
                           \        |
                            o---o---o  release_1.1

Coding style

We love the C++ core guidelines. If in doubt please try to follow them as well as our unwritten conventions in the existing parts of the code base. Please format your code with the provided clang-format and clang-tidy before raising a pull request. Nowadays, many IDEs read the clang-format file.

We created some handy rules to highlight some specifics that you might not be used to in other FOSS projects. They are helpful to build embedded systems for safety fields like automotive or avionics. It is possible that not the whole codebase follows these rules, things are work in progress.

1) No heap is allowed, static memory management hugely decreases the complexity of your software (e.g. cxx::vector without heap) 2) No exceptions are allowed, all function and methods need to have noexcept in their signature 3) No undefined behavior, zero-cost abstract is not feasible in high safety environments 4) Use C++14 5) Rule of Five, if there is a non-default destructor needed, the rule of five has to be applied 6) STL, we aim to be compatible towards the STL, but our code may contain additions which are not compatible with the STL (e.g. iox::cxx::vector::emplace_back() does return a bool) 7) Always use iox::log::Logger, instead of printf() 8) Always use iox::ErrorHandler or iox::cxx::Expects/iox::cxx::Ensures, when an error occurs that cannot or shall not be propagated via an iox::cxx::expected 9) Not more than two-level nested namespaces, three-level nested namespace can be used sparsely

See error-handling.md for additional information about logging and error handling.

Naming conventions

• File names with lower_snake_case: my_thing.hpp
• Structs, classes and enum classes in UpperCamelCase: class MyClass{}
• Methods and variables in lowerCamelCase: uint16_t myVariable
• Compile time constants, also enum values in UPPER_SNAKE_CASE: static constexpr uint16_t MY_CONSTANT
• Class members start with m_: m_myMember
  • Public members of structs and classes do not have the m_ prefix
• Namespaces in lower_snake_case : my_namespace
• Aliases have a _t postfix : using FooString_t = iox::cxx::string<100>;
• Objects created from a method returning a cxx::optional<Foo> shall be named maybeFoo
• Objects created from a method returning a cxx::expected<Foo, FooError> shall contain the name result e.g. getChunkResult for the method getChunk()

Doxygen

Please use doxygen to document your code.

The following doxygen comments are required for public API headers:

/// @brief short description
/// @param[in] / [out] / [in,out] name description
/// @return description

For overrides of virtual methods the copydoc tag can be used:

/// @copydoc BaseClass::method
/// @note Optional describe some specifics to the override

A good example for code formatting and doxygen structure can be found in swe_docu_guidelines.md (WIP)

Folder structure

The folder structure boils down to:

• iceoryx_COMPONENT
  • cmake: all CMake files go here, needed for find_pkg()
  • doc: manuals and documentation
  • include: public headers with stable API
    • internal: public headers with internal API, which might change quite frequently
  • source: implementation files
  • test: unit and integration tests
  • CMakeLists.txt: build the component separately
• examples_iceoryx: Examples described in iceoryx_examples

All new code should follow the folder structure.

How to add a new example

1. Add the example in the “List of all examples”
2. Create a new file in doc/website/examples/foobar.md and add it to doc/website/examples/.pages. This file shall only set the title and include the readme from /iceoryx_examples/foobar/README.md
3. Add an add_subdirectory directive into iceoryx_meta/CMakeLists.txt in the if(EXAMPLES) section.
4. Consider using geoffrey for syncing code in code blocks with the respective source files
5. Add integration test for example
6. Record an asciicast and embed image link

Testing

We use Google test for our unit and integration tests. We require compatibility with the version 1.10.0.

Have a look at our best practice guidelines for writing tests and installation guide for contributors for building them.

Unit tests (aka module tests)

Unit tests are black box tests that test the public interface of a class. They are required for all new code.

Each unit test case needs a unique identifier (UUID according to RFC 4122) in the form of:

::testing::Test::RecordProperty("TEST_ID", "12345678-9ab-cdef-fedc-1234567890ab");

UUID can be for example generated with Python or the command line tool:

import uuid
uuid.uuid4()

uuidgen -r

Integration tests

Integration tests test the interaction of several classes. They are optional for new code.

Coverage Scan

To ensure that the provided test code covers the productive code you can do a coverage scan with gcov. The reporting is done with lcov and htmlgen. You will need to install the following packages:

sudo apt install lcov

In iceoryx we have multiple test levels for test coverage: unit, integration, component and all for all test levels together. You can create reports for these different test levels or for all tests. Coverage is done with gcc. The coverage scan applies to Quality level 3 and partly level 2 with branch coverage.

To generate a coverage report, iceoryx needs to be compiled with coverage flags and the tests need to be executed. You can do this with one command in the iceoryx folder like this:

./tools/iceoryx_build_test.sh clean build-all -c <testlevel>

Optionally, you can use the build-all option to get the coverage for extensions like DDS or the C-Binding. The -c flag indicates that you want to generate a coverage report and requires you to pass the test level. By default the test level is set to all.

./tools/iceoryx_build_test.sh clean debug build-all -c unit

NOTE iceoryx needs to be built as static library to work with gcov flags. The script does this automatically.

The flag -c unit is for generating only reports for unit tests. In the script tools/scripts/lcov_generate.sh, the initial scan, filtering and report generation is done automatically.

All reports are stored locally in build/lcov as html report (index.html). In GitHub, we are using Codecov for general reporting of the code coverage. Codecov gives a brief overview of the code coverage and also indicates in Pull-Requests if newly added code is not covered by tests. If you want to see detailed html reports for specific Pull-Requests or branches, you can check here.

Legal & Compliance

Safety & security

The iceoryx maintainers aim for ASIL-D compliance. The ISO26262 is a good read if you want to learn more about automotive safety. A nice introduction video was presented on CppCon 2019.

If you want to report a vulnerability, please use the Eclipse process.

Static code analysis

The iceoryx maintainers have a partnership with Axivion and use their Axivion Suite to run a static code analysis.

Github labels are used to group issues into the rulesets:

If one of the rules is not followed, a rationale is added in the following manner:

Either with a comment in the same line:

*mynullptr = foo; // PRQA S 4242 # Short description why

Or with a comment one line above (the number after the warning number indicates that next ’n’ lines are inclusive)

// PRQA S 4242 1 # Short description why
*mynullptr = foo;

Scan results of the master branch are available on a Axivion dashboard. Please contact one of the maintainers, if you’re interested in getting access.

Don’t be afraid if you don’t have Axivion available. As we want to make it easy for developers to contribute, please raise a pull request and one of the maintainers will provided you access to the dashboard.

As an alternative it is also possible to use Perforce’s Helix QAC++ to perform a static-code analysis.

Header

Each source file needs to have this header:

// Copyright (c) [YEAR OF INITIAL CONTRIBUTION] - [YEAR LAST CONTRIBUTION] by [CONTRIBUTOR]. All rights reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
// SPDX-License-Identifier: Apache-2.0

NOTE: The date is either a year or a range of years with the first and last years of the range separated by a dash. For example: “2004” (initial and last contribution in the same year) or “2000 - 2004”. The first year is when the contents of the file were first created and the last year is when the contents were last modified. The years of contribution should be ordered in chronological order, thus the last date in the list should be the year of the most recent contribution. If there is a gap between contributions of one or more calendar years, use a comma to separate the disconnected contribution periods (e.g. “2000 - 2004, 2006”).

Example:

// Copyright (c) 2019 - 2020, 2022 by Acme Corporation. All rights reserved.
// Copyright (c) 2020 - 2022 by Jane Doe <jane@example.com>. All rights reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
// SPDX-License-Identifier: Apache-2.0

NOTE: For scripts or CMake files you can use the respective comment syntax # for the header.

Quality levels

The CMake targets are developed according to the ROS quality levels. Despite developing some targets according to automotive standards like ISO26262, the code base standalone does NOT legitimize the usage in a safety-critical system. All requirements of a lower quality level are included in higher quality levels e.g. quality level 4 is included in quality level 3.

Quality level 5

This quality level is the default quality level. It is meant for examples and helper tools.

• Derived from ROS quality level 5
  • Reviewed by two approvers
  • No compiler warnings
  • License and copyright statements available
  • No version policy required
  • No unit tests required

Quality level 4

• Derived from ROS quality level 4
  • Basic unit tests are required
  • Builds and runs on Windows, MacOS, Linux and QNX

Quality level 3

• Derived from ROS quality level 3
  • Doxygen and documentation required
  • Test specification required
  • Version policy required

Quality level 2

This quality level is meant for all targets that need tier 1 support in ROS 2.

• Derived from ROS quality level 2
  • Must have a quality declaration document

Quality level 1

• Derived from ROS quality level 1
  • Version policy for stable API and ABI required
  • ASPICE SWE.6 tests available
  • Performance tests and regression policy required
  • Static code analysis warnings in Axivion addressed
  • Enforcing the code style is required
  • Unit tests have full statement and branch coverage

Quality level 1+

This quality level goes beyond the ROS quality levels and contains extensions.

• Code coverage according to MC/DC available

Training material recommended for contributors

• Effective C++ by Scott Meyers
• Unit Testing and the Arrange, Act and Assert (AAA) Pattern by Paulo Gomes
• The C++ Standard Library by Nicolai M. Josuttis
• Modern C++ Programming with Test-Driven Development: Code Better, Sleep Better by Jeff Langr
• Modern C++ Design: Generic Programming and Design Patterns by Andrei Alexandrescu
• Exceptional C++ by Herb Sutter
• C++ Concurrency in Action by Anthony Williams