-
 

ffmpeg_image_transport package from ffmpeg_image_transport repo

ffmpeg_image_transport

Package Summary

Tags No category tags.
Version 1.1.2
License Apache-2
Build type AMENT_CMAKE
Use RECOMMENDED

Repository Summary

Checkout URI https://github.com/ros-misc-utilities/ffmpeg_image_transport.git
VCS Type git
VCS Version humble
Last Updated 2024-04-17
Dev Status DEVELOPED
CI status No Continuous Integration
Released RELEASED
Tags No category tags.
Contributing Help Wanted (0)
Good First Issues (0)
Pull Requests to Review (0)

Package Description

ffmpeg_image_transport provides a plugin to image_transport for transparently sending an image stream encoded with ffmpeg.

Additional Links

Maintainers

  • Bernd Pfrommer

Authors

  • Bernd Pfrommer

ROS2 image transport for FFmpeg encoding

The ROS2 image transport supports encoding/decoding with the FFMpeg library, for example encoding h264 and h265 or HEVC, using Nvidia or other hardware acceleration when available. This package is a complete rewrite of an older ROS1 ffmpeg_image_transport package.

The publisher plugin of the transport produces ffmpeg image transport messages. These are raw, encoded packets that are then transmitted and decoded by the subscriber plugin of the transport. The transport library contains both the publisher(encoder) and subscriber(decoder) plugin and therefore must be installed on both sides to be useful.

To extract e.g. frames or an mp4 file from a recorded bag, have a look at the ffmpeg_image_transport_tools repository.

Supported systems

Continuous integration is tested under Ubuntu with the following ROS2 distros:

Build Status Build Status Build Status

Installation

From packages

sudo apt-get install ros-${ROS_DISTRO}-ffmpeg-image-transport

From source

Set the following shell variables:

repo=ffmpeg_image_transport
url=https://github.com/ros-misc-utilities/${repo}.git

and follow the instructions here

Make sure to source your workspace’s install/setup.bash afterwards. If all goes well you should see the transport show up:

ros2 run image_transport list_transports

should give output (among other transport plugins):

"image_transport/ffmpeg"
 - Provided by package: ffmpeg_image_transport
 - Publisher: 
      This plugin encodes frames into ffmpeg compressed packets
    
 - Subscriber: 
      This plugin decodes frames from ffmpeg compressed packets

Remember to install the plugin on both hosts, the one that is encoding and the one that is decoding (viewing).

Parameters

The plugin has a few parameters that allow for some amount of control.

Publisher (camera driver)

  • encoding: Only ever tested: libx264, h264_nvenc, h264, hevc_nvenc, h264_vaapi. If you have an Nvidia card it most likely supports hevc_nvenc. This will dramatically reduce the CPU load compare to libx264 (the default). You can list all available codecs with ffmpeg -codecs. In the relevant row, look for what it says under (encoders).
  • preset: For instance slow, ll (low latency) etc. To find out what presets are available, run e.g. fmpeg -hide_banner -f lavfi -i nullsrc -c:v libx264 -preset help -f mp4 - 2>&1
  • profile: For instance baseline, main. See the ffmpeg website.
  • tune: See the ffmpeg website.
  • gop_size: The number of frames inbetween keyframes. Default is 15. The larger this number the more latency you will have, but also the more efficient the transmission becomes.
  • bit_rate: The max bit rate [in bits/s] that the encoding will target. Default is ``8242880`.

The parameters are under the ffmpeg_image_transport variable block. So if you launch your publisher node (camera driver), you can give it a parameter list on the way like so:

        parameters=[{'ffmpeg_image_transport.encoding': 'hevc_nvenc',
                     'ffmpeg_image_transport.profile': 'main',
                     'ffmpeg_image_transport.preset': 'll',
                     'ffmpeg_image_transport.gop': 15}]

Subscriber (viewer)

The subscriber has only one parameter, which is the map between the encoding that was used to encode the frames, and the decoder to be used for decoding. The mapping is done via parameters. To tell the subscriber to use the hevc decoder instead of the default hevc_cuvid decoder for decoding incoming hevc_nvenc packets set a parameter like so after you started the viewer:

ros2 param set <name_of_your_viewer_node> ffmpeg_image_transport.map.hevc_nvenc hevc

You also need to refresh the subscription (drop down menu in the viewer) for the parameter to take hold. If anyone ever figures out how to set the parameters when starting the viewer, please report back.

Republishing

The image_transport allows you to republish the decoded image locally, see for instance here. Here the ROS parameters work as expected to modify the mapping between encoding and decoder.

The following line shows how to specify the decoder when republishing. For example to decode incoming hevc_nvenc packets with the hevc decoder:

ros2 run image_transport republish ffmpeg in/ffmpeg:=image_raw/ffmpeg raw out:=image_raw/uncompressed --ros-args -p "ffmpeg_image_transport.map.hevc_nvenc:=hevc"

Republishing is generally not necessary so long as publisher and subscriber both properly use an image transport. Some nodes however, notably the rosbag player, do not support a proper transport, rendering republishing necessary.

Republishing raw images from rosbags in ffmpeg format

Suppose you have raw images in a rosbag but want to play them across a network using the ffmpeg_image_transport. In this case run a republish node like this (assuming your rosbag topic is /my_camera/image_raw):

ros2 run image_transport republish raw in:=/my_camera/image_raw

The republished topic will be under a full transport, meaning you can now view them with e.g. rqt_image_view under the topic /out/ffmpeg.

You can record them in ffmpeg format by e.g ros2 bag record /out/ffmpeg.

Republishing compressed images from rosbags

Let’s say you have stored images as ffmpeg packets in a rosbag under the topic /my_camera/ffmpeg. To view them use this line:

ros2 run image_transport republish ffmpeg in/ffmpeg:=/my_camera/ffmpeg raw


This will republish the topic with full image transport support.

Setting encoding parameters when launching camera driver

The launch directory contains an example launch file cam.launch.py that demonstrates how to set encoding profile and preset for e.g. a usb camera.

How to use a custom version of libav (aka ffmpeg)

Compile and install ffmpeg. Let’s say the install directory is /home/foo/ffmpeg/build, then for it to be found while building, run colcon like this:

colcon build --symlink-install --cmake-args --no-warn-unused-cli -DFFMPEG_PKGCONFIG=/home/foo/ffmpeg/build/lib/pkgconfig -DCMAKE_BUILD_TYPE=RelWithDebInfo 

This will compile against the right headers, but at runtime it may still load the system ffmpeg libraries. To avoid that, set LD_LIBRARY_PATH at runtime:

export LD_LIBRARY_PATH=/home/foo/ffmpeg/build/lib:${LD_LIBRARY_PATH}

License

This software is issued under the Apache License Version 2.0.

CHANGELOG

Changelog for package ffmpeg_image_transport

1.1.2 (2024-04-17)

  • use appropriate header for cv_bridge
  • Contributors: Bernd Pfrommer

1.1.1 (2024-03-11)

  • added documentation
  • move encoder->decoder map to decoder for public use
  • added frame delay control
  • changed install directory so other pkgs can ament_target_depend on this library
  • Contributors: Bernd Pfrommer, Toby Buckley

1.1.0 (2024-01-11)

  • initial release of ROS2 package
  • Contributors: Akshay Srinivasan, Bernd Pfrommer, Francesco Ferroni, Michal Sojka, akssri

Wiki Tutorials

This package does not provide any links to tutorials in it's rosindex metadata. You can check on the ROS Wiki Tutorials page for the package.

Launch files

No launch files found

Messages

No message files found.

Services

No service files found

Plugins

Recent questions tagged ffmpeg_image_transport at Robotics Stack Exchange

ffmpeg_image_transport package from ffmpeg_image_transport repo

ffmpeg_image_transport

Package Summary

Tags No category tags.
Version 1.2.1
License Apache-2
Build type AMENT_CMAKE
Use RECOMMENDED

Repository Summary

Checkout URI https://github.com/ros-misc-utilities/ffmpeg_image_transport.git
VCS Type git
VCS Version iron
Last Updated 2024-03-16
Dev Status DEVELOPED
CI status No Continuous Integration
Released RELEASED
Tags No category tags.
Contributing Help Wanted (0)
Good First Issues (0)
Pull Requests to Review (0)

Package Description

ffmpeg_image_transport provides a plugin to image_transport for transparently sending an image stream encoded with ffmpeg.

Additional Links

Maintainers

  • Bernd Pfrommer

Authors

  • Bernd Pfrommer

ROS2 image transport for FFmpeg encoding

The ROS2 image transport supports encoding/decoding with the FFMpeg library, for example encoding h264 and h265 or HEVC, using Nvidia or other hardware acceleration when available. This package is a complete rewrite of an older ROS1 ffmpeg_image_transport package.

The publisher plugin of the transport produces ffmpeg image transport messages. These are raw, encoded packets that are then transmitted and decoded by the subscriber plugin of the transport. The transport library contains both the publisher(encoder) and subscriber(decoder) plugin and therefore must be installed on both sides to be useful.

To extract e.g. frames or an mp4 file from a recorded bag, have a look at the ffmpeg_image_transport_tools repository.

Supported systems

Continuous integration is tested under Ubuntu with the following ROS2 distros:

Build Status Build Status Build Status

Installation

From packages

sudo apt-get install ros-${ROS_DISTRO}-ffmpeg-image-transport

From source

Set the following shell variables:

repo=ffmpeg_image_transport
url=https://github.com/ros-misc-utilities/${repo}.git

and follow the instructions here

Make sure to source your workspace’s install/setup.bash afterwards. If all goes well you should see the transport show up:

ros2 run image_transport list_transports

should give output (among other transport plugins):

"image_transport/ffmpeg"
 - Provided by package: ffmpeg_image_transport
 - Publisher: 
      This plugin encodes frames into ffmpeg compressed packets
    
 - Subscriber: 
      This plugin decodes frames from ffmpeg compressed packets

Remember to install the plugin on both hosts, the one that is encoding and the one that is decoding (viewing).

Parameters

The plugin has a few parameters that allow for some amount of control.

Publisher (camera driver)

  • encoding: Only ever tested: libx264, h264_nvenc, h264, hevc_nvenc, h264_vaapi. If you have an Nvidia card it most likely supports hevc_nvenc. This will dramatically reduce the CPU load compare to libx264 (the default). You can list all available codecs with ffmpeg -codecs. In the relevant row, look for what it says under (encoders).
  • preset: For instance slow, ll (low latency) etc. To find out what presets are available, run e.g. fmpeg -hide_banner -f lavfi -i nullsrc -c:v libx264 -preset help -f mp4 - 2>&1
  • profile: For instance baseline, main. See the ffmpeg website.
  • tune: See the ffmpeg website.
  • gop_size: The number of frames inbetween keyframes. Default is 15. The larger this number the more latency you will have, but also the more efficient the transmission becomes.
  • bit_rate: The max bit rate [in bits/s] that the encoding will target. Default is ``8242880`.

The parameters are under the ffmpeg_image_transport variable block. So if you launch your publisher node (camera driver), you can give it a parameter list on the way like so:

        parameters=[{'ffmpeg_image_transport.encoding': 'hevc_nvenc',
                     'ffmpeg_image_transport.profile': 'main',
                     'ffmpeg_image_transport.preset': 'll',
                     'ffmpeg_image_transport.gop': 15}]

Subscriber (viewer)

The subscriber has only one parameter, which is the map between the encoding that was used to encode the frames, and the decoder to be used for decoding. The mapping is done via parameters. To tell the subscriber to use the hevc decoder instead of the default hevc_cuvid decoder for decoding incoming hevc_nvenc packets set a parameter like so after you started the viewer:

ros2 param set <name_of_your_viewer_node> ffmpeg_image_transport.map.hevc_nvenc hevc

You also need to refresh the subscription (drop down menu in the viewer) for the parameter to take hold. If anyone ever figures out how to set the parameters when starting the viewer, please report back.

Republishing

The image_transport allows you to republish the decoded image locally, see for instance here. Here the ROS parameters work as expected to modify the mapping between encoding and decoder.

The following line shows how to specify the decoder when republishing. For example to decode incoming hevc_nvenc packets with the hevc decoder:

ros2 run image_transport republish ffmpeg in/ffmpeg:=image_raw/ffmpeg raw out:=image_raw/uncompressed --ros-args -p "ffmpeg_image_transport.map.hevc_nvenc:=hevc"

Republishing is generally not necessary so long as publisher and subscriber both properly use an image transport. Some nodes however, notably the rosbag player, do not support a proper transport, rendering republishing necessary.

Republishing raw images from rosbags in ffmpeg format

Suppose you have raw images in a rosbag but want to play them across a network using the ffmpeg_image_transport. In this case run a republish node like this (assuming your rosbag topic is /my_camera/image_raw):

ros2 run image_transport republish raw in:=/my_camera/image_raw

The republished topic will be under a full transport, meaning you can now view them with e.g. rqt_image_view under the topic /out/ffmpeg.

You can record them in ffmpeg format by e.g ros2 bag record /out/ffmpeg.

Republishing compressed images from rosbags

Let’s say you have stored images as ffmpeg packets in a rosbag under the topic /my_camera/ffmpeg. To view them use this line:

ros2 run image_transport republish ffmpeg in/ffmpeg:=/my_camera/ffmpeg raw


This will republish the topic with full image transport support.

Setting encoding parameters when launching camera driver

The launch directory contains an example launch file cam.launch.py that demonstrates how to set encoding profile and preset for e.g. a usb camera.

How to use a custom version of libav (aka ffmpeg)

Compile and install ffmpeg. Let’s say the install directory is /home/foo/ffmpeg/build, then for it to be found while building, run colcon like this:

colcon build --symlink-install --cmake-args --no-warn-unused-cli -DFFMPEG_PKGCONFIG=/home/foo/ffmpeg/build/lib/pkgconfig -DCMAKE_BUILD_TYPE=RelWithDebInfo 

This will compile against the right headers, but at runtime it may still load the system ffmpeg libraries. To avoid that, set LD_LIBRARY_PATH at runtime:

export LD_LIBRARY_PATH=/home/foo/ffmpeg/build/lib:${LD_LIBRARY_PATH}

License

This software is issued under the Apache License Version 2.0.

CHANGELOG

Changelog for package ffmpeg_image_transport

Forthcoming

  • use appropriate header for cv_bridge
  • cleaned up includes
  • change build to test dependency
  • Contributors: Bernd Pfrommer

1.2.1 (2024-03-11)

  • added documentation and improved cmake exporting
  • move encoder->decoder map to decoder for public use
  • added frame delay control
  • changed install directory so other pkgs can ament_target_depend on this library
  • Contributors: Bernd Pfrommer, Toby Buckley

1.2.0 (2024-01-11)

  • initial release of ROS2 package
  • Contributors: Akshay Srinivasan, Bernd Pfrommer, Francesco Ferroni, Michal Sojka, akssri

Wiki Tutorials

This package does not provide any links to tutorials in it's rosindex metadata. You can check on the ROS Wiki Tutorials page for the package.

Launch files

No launch files found

Messages

No message files found.

Services

No service files found

Plugins

Recent questions tagged ffmpeg_image_transport at Robotics Stack Exchange

ffmpeg_image_transport package from ffmpeg_image_transport repo

ffmpeg_image_transport

Package Summary

Tags No category tags.
Version 1.0.1
License Apache-2
Build type AMENT_CMAKE
Use RECOMMENDED

Repository Summary

Checkout URI https://github.com/ros-misc-utilities/ffmpeg_image_transport.git
VCS Type git
VCS Version rolling
Last Updated 2024-04-17
Dev Status DEVELOPED
CI status No Continuous Integration
Released RELEASED
Tags No category tags.
Contributing Help Wanted (0)
Good First Issues (0)
Pull Requests to Review (0)

Package Description

ffmpeg_image_transport provides a plugin to image_transport for transparently sending an image stream encoded with ffmpeg.

Additional Links

Maintainers

  • Bernd Pfrommer

Authors

  • Bernd Pfrommer

ROS2 image transport for FFmpeg encoding

The ROS2 image transport supports encoding/decoding with the FFMpeg library, for example encoding h264 and h265 or HEVC, using Nvidia or other hardware acceleration when available. This package is a complete rewrite of an older ROS1 ffmpeg_image_transport package.

The publisher plugin of the transport produces ffmpeg image transport messages. These are raw, encoded packets that are then transmitted and decoded by the subscriber plugin of the transport. The transport library contains both the publisher(encoder) and subscriber(decoder) plugin and therefore must be installed on both sides to be useful.

To extract e.g. frames or an mp4 file from a recorded bag, have a look at the ffmpeg_image_transport_tools repository.

Supported systems

Continuous integration is tested under Ubuntu with the following ROS2 distros:

Build Status Build Status Build Status

Installation

From packages

sudo apt-get install ros-${ROS_DISTRO}-ffmpeg-image-transport

From source

Set the following shell variables:

repo=ffmpeg_image_transport
url=https://github.com/ros-misc-utilities/${repo}.git

and follow the instructions here

Make sure to source your workspace’s install/setup.bash afterwards. If all goes well you should see the transport show up:

ros2 run image_transport list_transports

should give output (among other transport plugins):

"image_transport/ffmpeg"
 - Provided by package: ffmpeg_image_transport
 - Publisher: 
      This plugin encodes frames into ffmpeg compressed packets
    
 - Subscriber: 
      This plugin decodes frames from ffmpeg compressed packets

Remember to install the plugin on both hosts, the one that is encoding and the one that is decoding (viewing).

Parameters

The plugin has a few parameters that allow for some amount of control.

Publisher (camera driver)

  • encoding: Only ever tested: libx264, h264_nvenc, h264, hevc_nvenc, h264_vaapi. If you have an Nvidia card it most likely supports hevc_nvenc. This will dramatically reduce the CPU load compare to libx264 (the default). You can list all available codecs with ffmpeg -codecs. In the relevant row, look for what it says under (encoders).
  • preset: For instance slow, ll (low latency) etc. To find out what presets are available, run e.g. fmpeg -hide_banner -f lavfi -i nullsrc -c:v libx264 -preset help -f mp4 - 2>&1
  • profile: For instance baseline, main. See the ffmpeg website.
  • tune: See the ffmpeg website.
  • gop_size: The number of frames inbetween keyframes. Default is 15. The larger this number the more latency you will have, but also the more efficient the transmission becomes.
  • bit_rate: The max bit rate [in bits/s] that the encoding will target. Default is ``8242880`.

The parameters are under the ffmpeg_image_transport variable block. So if you launch your publisher node (camera driver), you can give it a parameter list on the way like so:

        parameters=[{'ffmpeg_image_transport.encoding': 'hevc_nvenc',
                     'ffmpeg_image_transport.profile': 'main',
                     'ffmpeg_image_transport.preset': 'll',
                     'ffmpeg_image_transport.gop': 15}]

Subscriber (viewer)

The subscriber has only one parameter, which is the map between the encoding that was used to encode the frames, and the decoder to be used for decoding. The mapping is done via parameters. To tell the subscriber to use the hevc decoder instead of the default hevc_cuvid decoder for decoding incoming hevc_nvenc packets set a parameter like so after you started the viewer:

ros2 param set <name_of_your_viewer_node> ffmpeg_image_transport.map.hevc_nvenc hevc

You also need to refresh the subscription (drop down menu in the viewer) for the parameter to take hold. If anyone ever figures out how to set the parameters when starting the viewer, please report back.

Republishing

The image_transport allows you to republish the decoded image locally, see for instance here. Here the ROS parameters work as expected to modify the mapping between encoding and decoder.

The following line shows how to specify the decoder when republishing. For example to decode incoming hevc_nvenc packets with the hevc decoder:

ros2 run image_transport republish ffmpeg in/ffmpeg:=image_raw/ffmpeg raw out:=image_raw/uncompressed --ros-args -p "ffmpeg_image_transport.map.hevc_nvenc:=hevc"

Republishing is generally not necessary so long as publisher and subscriber both properly use an image transport. Some nodes however, notably the rosbag player, do not support a proper transport, rendering republishing necessary.

Republishing raw images from rosbags in ffmpeg format

Suppose you have raw images in a rosbag but want to play them across a network using the ffmpeg_image_transport. In this case run a republish node like this (assuming your rosbag topic is /my_camera/image_raw):

ros2 run image_transport republish raw in:=/my_camera/image_raw

The republished topic will be under a full transport, meaning you can now view them with e.g. rqt_image_view under the topic /out/ffmpeg.

You can record them in ffmpeg format by e.g ros2 bag record /out/ffmpeg.

Republishing compressed images from rosbags

Let’s say you have stored images as ffmpeg packets in a rosbag under the topic /my_camera/ffmpeg. To view them use this line:

ros2 run image_transport republish ffmpeg in/ffmpeg:=/my_camera/ffmpeg raw


This will republish the topic with full image transport support.

Setting encoding parameters when launching camera driver

The launch directory contains an example launch file cam.launch.py that demonstrates how to set encoding profile and preset for e.g. a usb camera.

How to use a custom version of libav (aka ffmpeg)

Compile and install ffmpeg. Let’s say the install directory is /home/foo/ffmpeg/build, then for it to be found while building, run colcon like this:

colcon build --symlink-install --cmake-args --no-warn-unused-cli -DFFMPEG_PKGCONFIG=/home/foo/ffmpeg/build/lib/pkgconfig -DCMAKE_BUILD_TYPE=RelWithDebInfo 

This will compile against the right headers, but at runtime it may still load the system ffmpeg libraries. To avoid that, set LD_LIBRARY_PATH at runtime:

export LD_LIBRARY_PATH=/home/foo/ffmpeg/build/lib:${LD_LIBRARY_PATH}

License

This software is issued under the Apache License Version 2.0.

CHANGELOG

Changelog for package ffmpeg_image_transport

1.0.1 (2024-04-17)

  • use appropriate header for cv_bridge
  • added documentation and improved cmake exporting
  • move encoder->decoder map to decoder for public use
  • added frame delay control
  • changed install directory so other pkgs can ament_target_depend on this library
  • Contributors: Bernd Pfrommer, Toby Buckley

1.0.0 (2024-01-11)

  • initial release of ROS2 package
  • Contributors: Akshay Srinivasan, Bernd Pfrommer, Francesco Ferroni, Michal Sojka, akssri

Wiki Tutorials

This package does not provide any links to tutorials in it's rosindex metadata. You can check on the ROS Wiki Tutorials page for the package.

Launch files

No launch files found

Messages

No message files found.

Services

No service files found

Plugins

Recent questions tagged ffmpeg_image_transport at Robotics Stack Exchange

ffmpeg_image_transport package from ffmpeg_image_transport repo

ffmpeg_image_transport

Package Summary

Tags No category tags.
Version 1.0.1
License Apache-2
Build type AMENT_CMAKE
Use RECOMMENDED

Repository Summary

Checkout URI https://github.com/ros-misc-utilities/ffmpeg_image_transport.git
VCS Type git
VCS Version rolling
Last Updated 2024-04-17
Dev Status DEVELOPED
CI status No Continuous Integration
Released RELEASED
Tags No category tags.
Contributing Help Wanted (0)
Good First Issues (0)
Pull Requests to Review (0)

Package Description

ffmpeg_image_transport provides a plugin to image_transport for transparently sending an image stream encoded with ffmpeg.

Additional Links

Maintainers

  • Bernd Pfrommer

Authors

  • Bernd Pfrommer

ROS2 image transport for FFmpeg encoding

The ROS2 image transport supports encoding/decoding with the FFMpeg library, for example encoding h264 and h265 or HEVC, using Nvidia or other hardware acceleration when available. This package is a complete rewrite of an older ROS1 ffmpeg_image_transport package.

The publisher plugin of the transport produces ffmpeg image transport messages. These are raw, encoded packets that are then transmitted and decoded by the subscriber plugin of the transport. The transport library contains both the publisher(encoder) and subscriber(decoder) plugin and therefore must be installed on both sides to be useful.

To extract e.g. frames or an mp4 file from a recorded bag, have a look at the ffmpeg_image_transport_tools repository.

Supported systems

Continuous integration is tested under Ubuntu with the following ROS2 distros:

Build Status Build Status Build Status

Installation

From packages

sudo apt-get install ros-${ROS_DISTRO}-ffmpeg-image-transport

From source

Set the following shell variables:

repo=ffmpeg_image_transport
url=https://github.com/ros-misc-utilities/${repo}.git

and follow the instructions here

Make sure to source your workspace’s install/setup.bash afterwards. If all goes well you should see the transport show up:

ros2 run image_transport list_transports

should give output (among other transport plugins):

"image_transport/ffmpeg"
 - Provided by package: ffmpeg_image_transport
 - Publisher: 
      This plugin encodes frames into ffmpeg compressed packets
    
 - Subscriber: 
      This plugin decodes frames from ffmpeg compressed packets

Remember to install the plugin on both hosts, the one that is encoding and the one that is decoding (viewing).

Parameters

The plugin has a few parameters that allow for some amount of control.

Publisher (camera driver)

  • encoding: Only ever tested: libx264, h264_nvenc, h264, hevc_nvenc, h264_vaapi. If you have an Nvidia card it most likely supports hevc_nvenc. This will dramatically reduce the CPU load compare to libx264 (the default). You can list all available codecs with ffmpeg -codecs. In the relevant row, look for what it says under (encoders).
  • preset: For instance slow, ll (low latency) etc. To find out what presets are available, run e.g. fmpeg -hide_banner -f lavfi -i nullsrc -c:v libx264 -preset help -f mp4 - 2>&1
  • profile: For instance baseline, main. See the ffmpeg website.
  • tune: See the ffmpeg website.
  • gop_size: The number of frames inbetween keyframes. Default is 15. The larger this number the more latency you will have, but also the more efficient the transmission becomes.
  • bit_rate: The max bit rate [in bits/s] that the encoding will target. Default is ``8242880`.

The parameters are under the ffmpeg_image_transport variable block. So if you launch your publisher node (camera driver), you can give it a parameter list on the way like so:

        parameters=[{'ffmpeg_image_transport.encoding': 'hevc_nvenc',
                     'ffmpeg_image_transport.profile': 'main',
                     'ffmpeg_image_transport.preset': 'll',
                     'ffmpeg_image_transport.gop': 15}]

Subscriber (viewer)

The subscriber has only one parameter, which is the map between the encoding that was used to encode the frames, and the decoder to be used for decoding. The mapping is done via parameters. To tell the subscriber to use the hevc decoder instead of the default hevc_cuvid decoder for decoding incoming hevc_nvenc packets set a parameter like so after you started the viewer:

ros2 param set <name_of_your_viewer_node> ffmpeg_image_transport.map.hevc_nvenc hevc

You also need to refresh the subscription (drop down menu in the viewer) for the parameter to take hold. If anyone ever figures out how to set the parameters when starting the viewer, please report back.

Republishing

The image_transport allows you to republish the decoded image locally, see for instance here. Here the ROS parameters work as expected to modify the mapping between encoding and decoder.

The following line shows how to specify the decoder when republishing. For example to decode incoming hevc_nvenc packets with the hevc decoder:

ros2 run image_transport republish ffmpeg in/ffmpeg:=image_raw/ffmpeg raw out:=image_raw/uncompressed --ros-args -p "ffmpeg_image_transport.map.hevc_nvenc:=hevc"

Republishing is generally not necessary so long as publisher and subscriber both properly use an image transport. Some nodes however, notably the rosbag player, do not support a proper transport, rendering republishing necessary.

Republishing raw images from rosbags in ffmpeg format

Suppose you have raw images in a rosbag but want to play them across a network using the ffmpeg_image_transport. In this case run a republish node like this (assuming your rosbag topic is /my_camera/image_raw):

ros2 run image_transport republish raw in:=/my_camera/image_raw

The republished topic will be under a full transport, meaning you can now view them with e.g. rqt_image_view under the topic /out/ffmpeg.

You can record them in ffmpeg format by e.g ros2 bag record /out/ffmpeg.

Republishing compressed images from rosbags

Let’s say you have stored images as ffmpeg packets in a rosbag under the topic /my_camera/ffmpeg. To view them use this line:

ros2 run image_transport republish ffmpeg in/ffmpeg:=/my_camera/ffmpeg raw


This will republish the topic with full image transport support.

Setting encoding parameters when launching camera driver

The launch directory contains an example launch file cam.launch.py that demonstrates how to set encoding profile and preset for e.g. a usb camera.

How to use a custom version of libav (aka ffmpeg)

Compile and install ffmpeg. Let’s say the install directory is /home/foo/ffmpeg/build, then for it to be found while building, run colcon like this:

colcon build --symlink-install --cmake-args --no-warn-unused-cli -DFFMPEG_PKGCONFIG=/home/foo/ffmpeg/build/lib/pkgconfig -DCMAKE_BUILD_TYPE=RelWithDebInfo 

This will compile against the right headers, but at runtime it may still load the system ffmpeg libraries. To avoid that, set LD_LIBRARY_PATH at runtime:

export LD_LIBRARY_PATH=/home/foo/ffmpeg/build/lib:${LD_LIBRARY_PATH}

License

This software is issued under the Apache License Version 2.0.

CHANGELOG

Changelog for package ffmpeg_image_transport

1.0.1 (2024-04-17)

  • use appropriate header for cv_bridge
  • added documentation and improved cmake exporting
  • move encoder->decoder map to decoder for public use
  • added frame delay control
  • changed install directory so other pkgs can ament_target_depend on this library
  • Contributors: Bernd Pfrommer, Toby Buckley

1.0.0 (2024-01-11)

  • initial release of ROS2 package
  • Contributors: Akshay Srinivasan, Bernd Pfrommer, Francesco Ferroni, Michal Sojka, akssri

Wiki Tutorials

This package does not provide any links to tutorials in it's rosindex metadata. You can check on the ROS Wiki Tutorials page for the package.

Launch files

No launch files found

Messages

No message files found.

Services

No service files found

Plugins

Recent questions tagged ffmpeg_image_transport at Robotics Stack Exchange

No version for distro noetic. Known supported distros are highlighted in the buttons above.
No version for distro galactic. Known supported distros are highlighted in the buttons above.
No version for distro melodic. Known supported distros are highlighted in the buttons above.