Packaging ROS with GitHub Actions
Creating distributions of ROS packages in binary format can be challenging, and many solutions exist, primarily the ROS build farm. However, it can be difficult to set up yourself because you need to use tools like Bloom and Jenkins which need a lot of maintenance. Using standard Debian tooling, we have created a simple GitHub Action to convert a repos file of ROS packages to an apt repository. This can be useful for designing an environment where you need tighter control of versioning or testing your own packages before they can be synced to the official ROS repos. Our Action supports ROS 1 and 2 on various Debian and Ubuntu versions for both public and private repositories. Getting started is as simple as creating a new repository on GitHub, adding the Action yaml and a repos file, then waiting for GitHub-provided CI to publish packages into a new branch. Find it at github.com/marketplace/actions/ros-buildfarm
Why Debian Packages
Binary distributions like Debian are omnipresent because they make installing software a breeze. They work as a binary cache, so users do not have to wait for the compiler. Similarly, one of the critical components to the success of ROS is that binary packages are available.
Traditional build farms, such as the one maintained by OSRF for ROS, still need a lot of work to set up and operate. If the build farm depends on the state of the system it runs on or allows downloading files during compilation, the result can be non-verifiable builds. Another critical but often poorly implemented feature is logging. Good logs make it possible to understand and reproduce build failures.
In Debian, we solved all that long ago by using sbuild.
Every build creates a minimal throw-away system with only declared build dependencies installed.
It can use Linux user namespaces, shielding package creation from almost all outside influences.
This allows a normal non-root user to build and verify every package.
The implemented Action uses this tooling and should also be easy enough that anyone can reproduce and run it on a local system.
Lately, container-based systems like Docker or Flatpak/Snap have gained more popularity as an alternative to Debian packaging. Still, they have many disadvantages in comparison. Updates are enormous, as you often have to download complete images instead of single packages. Images often contain outdated software with security issues and there is no way to verify the origin of the content. Containers are often run in a privileged mode which grants extended access to the host system resulting in running untrusted software with full rights on the host system. Lastly, they all assume that the operating system managing all the containers is somehow maintained. There is more in depth discussion on this in ROS Docker; 6 reasons why they are not a good fit
Distributions like Debian or Ubuntu ensure that you can update your system seamlessly and provide security updates and maintain the complete system. This includes the kernel, bootloader, and sometimes even the firmware. Distributions also serve as trust anchors for verified sources. One thing they can’t shield is when the users install third-party software, so you should still be careful what you install. Containers use a Linux distribution inside, so containers don’t work without distributions, but distributions work without containers.
When would you use this Debian build farm?
Note that this is not meant to replace the ROS build farm but as an addition. Also note that it does not need an extra bloom release repo. Use cases are:
- Nightly packages.
- Test packages before pushing them to the official build farm.
- Building fork of official packages.
- Building distributions no longer supported by OSRF.
- Building closed source packages.
How can you use it as a build farm on GitHub Actions?
We created a repository to host nightly packages of MoveIt 2 for the tutorials: github.com/moveit/moveit2_packages. The input consists of one yaml file controlling the GitHub Action:
name: build
on:
workflow_dispatch:
schedule:
- cron: '17 1 * * *'
jobs:
build_humble:
runs-on: ubuntu-22.04
steps:
- uses: jspricke/ros-deb-builder-action@main
with:
DEB_DISTRO: jammy
ROS_DISTRO: humble
REPOS_FILE: https://raw.githubusercontent.com/ros-planning/moveit2_tutorials/humble/moveit2_tutorials.repos
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
SQUASH_HISTORY: true
build_rolling:
runs-on: ubuntu-22.04
steps:
- uses: jspricke/ros-deb-builder-action@main
with:
DEB_DISTRO: jammy
ROS_DISTRO: rolling
REPOS_FILE: https://raw.githubusercontent.com/ros-planning/moveit2_tutorials/main/moveit2_tutorials.repos
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
SQUASH_HISTORY: true
This defines a scheduled run each night at 17 minutes past 1am, the ROS (humble, rolling) and Ubuntu (jammy) distributions to compile against, and the repos file used as a source.
In this case, it uses the repos defined in moveit2_tutorials, but a local sources.repo also works.
The Action result is a new branch, named <deb_distro>-<ros_distro> with the generated packages:
github.com/moveit/moveit2_packages/tree/jammy-rolling
How to use your built packages
A readme is created in the output branch with instructions on how to use it, reproduced here:
echo "deb [trusted=yes] https://raw.githubusercontent.com/moveit/moveit2_packages/jammy-rolling/ ./" | sudo tee /etc/apt/sources.list.d/moveit_moveit2_packages.list
echo "yaml https://raw.githubusercontent.com/moveit/moveit2_packages/jammy-rolling/local.yaml rolling" | sudo tee /etc/ros/rosdep/sources.list.d/1-moveit_moveit2_packages.list
A user only needs to run these two commands to have a source for daily updated packages of MoveIt 2! Note; we still recommend using the official releases from packages.ros.org for long-term stable development.
Further considerations
You may have noticed the SQUASH_HISTORY parameter in the Action configuration.
Squashing the history of the output branch keeps the repository smaller as it does not keep adding binaries.
You can remove that, and the Action will keep the history of old commits with packages.
If you keep the old commits, you can revert to old or stable versions by adopting the commit hash in the apt source.
Additionally, you can bisect bugs without compiling.
It is also possible to chain the GitHub Action via multiple branches or repositories if you want to separate packaging actively developed projects nightly from slow-moving upstream dependencies.
Lastly, you may have noticed the [trusted=yes] parameter in the apt source.
Distributions like Debian or Ubuntu use gnupg to sign all metadata, making it impossible for a third party to alter them and induce malware.
The current Action implementation does not do that and only relies on HTTPS as transport security.
This is not as secure, but as we aim for an automated setup that trusts GitHub anyway, it does not make a difference.
Final words
This relies on GitHub for providing CI minutes and git storage for free, so please don’t overuse it. Thanks a lot to GitHub for providing the infrastructure to open source projects! Go ahead, use it and build packages!
