`rsdk`

rsdk is the latest system development kit from Radxa.

It is built upon our previous experiences with rbuild and debos-radxa, while also adds many other utilities to improve the developer / maintainer experience.

Why another system builder?

rsdk is yet another evolution of Radxa's system builder. The primary goal is to move away from debos for following reasons:

(Previously) Lack of ARM64 support.
We use pagefile to workaround dkms build failure, which requires KVM to be available.
Go templates leave much to be desired.
Provided actions are becoming limiting to achieve our desired result.

Beyond those, our system has grown into 100+ repositories, interconnected by various CI/CD workflows. From an operational standpoint we also need a tool to centrally manage them, with the added bonus that customers can use the very same tool to reproduce our setup for their own use.

Finally, we want to have a short answer to the commonly asked question "where can I download the SDK for X". So far, our answer has always been a brief explanation of the system design, because of its complexity. We want to have something that matches customer's expectation more closely, where there is only a single entry to all related Radxa source code.

Thus, rsdk is born.

Run `rsdk` with Visual Studio Code & devcontainer

Info

This is the preferred method to run rsdk.

First, please install the required dependencies:

sudo apt-get update
sudo apt-get install git qemu-user-static binfmt-support

Then follow Visual Studio Code's documentation to:

Then clone the project with git:

git clone --recurse-submodules https://github.com/RadxaOS-SDK/rsdk.git

Open the project in Visual Studio Code. A notification will pop up on the corner asking if you want to reopen in devcontainer. Click yes and wait for the container to be set up.

This can be combined with VS Code remote development extension to run rsdk in other systems.

Common issues

devcontainer setup paused with You might be rate limited by GitHub. message

You might be rate limited by GitHub. Please follow the instruction listed in the output.
Failed to launch devcontainer.

Please edit .devcintainer/devcontainer.json, and adjust runArgs property.

Run `rsdk` with devcontainer alone

This is similar to how we build system images in the CI pipelines.

First, please install the required dependencies:

sudo apt-get update
sudo apt-get install git qemu-user-static binfmt-support
sudo apt-get install npm docker.io
sudo usermod -a -G docker $USER

If you were not in the docker group before, you will need to log out and log back in.

For SSH, simple disconnect the current session and reconnect.

Then clone the project with git and install devcontainer:

git clone --recurse-submodules https://github.com/RadxaOS-SDK/rsdk.git
cd rsdk
npm install @devcontainers/cli
export PATH="$PWD/src/bin:$PWD/node_modules/.bin:$PATH"
rsdk devcon up
rsdk devcon

You are now inside the rsdk's devcontainer shell.

The following recording demostates how to set up rsdk and build an image on a fresh Debian 12 install:

Common issues

devcontainer setup paused with You might be rate limited by GitHub. message

You might be rate limited by GitHub. Please follow the instruction listed in the output.
Failed to launch devcontainer.

Please edit .devcontainer/devcontainer.json, and adjust runArgs property.

Run `rsdk` using released Debian package

To be released.

Run `rsdk` natively

Warning

This is not a well supported or tested installation method.

When you have issues, please check the source code for up-to-date runtime dependencies.

We do not answer support questions related to this type of installation.

To run rsdk natively, you will ideally need an Ubuntu system, as it is the base system used in Dev Container.

Please first install devenv on your system.

Optionally you can setup direnv, then run direnv allow within the project folder to enable it.

You can then run devenv shell to enter the development shell. This shell will manipulate your PATH to have the development dependency available.

If you have direnv setup, you don't have to run the above command when you enter the project directory to use the SDK. However, the direnv shell lacks starship integration as well as rsdk auto completion, as it only saves the environmental variables.

There are some additional system configurations and packages that are currently not managed by devenv. They are mostly covered by install_native_dependency function in rsdk-setup.

Basic usage

rsdk comes with bash-completion to assist its CLI usage.

After typing rsdk into your terminal, if there is no subcommand suggestion after you press Tab key twice, you should run rsdk shell to enter the development environment.

From there, you can use autocompletion to query the supported command list.

Running a command with --help argument will display its help text.

Use TUI to run guided tasks

rsdk command is the common CLI entry point.

When it is run without any argument, rsdk-tui will be run instead.

Start TUI Wizard by running rsdk in the terminal.
Select the task you want to run.
You can use arrow key to navigate, use Enter key to select, and use Escape key to leave current window.
In this example, we will choose Build system image:

┌─────────────────┤ RSDK ├──────────────────┐
│ Please select a task:                     │
│                                           │
│            Build system image             │
│            =========                      │
│            About                          │
│                                           │
│         <Ok>             <Cancel>         │
│                                           │
└───────────────────────────────────────────┘

Select the product you want to build.
You can use the space bar to select the product, and the chosen product will have asterisk symbol placed in the parentheses.
Enter key does not select the product when used alone!
In this example, we will choose rock-5b-6_1:

┌─────────────────┤ RSDK ├──────────────────┐
│ Please select a product:                  │
│                                           │
│  ( ) radxa-e25                        ▒   │
│  (*) rock-5b-6_1                      ▒   │
│  ( ) rock-5b                          ▒   │
│                                           │
│         <Ok>             <Cancel>         │
│                                           │
└───────────────────────────────────────────┘

Select Yes to start the build process.
rsdk-tui will then run the associated CLI commands to complete the task:

┌─────────────────┤ RSDK ├──────────────────┐
│                                           │
│ Are you sure to build for 'rock-5b-6_1'?  │
│                                           │
│                                           │
│          <Yes>             <No>           │
│                                           │
└───────────────────────────────────────────┘

Advanced build should use rsdk-build command instead.

Migrating from old toolchain

`rbuild`

rsdk build is the rbuild replacement. It uses the same argument ordering, and support similar input values.

When no suite or edition is supplied, rsdk build will use the value defined in src/share/rsdk/configs/products.json instead of bullseye cli.

Not all rbuild options are supported by rsdk build. Please check the command help for more details.

.rbuild-config file is now replaced by devenv.local.nix, and option names are adjusted. You can check devenv.local.nix.example for syntax.

`rbuild json`

This subcommand is replaced by configuration files under src/share/rsdk/configs.

`rbuild shrink-image`

This subcommand is no longer needed, as now image shrinking is no longer a root operation, and comes standard.

`rbuild write-image`

rsdk write-image is the rbuild write-image replacement. It uses the same argument ordering.

Build customization

rsdk uses devenv to manage its development envrionronment, but itself is not nix based. nix has good support generating NixOS images, but here we are dealing with a Debian system, and some parts are currently missing.

We currently uses bdebstrap, which is a YAML frontend for mmdebstrap. mmdebstrap is the recommended tool for building RISC-V systems compared to traditional debootstrap.

YAML itself is not a templating language, so it cannot dynamically generate different configurations for different builds. We use jsonnet for this purpose.

Lastly, mmdebstrap does not handle disk image generation. We again use jsonnet to dynamically generate a guestfish script to handle this task.

All of those tools are glued by bash to provide a frontend, known as rsdk-build.

Depending on your goal, you would need the knowledge of some of the above tools.

Rootfs customization

rootfs.jsonnet is the entry point for rootfs template. It collects the various inputs and passes them to different modules.

Module loading order matters, as that will determine *-hooks's execution order. The safest way is to only edit the customize-hooks field in rootfs.jsonnet and only adding new entries after the existing ones.

You can also edit the packages field in rootfs.jsonnet to add additional packages to the system. customize-hooks will be run after packages are installed in the rootfs.

Checkout Work with local packages if you have local packages to install.

Disk image customization

image.jsonnet is the template for the deployment script. It is generally not necessary to change this part.

Adding support for new device

Every device supported by RadxaOS requires several metapackages available in the apt repositories. They are usually unique to each hardware, as such cannot be shared between devices.

It is intentional to keep this info in form of package dependencies, and spread out in several packages, instead of a single hard coded config file. This is because:

Different build tools don't need to track this config in order to build with correct packages.
Ease of migration between tools.
For example, from rbuild to rsdk, we use the same simple package logic instead doing data transform, which was the case for rbuild's configs files.
Allow underlying package to be changed/updated in the future.

Adding new device

Currently, there are 4 places that need to be updated:

Kernel metapackages
Those includes Linux kernel image as well as kernel header, and some other less used packages.
Currently you need to edit the related bsp Linux profile to include the new device.
Firmware metapackage
This is similar to the kernel package in case of U-Boot, where the U-Boot profile needs to be updated.
For EDK2 you will need to add the metapackage in the related repo.
Product metapackage
Each product also needs its own package to pull device-specific drivers, and some common config options. Those are defined in radxa-profiles repo.
products.json
This registers the supported products for rsdk and their default configurations.

Right now, product metapackage does not specify dependencies to kernel and firmware (even though we have product-agnostic metapackages for those two). This is to allow users to install custom kernels only without breaking the dependencies.

Note that vendor metapackage is listed as recommended in product metapackages. This way users can install the system without SoC vendors' binary packages.

Finally, create the new product build repo under radxa-build with rsdk-infra-product-update command.

What if we introduce a new SoC?

You need to additionally update following places:

Vendor metapackage
Each SoC needs their own package to pull SoC-specific packages, and some common config options. Those are defined in vendor-profiles repo.
socs.json
New SoC MUST be added to soc_specific_repo array as well.
New SoC-specific apt repo under radxa-repo organization.
SoC-specific package repo
In case of Rockchip, those are managed by rockchip-prebuilt repo, and we create Rockchip SDK prebuilt packages in GitHub Releases, which will be uploaded to the SoC-specific apt repo.
If the new SoC is only a variant of already supported SoC, then you only need to edit the related pkg.conf to include the new SoC-specific apt repo.

What if we introduce a new SoC vendor?

You need to additionally update following places:

Vendor metapackage
Each vendor needs their own package to pull common config options. Those are defined in vendor-profiles repo.
socs.json
There should be a new item for the new vendor.

Global build options

Like rbuild, rsdk also allows defining global build options. The underlying mechanical is the same: predefined environmental variables. However, since we use devenv to manage the environment, the way to define them is also changed.

For now, you will need to add your definitions in devenv.local.nix file. We provide devenv.local.nix.example as a reference for file structure. This is also what we use in the office, but the server is locally hosted, so you won't be able to use it.

Example: set up local apt cache to speed up build

If you want to speed up image building time, one way is to use a locally hosted apt cache to reduce download time.

First, copy devenv.local.nix.example to devenv.local.nix. You can then remove RSDK_OPTION_REPO_SUFFIX if you do not want to build test image by default, and then change RSDK_OPTION_*_MIRROR to your own address.

Below are example NixOS configuration to set up a local apt cache service, as well as the mirror definition for acng.conf file. They may not be complete, so adjust to match your own use case.

{
  virtualisation = {
    oci-containers = {
      backend = "podman";
      containers = {
        acng = {
          image = "docker.io/mbentley/apt-cacher-ng";
          ports = [
            "3142:3142"
          ];
          volumes = [
            "/home/acng/containers/acng.conf:/etc/apt-cacher-ng/acng.conf"
            "/home/acng/containers/acng/:/var/cache/apt-cacher-ng/"
          ];
          environment = {
            PUID = "1000";
            PGID = "100";
            TZ = "Asia/Shanghai";
          };
        };
      };
    };
  };
}

Remap-debian: /debian ; http://mirrors.tuna.tsinghua.edu.cn/debian
Remap-debian-security: /debian-security ; http://mirrors.tuna.tsinghua.edu.cn/debian-security
Remap-ubuntu: /ubuntu ; http://mirrors.tuna.tsinghua.edu.cn/ubuntu
Remap-ubuntu-ports: /ubuntu-ports ; http://mirrors.tuna.tsinghua.edu.cn/ubuntu-ports
Remap-armbian: /armbian ; http://mirrors.tuna.tsinghua.edu.cn/armbian
Remap-proxmox: /proxmox ; http://mirrors.tuna.tsinghua.edu.cn/proxmox/debian
Remap-proxmox-new: /debian/pve ; http://mirrors.tuna.tsinghua.edu.cn/proxmox/debian/pve
Remap-rbuild: /rbuild ; http://radxa-repo.github.io/apt
Remap-rbuild-buster: /rbuild-buster ; http://radxa-repo.github.io/buster
Remap-rbuild-buster-test: /rbuild-buster-test ; http://radxa-repo.github.io/buster-test
Remap-rbuild-bullseye: /rbuild-bullseye ; http://radxa-repo.github.io/bullseye
Remap-rbuild-bullseye-test: /rbuild-bullseye-test ; http://radxa-repo.github.io/bullseye-test
Remap-rbuild-rk3528a-bullseye-test: /rbuild-rk3528a-bullseye-test ; http://radxa-repo.github.io/rk3528a-bullseye-test
Remap-rbuild-focal: /rbuild-focal ; http://radxa-repo.github.io/focal
Remap-rbuild-focal-test: /rbuild-focal-test ; http://radxa-repo.github.io/focal-test
Remap-rbuild-jammy: /rbuild-jammy ; http://radxa-repo.github.io/jammy
Remap-rbuild-jammy-test: /rbuild-jammy-test ; http://radxa-repo.github.io/jammy-test
Remap-rbuild-bookworm: /rbuild-bookworm ; http://radxa-repo.github.io/bookworm
Remap-rbuild-bookworm-test: /rbuild-bookworm-test ; http://radxa-repo.github.io/bookworm-test
Remap-rbuild-rk3588-bookworm-test: /rbuild-rk3588-bookworm-test ; http://radxa-repo.github.io/rk3588-bookworm-test
Remap-rbuild-rk3588s2-bookworm-test: /rbuild-rk3588s2-bookworm-test ; http://radxa-repo.github.io/rk3588s2-bookworm-test
Remap-rbuild-sid: /rbuild-sid ; http://radxa-repo.github.io/sid
Remap-rbuild-sid-test: /rbuild-sid-test ; http://radxa-repo.github.io/sid-test
Remap-rbuild-noble-test: /rbuild-noble-test ; http://radxa-repo.github.io/noble-test
Remap-rbuild-vscodium: /rbuild-debs ; https://paulcarroty.gitlab.io/vscodium-deb-rpm-repo/debs

Work with local packages

Background

When working at the bleeding edge, developer needs to test some locally built packages before committing the changes. However, to ensure the build is reproducible, maintainer wants to lock up all inputs, usually in a version-controlled way.

At Radxa, this conflict primarily impacts our BSP developers, who are not involved in toolchain development. As such, rbuild provides some helper flags to allow better integration with bsp's build outputs. The scope of local packages is intentionally limited to kernel and firmware packages, since the development workflow for the OS maintainers is different: just test the package on a live system.

rsdk initially dropped this support. As an evolution of rbuild, the package situation was stable enough that BSP developers did not need to build images with custom packages very often. The support is now available in more general form, in hope to support fully offline system building in the future.

`rsdk-build --debs`

rsdk-build now supports --debs <debs_dir> flag. A folder containing locally built packages are needed with this flag.

When this flag is specified, the content of debs_dir will be copied inside the target system, and will be added as a local apt repository under /srv/local-apt-repository, and it will have pin priority of 1999, effecitive making it the only provider of the included package.

Warning

Unlike rbuild, this local apt repository will persist in the rootfs. This will block future package upgrade via apt if it is available in an online source, and also serves as the build input for future reproduceible build.

Consider clear /srv/local-apt-repository after the build if this behaviour is undesired.

When running rsdk in devcontainer, we recommend debs_dir to be inside rsdk project folder (for example, the debs folder in the project root). As the host path may not be mapped inside devcontainer, rsdk-build may not be able to access it.

Add packages that are not part of the default install

Because the package are not explicitly installed like the case of rbuild, if you want a package to be installed, it has to also be installed by the normal build steps.

If your package is not installed by default, you will need to follow Rootfs customization to add them to the build steps.

Override packages that are part of the default install

For example, if we want to replace the U-Boot for ROCK 4SE, we need to provide both the u-boot-rock-4se metapackage, which is installed by the build script, and the u-boot-latest binary package, which provides the bootloader and is u-boot-rock-4se's dependency.

Usually you should copy every generated packages to debs_dir for each project you are going to override. This is to reduce the likelihood of missing a dependency.

rsdk

rsdk command is the common CLI entry point.

When it is run without any argument, it will run rsdk-tui instead.

Build RadxaOS image

When no suite or edition options is supplied, rsdk-build will use the product-specific default values, which are defined in src/share/rsdk/configs/products.json as the first element of the respective array.
Using ROCK 3C as an example, if you want to build a CLI image for RadxaOS Bullseye, you can run the following command:

rsdk build rock-3c bullseye cli

RadxaOS output path

you can find the generated RadxaOS image as out/${product}_${suite}_${edition}/output.img.

`rsdk-shell`

It is recommended to run rsdk shell first to set up the work environment before running other commands.
However, most commands should run without issue.
Some enhancements (for example, bash completion and shell prompt) may not be available outside of rsdk shell by default.