Setting Up Yocto Projects with kas

Kas makes the setup of a Yocto build environment super simple and super fast. We call kas with a project configuration file: kas-container build ./eu-terminal-distro.yml. Kas starts a Docker container, clones the layer repositories, initialises the Yocto configuration files (local.conf and bblayers.conf), and starts building the embedded Linux system. Most Linux BSP providers don’t make a kas configuration file available. I’ll show how to convert a repo manifest file into a kas configuration file in this post.

Introduction

In the post Qt Embedded Systems – Part 1: Building a Linux Image with Yocto, I go through the steps how to build the Linux image for a Raspberry Pi 3.

• We build a Docker container image and run a Linux shell in the container.
• We download the layer repositories as given in a repo manifest file.
• We edit the configuration files bblayers.conf and local.conf.
• We build the target Linux image in the container.

Each of the four steps consists of a couple of smaller steps that we must execute manually. Setting up a Yocto build environment this way and reaching the point where we can build the target Linux image with BitBake is tedious process.

Given a project configuration file, kas reduces this process to a single command:

$ kas-container build project-configuration.yml

The project configuration file, which is written in YAML, is very similar to the repo manifest file. It also contains all the information needed for generating the configuration files bblayers.conf and local.conf – in contrast to the manifest file. The kas-container command performs all the four steps from the traditional setup in one go. Super simple and super fast!

In the remainder of this post, I’ll describe how to create the kas project configuration file for the Toradex Verdin iMX8M Mini. I’ll only use the information available from the traditional setup.

Installing kas

We start the installation with cloning the kas repository in any directory (e.g., /public/Projects), where we have read and write access as a normal user.

$ cd /public/Projects
$ git clone https://github.com/siemens/kas.git

Kas is written in Python3. We must install Python3, Pip3 and some Python packages.

$ sudo apt install python3 python3-pip
$ sudo pip3 distro jsonschema PyYAML

If the command

/public/Projects/kas/run-kas --help

doesn’t show any Python error messages but just the usage of run-kas, our Python installation is OK.

The kas documentation lists four options how to use kas.

• As seen above, we can run the script /public/Projects/kas/run-kas from the kas repository. If we add /public/Projects/kas to $PATH, we can simply type run-kas on the command line.
• We can install kas as a system Python script by running sudo pip3 install . in the directory /public/Projects/kas. Then we can type kas at the Linux prompt.
• We can run kas in a container locally with the script /public/Projects/kas/kas-container. If we add /public/Projects/kas to $PATH, we can simply type kas-container on the command line.
• We can run kas in a container remotely for CI. This is out of scope for this post.

Yocto builds fail, when we run them on too old or on too new host Linux versions. Hence, the first two options will run into problems, as soon as we build target Linux systems with different Yocto versions, e.g., for the next product version or for multipe products. We avoid these problems by running the Yocto builds in a container based on a suitable Linux version. So, we choose the third option with kas-container.

When we run kas-container for the first time, it will build a Docker container image. It will run kas commands like shell, checkout and build in the Docker container. All the kas commands take a project configuration file as an argument. So, we need to create one first.

Converting a Repo Manifest into a Kas Project Configuration

The target system is Toradex Verdin iMX8M Mini computer-on-module on a Verdin carrier board. Toradex provide a manifest file tdxref/next.xml (branch master) for the repo tool at the toradex-manifest repository. The manifest file takes all the layers from Yocto Dunfell.

Toradex don’t provide a kas project configuration file. So, we must convert the manifest file into a kas project configuration.

We create an empty project configuration file in YAML format, say, /public/Projects/terminal-distro/terminal-distro.yml, and add the following properties.

header:
    version: 10
distro: tdx-xwayland
build_system: oe
machine: verdin-imx8mm
target: tdx-reference-multimedia-image

header.version is the version of the configuration format. 10 is the latest version at the time of writing. The Toradex Linux image tdx-reference-multimedia-image is based on Toradex’s reference Linux distribution tdx-xwayland and is built with OpenEmbedded oe. The target device is verdin-imx8mm. Kas adds the distro and machine to the local configuration file build/conf/local.conf.

MACHINE ??= "verdin-imx8mm"
DISTRO ??= "tdx-xwayland"

When we set up a Yocto build environment without kas, we would define the distro and machine with the environment variables DISTRO and MACHINE, respectively. We would pass the image name tdx-reference-multimedia-image to the bitbake call. We can retrieve this information from the documentation page Build a Reference Image with Yocto Project/OpenEmbedded.

The manifest file tdxref/next.xml defines which revision from which repository (remote) the repo tool shall fetch for a given Yocto layer (project). The entries for the meta-qt5 layer look as follows.

<remote alias="repo" fetch="https://github.com/meta-qt5" name="githq"/>
<project name="meta-qt5.git" path="layers/meta-qt5" remote="githq" revision="dunfell"/>

The corresponding entry in the kas configuration file looks like this.

repos:
    meta-qt5:
        url: "https://github.com/meta-qt5/meta-qt5.git"
        refspec: "dunfell"
        path: "layers/meta-qt5"

The repository ID (here: meta-qt5) in the project configuration is the project.name without the extension. The url is the concatenation of remote.fetch and project.name. The refspec is the project.revision. The path is the project.path.

The above entry tells kas to clone revision refspec of the repository url into the path relative to the current working directory. We can check this by running kas in a work directory of our choice (e.g., /public/Projects/terminal-distro/) . The final error will go away once we add the openembedded-core and meta-openembedded .

$ kas-container checkout ./terminal-distro.yml 
2021-09-17 10:20:07 - INFO     - kas 2.5 started
...
2021-09-17 10:20:07 - INFO     - /work$ git clone -q https://github.com/meta-qt5/meta-qt5.git /work/layers/meta-qt5
2021-09-17 10:20:10 - INFO     - Repository meta-qt5 cloned
...
2021-09-17 10:20:10 - INFO     - /work/layers/meta-qt5$ git checkout -q b4d24d70aca75791902df5cd59a4f4a54aa4a125 -B dunfell
2021-09-17 10:20:10 - ERROR    - Did not find any init-build-env script

For each repository entry repo-id in the project configuration file, kas creates a directory layers/repo-id containing the corresponding repository and adds an entry to the build environment layer configuration file build/conf/bblayers.conf. For meta-qt5, kas adds this line:

    /work/layers/meta-qt5 \

The conversion works fine except for the repositories bitbake and meta-openembedded from base/integration.xml.

<remote alias="repo" fetch="https://github.com/openembedded" name="oe"/>

<project name="bitbake.git" path="layers/openembedded-core/bitbake" remote="oe" revision="1.46"/>
<project name="openembedded-core.git" path="layers/openembedded-core" remote="oe" revision="dunfell"/>

The direct conversion would be:

repos:
    bitbake:
        url: "https://github.com/openembedded/bitbake.git"
        refspec: "1.46"
        path: "layers/openembedded-core/bitbake"
    openembedded-core:
        url: "https://github.com/openembedded/openembedded-core.git"
        refspec: "dunfell"
        path: "layers/openembedded-core"

Kas clones the bitbake repository successfully, but fails when cloning the openembedded-core repository.

2021-09-17 11:30:16 - INFO     - /work$ git clone -q https://github.com/openembedded/bitbake.git /work/layers/openembedded-core/bitbake
2021-09-17 11:30:16 - INFO     - /work/layers/openembedded-core$ git remote set-url origin https://github.com/openembedded/openembedded-core.git
2021-09-17 11:30:16 - ERROR    - Command "/work/layers/openembedded-core$ git remote set-url origin https://github.com/openembedded/openembedded-core.git" failed
--- Error summary ---
fatal: not a git repository (or any parent up to mount point /)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).

Cloning bitbake creates a directory tree at layers/openembedded-core/bitbake. Kas clones the repository openembedded-core into this existing directory tree – and fails. If we change the order in the project configuration (openembedded-core first and bitbake second), the checkout works fine.

When checking out several repositories for the first time, we may encounter an error like this.

2021-09-17 11:59:51 - ERROR    - fatal: unable to access 'https://git.yoctoproject.org/git/meta-yocto.git/': Could not resolve host: git.yoctoproject.org
2021-09-17 11:59:51 - ERROR    - Command "/work$ git clone -q https://git.yoctoproject.org/git/meta-yocto.git /work/layers/meta-yocto" failed

If the URL is correct, the error disappears by running kas again. Sometimes we need several runs.

The repositories meta-openembedded, openembedded-core and meta-yocto are not layers, because the don’t provide a layer configuration file conf/layer.conf. However, they are layer containers. meta-openembedded, for example, contains 10 layers like meta-oe, meta-networking and meta-multimedia. Each of these layers contains a layer configuration file.

We must explicitly list the layers that kas shall add to the layer configuration. We find the required layers in the template bitbake configuration file layers/meta-toradex-distro/buildconf/bblayers.conf. We specify the eight required layers for meta-openembedded as follows:

    meta-openembedded:
        url: "https://github.com/openembedded/meta-openembedded.git"
        refspec: "dunfell"
        path: "layers/meta-openembedded"
        layers:
            meta-oe:
            meta-filesystems:
            meta-gnome:
            meta-xfce:
            meta-initramfs:
            meta-networking:
            meta-multimedia:
            meta-python:        

Instead of one entry for the repository meta-openembedded, kas will create one entry for each layer in build/conf/bblayers.conf. Similarly, we specify meta-poky as the required layer for the repository meta-yocto and meta for openembedded-core. If we forget to specify the layers, the kas build will complain about a missing conf/layer.conf file.

ERROR: Unable to parse /work/layers/openembedded-core/conf/layer.conf: [Errno 2] file /work/layers/openembedded-core/conf/layer.conf not found

The repository bitbake needs special treatment, because it is neither a layer nor a layer container. It is just a repository providing the source code for bitbake. Kas must not create an line in bblayers.conf for the bitbake repository. Marking the bitbake “layer” as excluded does the trick.

    bitbake:
        url: "https://github.com/openembedded/bitbake.git"
        refspec: "1.46"
        path: "layers/openembedded-core/bitbake"
        layers:
            bitbake: excluded

Completing the Yocto Configuration Files

From the project configuration file, kas generates the two Yocto configuration files build/conf/bblayers.conf and build/conf/local.conf, which are not yet complete.

The following entry makes Kas add the three lines in bold face to the beginning of bblayers.conf – before the definition of the layers. These three lines are standard for bblayers.conf files.

bblayers_conf_header:
    custom-bblayers-conf: |
        POKY_BBLAYERS_CONF_VERSION = "1"
        BBPATH = "${TOPDIR}"
        BBFILES ?= ""

We add another three lines (in bold face again) to the beginning of local.conf – before the definitions of the machine and the distro.

local_conf_header:
    custom-local-conf: |
        ACCEPT_FSL_EULA = "1"
        DL_DIR = "/work/downloads"
        SSTATE_DIR = "/work/sstate-cache"

The first line accepts the Freescale EULA automatically so that the build doesn’t end up waiting for user input. The second and third line move the downloads and sstate cache out of the build directory /work/build so that other developers can reuse them, say, over NFS. This saves other developers from rebuilding the whole Linux image themselves. They use most of the build from the fast build server and perform only small incremental builds on their PCs.

As we have done several times before, we test the project configuration with the command

$ kas-container checkout ../terminal-distro.yml

If the command runs without any errors and if the Yocto configuration files are up to the mark, we can build the Linux image.

Building the Linux Image

We build the Linux image specified in the project configuration file terminal-distro.yml with the command

$ kas-container build ../terminal-distro.yml

The command runs BitBake for the image tdx-reference-multimedia-image in a Docker container:

/build$ /work/layers/openembedded-core/bitbake/bin/bitbake -c build tdx-reference-multimedia-image

The Yocto build will take a couple of hours to finish.

We can build the SDK with the command

kas-container build ../terminal-distro.yml -c populate_sdk

Kas sets up the Yocto build environment in the container for interactive use with the command

kas-container shell ../terminal-distro.yml

We can then run Yocto commands like bitbake from the Linux prompt inside the container.

Useful Resources

• Github repository with the complete project configuration file from this post. The README explains how to build the embedded Linux image for the Toradex Verdin iMX8M Mini.
• Kas repository on Github.
• Siemens: Kas User Guide. Everything you need to about kas as a user and a developer.
• Paula Santamaria: Introduction to YAML. A brief introduction into YAML. Good enough to understand the YAML in the kas project configuration files.