Visualising the Architecture of Qt Embedded Systems: Context and Container Diagr...

I never got the hang of UML: overly complicated, hard to understand, too close to code. Hence, I was skeptical about C4 diagrams when I first heard about them. When I saw Simon Brown using them in his talk Modular Monoliths, I immediately knew: C4 diagrams are the simple and flexible notation I have always wanted. And here I am using them for visualising Qt embedded systems.

The C4 Model

Simon Brown created the C4 model for visualising software architecture. The C4 model looks at software on four abstraction levels.

• Context: An embedded system is part of a larger ecosystem: the system context. The Context diagram describes the interactions between the embedded system and other systems like users, machines, cloud services and operating conditions.
• Containers: An embedded system consists of one or more executable units: the containers. The containers of an infotainment system are the radio, media, phone and navigation applications and their services. The Container diagram shows the responsibilities of the executable units, their interactions and the technological choices for the interactions. Note: These containers have nothing to do with Docker containers.
• Components: Containers are decomposed into modules or components. The Component diagram shows the responsibilities of the components and their interactions. It also shows which technology (e.g., QML, Qt, Boost, Protobuf or C++) is used to implement a component.
• Code: Code diagrams should be generated automatically. I use them rarely, because expressive unit tests from TDD are typically more illuminating than code diagrams. A good use could be the visualisation of influential functional requirements that shape an architecture.

Simon Brown suggests a simple notation for these diagrams “that works well on whiteboards, paper, sticky notes, index cards and a variety of diagramming tools”. The brilliant thing about the notation is that we can use it on all four abstraction levels. There is no need to use different notations on different levels.

The three boxes in the top row are mostly used in Context or system-level diagrams. I added the brown box for constraints, which are unchangeable design decisions like the operating conditions. I will highlight the system, container or component of interest with a blue background. The bottom row shows the boxes for containers and components and the arrow for relationships between entities in the same diagram. The rest of this post will have plenty examples.

Let us apply the C4 model to the diagrams in my posts about architecture and visualise a harvester terminal on context, container, component and code level. I’ll look at the upper two Cs in this post and come back to the lower two Cs in a later post.

Level 1: Context

The Context diagram shows how the Terminal cooperates with other systems (Harvester ECUs, Harvester Cameras, IoT Cloud), users (Driver) and constraints (Operating Conditions).

• The Driver assesses the situation by looking at the Terminal’s display, at the Camera displays and out of the cabin’s windows. He acts on on the gathered information, for example, by slowing down the harvester to leave fewer roots on the field, by increasing the turbine speed to clean the roots better, or by cutting the leaves higher to avoid cutting into the roots. The Driver interacts with the Terminal via multi-touch, two rotary knobs and several hard buttons.
• The Harvester ECUs send parameter values to the Terminal so that the Driver can act properly. They receive parameter values from the Terminal, for example, to reduce the engine speed, to increase the turbine speed, or to lift the knives for cutting the leaves. The Terminal and the Harvester ECUs communicate over 2 CAN busses using the J1939 protocol. ECUs control the engine, steering, drive, header, bunker, conveyor belts, AC, radio, joystick, rotary knobs, hard buttons and other harvester parts.
• Harvester Cameras stream video over an Ethernet link to the Terminal. A sugar-beet harvester, for example, has a 270-degree rear-view camera and a turbine camera. Currently, the Driver must decide whether there is an obstacle behind or beside the harvester and whether the turbine cleans the dirt from the roots well enough. In the future, the cameras may use image recognition to automate these tasks. A third camera may decide whether the shovels must dig deeper or whether the knives must cut higher to avoid injuring the roots.
• The Terminal logs harvester data to the IoT Cloud using the MQTT protocol over an LTE-M link. Earlier Context diagrams may leave the protocols unspecified. The diagram below includes some architectural decisions (hopefully documented in Architectural Decision Records). The IoT Cloud logs data for fleet management and accounting. It also coordinates OTA updates of the Terminal, the Harvester ECUs and the Harvester Cameras.
• Operating Conditions like light, temperature, water, dust and vibration constrain the Terminal’s hardware and software. The vibration of a 1000-horse-power harvester renders combo boxes, menus and sliders nearly unusable. Working on the field at day and night makes a bright and adjustable display a must. I extended the C4 notation by a Constraint element for the Operating Conditions. Such a Constraint element is crucial to embedded systems.

So far, we looked at the Context diagram only from the technical perspective. We should also look at it from the social perspective: how to structure the teams that develop the Terminal, Harvester ECUs, Harvester Cameras and IoT Cloud. The link between the two perspectives is Conway’s law: The system architecture reflects the organisation’s structure. This rarely rarely results in a good, right and successful architecture.

The predecessor of the terminal, which I built for the ROPA sugar beet harvesters, was built by a single team. The Terminal and the ECUs were tightly coupled. It was impossible to replace the Terminal with another one, because the communication between Terminal and ECUs relied on internals from the other side. This is a clear case of supplier lock-in.

When developing the new terminal, we had three teams: one for the Terminal (including the Cameras), one for the Harvester ECUs and one for IoT Cloud. The result was a loosely coupled architecture of systems communicating through clearly defined interfaces. The Terminal used the Harvester ECUs and the IoT Cloud as services. While defining the interfaces, the collaboration between the respective two teams was high. Once defined, there was little communication needed. Each of the three teams was run as a feature team.

The Terminal team took care of the Harvester Cameras, as they just streamed video. If the cameras took decisions based on image recognition, we would have needed a separate complicated-subsystem team. The Terminal team would have had the expertise to do that. Matthew Skelton and Manuel Pais explain in their book Team Topologies how to structure teams based on the architecture (see also my review in Episode 27 of my newsletter).

Without doubt, the Context diagram is much more expressive than my makeshift diagram from Architecture of Qt Embedded Systems: Getting Started.

Level 2: Containers

The Container diagram focuses on the system of interest from the Context diagram. It shows the runnable units – applications with or without GUI – that make up the Terminal system and how they interact. In my post Architecture of Qt Embedded Systems: Single vs. Multiple GUI Applications, I describe how a system with a single GUI application evolves into a system with multiple GUI applications – driven by architecturally significant requirements (ASRs). This section provides the Container diagrams for this evolution.

When I built my first harvester terminal in 2012, the Terminal system consisted of a single GUI application. The system didn’t need a separate window manager, as the single GUI application rendered itself to the single window using Qt’s eglfs platform. The Container diagram is the same as the Context diagram except for the blue box in the middle: the name becomes Terminal Application and the type [Container: QML, Qt, C++].

Once the harvesters had a telematics unit with a 3G or LTE-M modem on board, the harvester OEMs had three more ASRs:

• They wanted remote support. Their support engineers should be able to see the display contents of each harvesters from their offices. A simple solution is to run a VNC server on the Terminal system. The VNC server sends the frame buffer contents to an office computer with a VNC client anywhere in the world.
• They wanted to update the application, the Qt libraries and possibly the whole Linux system over the air (OTA). The Terminal Application can update itself. However, it’s a lot easier if a second GUI application – the Update Application – performs the update. Especially for system updates, the Update Application may be executed by a Linux system running from a RAM disk.

• They wanted to log relevant machine data into the IoT Cloud, say, for fleet management and accounting. The Terminal needs an IoT Client that forwards the data from the Terminal System and the Harvester ECUs to the IoT Cloud. The IoT Client may be supplied by the IoT Cloud provider or developed by another team. Therefore, it is deployed as a separate non-GUI application.

Integrating a VNC server for remote support and an Update Application into the Terminal System makes a window and application manager nearly inevitable. Wayland-based compositors like Weston and the QML compositor have become the standard choice over the last 5-10 years.

Both the Terminal Application and the IoT Client read messages from the CAN busses. As soon as one application has received a message, the other won’t see it any more. Having two applications sending the same message with different parameter values needs some coordination as well. We must move the responsibility for receiving and sending CAN messages from the Terminal Application, the IoT Client and possibly other applications into the J1939 Service, a non-GUI application or service.

The Window & Application Manager uses the Wayland protocol to show and hide the GUI applications (purple boxes). The GUI applications and services (blue boxes) communicate with each other through Qt Remote Objects, which makes signals, slots and properties available for inter-process communication.

We could partition the Terminal Application into more GUI applications by its main responsibilities: Main, Customer, Camera, ECU Fine-Tuning, Diagnosis, Field Navigation, Calibration, Settings, Light and Climate Application. Drivers spend 90-95% of their time during the harvest in the Main Application. They need the other applications occasionally.

The partitioning leads to smaller applications with fast start-up times. It also makes the whole Terminal System more reliable. If one application has a bug, the other applications are not affected. This is paramount for the Main Application, which should run without any problems through the 6-8 week harvest.

This fine-grained partitioning is shown in the next Container diagram. App 1…N stands for more GUI applications like Field Navigation, Settings, Light and Climate.

We started with a Terminal System implemented by a monolithic Terminal Application. As good architects, we would have structured the application as a modular monolith. The containers from the above diagram would be visible as components or modules, waiting to be turned into applications or services. The Component diagram of the monolithic Terminal Application would look very similar to the Container diagram of the Terminal System above.