Lesson 3: Understanding Yocto — From Source Code to a Linux Image
3.1 First, why do we need Yocto?
Imagine you’re developing software for your Qualcomm board.
You need:
- A Linux kernel compatible with the processor and board hardware.
- Basic Linux commands and libraries.
- Your own applications, such as IDS and LCM.
- Configuration files and startup services.
- A root filesystem containing all the necessary software.
- A final image that can be deployed to the board.
You could theoretically assemble everything yourself.
You would download Linux, compile it, download each library, compile your applications, arrange the filesystem directories, handle dependencies, and construct the final image.
But imagine repeating this for every release, every developer, and every board variant.
It becomes difficult to manage.
Yocto helps automate and organize that entire build process.
One important correction: Yocto is not itself a Linux distribution, and it isn’t the software running on your board.
It is a project providing tools, build metadata, and methods for creating customized embedded Linux distributions.
Your board runs the Linux system produced by the build process, not the Yocto build tools themselves.
3.2 Yocto, BitBake, OpenEmbedded and Poky — what’s the difference?
You’ll frequently hear these four names together.
Think of a restaurant:
BitBake is especially important. It reads the available build instructions, works out which jobs depend on others, and executes the necessary tasks.
For example, imagine your IDS executable needs libcrypto.
BitBake can determine that the library must be available when IDS is built, provided the recipe declares the correct dependencies.
You don’t have to manually compile every prerequisite in the right order.
3.3 What are layers?
This is one of the most important Yocto concepts.
A layer is simply a directory containing build instructions, configurations, and related files.
It helps keep different parts of a project organized.
Imagine your company has three teams:
- Platform team: Maintains board support and the Linux kernel.
- IDS team: Builds intrusion-detection software.
- LCM team: Builds lifecycle-management software.
It would be messy if every team edited the same central build configuration for everything.
Instead, teams can organize their software into separate layers.
The name meta- is a convention you'll see in many Yocto projects.
A real Qualcomm example — Qualcomm maintains an OpenEmbedded/Yocto BSP layer called meta-qcom. It provides recipes and hardware-related configuration for supported Qualcomm platforms.
Qualcomm Linux also has distribution-related layers, such as meta-qcom-distro, in its published Yocto environments.
Your particular automotive board may use a separate vendor or company BSP. Don’t assume that the publicly available meta-qcom layer supports your exact hardware.
So how does Yocto know which layers to use?
Through a configuration file named:
build/conf/bblayers.conf
A simplified example:
BBLAYERS = " \
/work/poky/meta \
/work/meta-qcom \
/work/meta-platform \
/work/meta-ids \
/work/meta-lcm \
"These paths are illustrative.
It tells the build system which layers are enabled. It doesn’t automatically mean every recipe in those layers will be included in the final image.
That’s the next distinction.
3.4 What is a recipe?
A recipe is a file that tells BitBake how to build a particular piece of software.
Recipe filenames usually end in .bb.
For example:
ids-daemon_1.0.bbThe typical filename has two parts:
ids-daemon: The recipe's software name.1.0: Its version.
Think of a recipe as instructions for preparing a meal.
A recipe might describe:
- Where to obtain the source code.
- Which libraries are needed during compilation.
- Which compiler/build method to use.
- How to install the finished files into a temporary directory.
- Which packages those installed files should belong to.
For example, your IDS layer might look like:
meta-ids/
├── conf/
│ └── layer.conf
└── recipes-security/
└── ids-daemon/
├── ids-daemon_1.0.bb
└── files/
├── ids-daemon.cpp
├── ids-config.json
└── ids-daemon.serviceThe recipe is responsible for describing how these files are built and prepared for inclusion in packages.
Important: recipe vs layer vs image
A single layer can contain many recipes.
One recipe can generate multiple packages, such as a runtime package, a development package with headers, and a debugging package.
An image recipe chooses which packages and features belong in the final Linux image.
This distinction will help you understand why building IDS alone is different from building the complete operating system.
3.5 What does a real Yocto recipe look like?
We’ll create a tiny program that prints a message.
Start with a C example because it has no external library dependencies. The same principles apply to C++ services.
File 1: ids-demo.c
#include <stdio.h>
int main(void) {
printf("IDS demo running on embedded Linux!\n");
return 0;
}Now we want Yocto to compile this for our target board and install it under /usr/bin.
File 2: ids-demo_1.0.bb
SUMMARY = "Simple IDS demonstration"
LICENSE = "MIT"
LIC_FILES_CHKSUM = "file://${COMMON_LICENSE_DIR}/MIT;md5=0835ade698e0bcf8506ecda2f7b4f302"
SRC_URI = "file://ids-demo.c"
S = "${UNPACKDIR}"
do_compile() {
${CC} ${CFLAGS} ${LDFLAGS} ids-demo.c -o ids-demo
}
do_install() {
install -d ${D}${bindir}
install -m 0755 ids-demo ${D}${bindir}
}This follows the single-C-file recipe pattern documented by Yocto. The UNPACKDIR approach shown here applies to newer Yocto releases; older projects may use a different source-directory layout.
Let’s understand it.
There is one subtle but important point.
When do_install() runs, it does not install the application directly onto your Qualcomm board.
It places files into a temporary destination on the build machine. Yocto then packages those files for use in the image or for separate deployment.
That’s how a package is created.
3.6 How does a package become part of the image?
Suppose the recipe has successfully produced an ids-demo package.
Get Ankush Dhanda’s stories in your inbox
Join Medium for free to get updates from this writer.
That does not necessarily mean the package is included in your Linux image.
You must explicitly include it through image configuration or another dependency mechanism.
For example, during development, you could add this to the build’s conf/local.conf:
IMAGE_INSTALL:append = " ids-demo"The leading space inside the quotes is intentional.
This tells the build system to add the ids-demo package to the image's installed package set.
For a maintained product, you would normally express this in a proper image recipe, packagegroup, or product layer rather than leaving it as a developer-only override.
The resulting image could contain:
/
├── usr/
│ ├── bin/
│ │ └── ids-demo
│ └── lib/
│ └── ...
├── etc/
├── var/
└── ...After that image is flashed and the board boots, you could run:
/usr/bin/ids-demoExpected output:
IDS demo running on embedded Linux!For the real IDS service, we’d also account for shared libraries, configuration files, runtime dependencies, and a systemd service unit.
3.7 What happens internally when BitBake builds software?
Let’s say you execute the following on a properly configured Yocto build host:
bitbake ids-demoBitBake performs a series of tasks.
Some recipes require additional steps, and certain tasks may do very little depending on how the software is built. BitBake also handles dependencies and numerous quality checks along the way.
Now, notice the difference between these two commands:
bitbake ids-demoBuilds the application recipe and the work needed for it.
bitbake core-image-minimalBuilds a minimal Linux image according to the selected machine and distribution configuration.
core-image-minimal is a generic Yocto example target, not necessarily the image target for your Qualcomm board.
For your board, we’ll need to identify the image name supplied by its BSP or project.
3.8 What does Yocto generate after a build?
You might have seen a large directory called tmp inside a Yocto workspace.
A typical build output structure looks something like:
build/
├── conf/
│ ├── local.conf
│ └── bblayers.conf
│
├── downloads/
│
├── sstate-cache/
│
└── tmp/
├── work/
│ └── ...
│
└── deploy/
├── images/
│ └── <machine>/
│ ├── rootfs.ext4
│ ├── rootfs.tar.gz
│ ├── kernel-image
│ └── ...
│
├── rpm/
├── ipk/
├── deb/
└── sdk/This is illustrative. The actual files depend on your build configuration, and a project may generate only one package format.
Why is sstate-cache important?
Imagine your Yocto image contains 500 software components.
You modify only IDS.
Should BitBake compile all 500 components again?
Usually no.
If cached results are still valid and the dependencies haven’t changed, BitBake can reuse previous work.
That’s one reason incremental builds can be much faster than a clean build.
A first-time build may still need to download, compile, and package a large number of components.
3.9 Where do the toolchain and SDK fit?
Remember our example: your development laptop may have an x86–64 processor, but the Qualcomm board may run ARM.
A program compiled normally for your laptop is not automatically compatible with that board.
We need a cross-compiler that runs on the build machine but creates binaries for the target.
An SDK packages development tools and target-compatible development files so you can build applications outside a full Yocto image build.
For example:
Yocto image build
|
+----> Linux image
|
+----> SDK (when requested)
|
+--> Cross-compiler
+--> Headers
+--> Target libraries
+--> Build environment setupYocto can generate an SDK with a command such as:
bitbake <image-name> -c populate_sdkThe output normally appears in tmp/deploy/sdk/.
This is particularly useful for teams building IDS or LCM without running an entire image build for every code change.
We’ll explore SDKs, public headers, linking, and shared APIs in detail in Lesson 5.
3.10 Hands-on with your Qualcomm board
For now, we won’t flash or modify anything.
Our first goal is to understand the environment you already have.
There are two places you may need to work:
Build host
Linux workstation or CI build server where Yocto and BitBake execute.
Qualcomm board
The target device running the Linux system produced by the build.
Exercise A: Identify the Linux system on your board
Connect through SSH or its serial console, then execute:
uname -a
uname -m
cat /etc/os-release
cat /proc/cmdlineYou’ll learn the kernel version, CPU architecture, system identification information, and boot arguments.
Next:
findmnt /
df -h
cat /proc/mtdThese can help identify the root filesystem, available storage, and raw MTD flash devices. Some minimal images lack findmnt, and /proc/mtd may be absent.
The presence of particular files in /etc/os-release does not, by itself, prove a specific Yocto build was used.
Exercise B: Inspect a Yocto environment on your development machine
If you have access to the actual Yocto source/build workspace used for your board, initialize its build environment using the project’s documented setup script.
In a conventional Poky checkout, you might run:
source oe-init-build-envThen:
# Show enabled layers
bitbake-layers show-layers# Show available machine configurations
bitbake-layers show-machines# Inspect the current machine and distro
bitbake -e | grep -E '^(MACHINE|DISTRO)='
These commands are for the Yocto build host, not normally for the board.
You may discover layers such as meta-qcom, an automotive BSP, and company-specific platform layers.
Do not change the MACHINE setting merely because another Qualcomm board uses it. It must match your supported hardware configuration.
If you only have the board and not the company’s Yocto workspace, Exercise A is enough for today.
Lesson 3 recap
The biggest concept is that Yocto is a build system for producing a complete embedded Linux software environment, rather than an application that runs on the board.
- Layer organizes build configuration and recipes.
- Recipe explains how to build a software component.
- BitBake coordinates tasks and dependencies.
- Package contains built software and associated files.
- Image recipe chooses what belongs in the system image.
- Image artifact is what eventually gets deployed to storage.
- SDK helps developers compile applications for the target.
