Table of Contents

Overview

rpi-image-gen provides a flexible, hierarchical approach to managing build variables through multiple configuration sources with clear precedence rules. The system is built around the IGconf_ prefix namespace and uses YAML configuration files.

Core Architecture

Variable Naming Convention

Excluding environment keys, configuration variables in rpi-image-gen use the IGconf_ prefix followed by a structured naming pattern:

IGconf_<section>_<key>

Where:

  • section: Logical grouping (e.g., device, image)

  • key: Specific configuration parameter (e.g., class, storage_type, hostname)

Example variables:

IGconf_device_class=pi5
IGconf_device_storage_type=sd
IGconf_image_compression=zstd

Layer Variable Declaration

Variables are declared by layer YAML files using metadata headers within comment blocks:

# METABEGIN
# X-Env-Layer-Name: rpi5
# X-Env-Layer-Category: device
# X-Env-Layer-Desc: Raspberry Pi 5 specific device layer
#
# X-Env-VarPrefix: device
#
# X-Env-Var-class: pi5
# X-Env-Var-class-Desc: Device class
# X-Env-Var-class-Required: n
# X-Env-Var-class-Valid: keywords:pi5
# X-Env-Var-class-Set: y
#
# X-Env-Var-storage_type: sd
# X-Env-Var-storage_type-Desc: Pi5 storage type
# X-Env-Var-storage_type-Required: n
# X-Env-Var-storage_type-Valid: sd,nvme
# X-Env-Var-storage_type-Set: y
# METAEND

Variable Declaration Metadata

Each variable declaration includes several metadata fields:

  • X-Env-Var-<name>: Default value for the variable

  • X-Env-Var-<name>-Desc: Human-readable description

  • X-Env-Var-<name>-Required: Whether the variable is required (y/n)

  • X-Env-Var-<name>-Valid: Validation rules (keywords, patterns, etc.)

  • X-Env-Var-<name>-Set: Variable assignment policy

Variable Prefix

The X-Env-VarPrefix field specifies the section name used in the IGconf_ namespace:

X-Env-VarPrefix: device

Results in variables like:

IGconf_device_class=pi5
IGconf_device_storage_type=sd

Anchors

Anchors are declared in a variable’s metadata and are definitive remapping points for paths and other environment values. They allow locations to be described in an abstract way (eg, @IGROOT, @SRCROOT) and are resolved by the engine automatically, never leaking to the shell. Anchors enable support of different contexts (eg, host vs. container, with different root paths) without rewriting layer or config files. The resolver replaces ${@ANCHOR} with the concrete value from the anchor registry when producing the final environment. From a simplistic viewpoint, an anchor is essentially a shortcut to a variable’s value.

The engine automatically sets a small number of default anchors. These are listed below:

Anchor Scope Description

@IGROOT

Always

Root path of rpi-image-gen tree

@SRCROOT

-S only

Root path of source directory

An Anchor is declared in layer metadata as follows:

# X-Env-Var-dir: ${IGconf_sys_workroot}/${IGconf_target_context}
# X-Env-Var-dir-Anchor: @TARGETDIR
# X-Env-Var-dir-Desc: Directory containing the target and related build artefacts.

An anchor can be used in a config file as follows:

layer:
  app: example-kiosk
  extra: rpi-splash-screen

splash:
  image_path: ${@SRCROOT}/my.tga

In this case, when building sources via -S, IGconf_splash_image_path resolves to /path/to/source/dir/my.tga. This can be observed in the output:

$ rpi-image-gen build -S ./examples/webkiosk/ -c kiosk.yaml
...
FINAL ENV
-> IGconf_splash_image_path : /path/to/repos/raspberrypi/rpi-image-gen/examples/webkiosk/my.tga

Configuration File Support

The configuration system uses YAML configuration files with native support for hierarchical data and includes.

Mapping Sections

include:
  file: base.yaml

env:
  MYVAR: UNCHANGED

device:
  class: pi5
  storage_type: sd
  hostname: ${IGconf_device_class}-$(tr -dc 'a-z' < /dev/urandom | head -c 6)

image:
  compression: zstd
  boot_part_size: 200%
  root_part_size: 300%

List Sections

In addition to mapping (key-value) sections, YAML configuration files support sequence (list) sections. List items are assigned 0-based numeric keys, producing the same IGconf_<section>_<key> variable format as mapping sections.

packages:
  - vim
  - git
  - curl

This produces the following variables:

IGconf_packages_0=vim
IGconf_packages_1=git
IGconf_packages_2=curl

This is equivalent to the explicit mapping form:

packages:
  0: vim
  1: git
  2: curl

List sections are useful when the key names are not meaningful and only ordering matters. They follow the same include merging and override rules as mapping sections.

Warning

List sections do not merge with includes in the same way as mapping sections. Because list items are internally keyed by their position index, an include merge between two files defining the same list section produces a positional override rather than an append or replacement. For predictable results, avoid defining the same list section in both a base file and a file that includes it.

Configuration Includes

Configuration files support includes to promote reusability and modularity.

A single file is included using a mapping with a file key:

include:
  file: base.yaml

Multiple files are included using a list of filenames. Files are processed in order - later entries override earlier ones. The including file overrides all included content.

include:
  - device/cm5.yaml
  - device/cm5io.yaml

Include files are resolved using the following search order:

Priority Source Description

1 (Highest)

Parent directory of the including file

2

<src dir>/config

The source directory if specified via the command line

3 (Lowest)

<built-in>/config

Default in-tree location, eg /usr/share/rpi-image-gen/config/

Warning

The list-section merge warning above applies equally to multiple includes. If two included files both define a list section (e.g. packages), they merge positionally - later entries override earlier ones by index rather than appending. For predictable results, define list sections in only one file in an include chain.

Section Names

Any section name used in a configuration file maps directly to the IGconf_<section>_* variable namespace. A layer can declare a section via X-Env-VarPrefix in its metadata. This means that the set of meaningful section names in a build is determined by the layers included. Declaring a section allows a layer to expose a group of configuration variables. Some section names carry specific functional meaning because the build system actively responds to them.

Note

trait: (see trait) is the one exception to the rule above - it is never translated to IGconf_* variables at all.

device

The device section (X-Env-VarPrefix: device) holds variables describing the target hardware. Variables in this section are declared by device-base and downstream device layers.

The device.layer key has functional meaning: it’s one way to specify the device layer to include in the build.

image

The image section (X-Env-VarPrefix: image) holds variables that control image construction. Variables in this section are declared by image-base and downstream image layers.

The image.layer key has functional meaning: it’s one way to specify the image layer to include in the build.

layer

The layer section is used to specify layers to include in the build. Each key names an extra layer:

layer:
  app: my-app-layer
  extra: rpi-splash-screen

The key names are arbitrary - only the values matter. Lists are also supported. This is the primary mechanism for composing a build from multiple optional layers. Device and image layers can be specified here, too:

layer:
  - rpi-cm5
  - image-rota

packages

The packages section triggers automatic package installation into the target rootfs during the build. Each entry is either an apt package name or a path to a local .deb file on the build host.

packages:
  - curl
  - vim
  - git

Local .deb files are specified by path and are detected by the presence of the file on the build host at the time the hook runs:

packages:
  apt: curl
  local1: /path/to/mypkg_1.0_arm64.deb
  local2: myotherpkg_2.0_arm64.deb

Both list and mapping forms are supported. Local .deb files are installed before apt packages so that apt can satisfy any dependencies they introduce.

Absolute paths are fully supported. Relative paths are resolved from the source directory and require -S to be specified, falling back to the rpi-image-gen root directory.

trait

Unlike every other section, trait: is never translated to IGconf_<section>_* variables. It’s a reserved top-level key, passed through as-is and applied directly as a build-time override to the trait registry - taking precedence over anything declared by the registry itself or by a layer’s X-Env-Layer-Provides.

trait:
  hw:bluetooth: false
  hw:acme:can: true

false removes the named token (and any of its children, by prefix match) from the build’s provider index, even if a layer in the build declares it. Any other value - true for a boolean trait, or an explicit typed value otherwise - injects the token, exactly as if a layer had declared it. Trait overrides merge across includes like any other section - the including file’s entries win over an included file’s on a shared token, and non-overlapping tokens from both are kept.

See trait.adoc for the full trait registry reference, including token namespaces, hierarchy, and has().

Variable Sources and Precedence

The configuration system supports multiple variable sources with a clear precedence hierarchy:

Priority Source Description

1 (Highest)

Command Line Overrides

Variables specified after -- on the command line

2

Configuration Files

YAML configuration files

3 (Lowest)

Layer Defaults

Default values declared in layer metadata

Command Line Overrides

Variables can be set directly on the command line as overrides after --, taking the highest precedence:

rpi-image-gen build -c config.yaml -- IGconf_device_class=pi5 IGconf_image_compression=xz

Command-line overrides are handled by the calling shell before the engine sees them.

Variable Expansion

The engine supports deferred variable expansion, meaning that it’s possible to forward-declare references to other variables. Variables are resolved in the pipeline phase rather than the config file parsing phase. Provided a variable is declared, the engine will resolve it. Values assigned to all variables must adhere to POSIX‑sh syntax.

Environment Variable Expansion

Variable values may include parameter expansion or command substitution. To be expanded by the engine, they must use one of these forms:

  • ${variable}

  • $(command)

Any other $ (eg, $VAR or $6$…​) is treated as literal and will be escaped.

device:
  name: controllerbox
  rev: 2
  hostname: ${IGconf_device_name}-$(uuidgen | cut -d'-' -f1)

image:
  assetdir: ${WORKSPACE}/build/assets/${IGconf_device_name}-rev${IGconf_device_rev}

ssh:
  pubkey_user1: ${DEPLOY}/device/baseassets/id_rsa.pub
  pubkey_only: y

Care must be taken if using the command line to assign a value to a variable if the value contains spaces or newlines. Making sure the calling shell expands it into a single argument ensures the command substitution happens once and stays on one line:

rpi-image-gen build -c config.yaml -- "IGconf_ssh_pubkey_user1=$(< /path/to/.ssh/id_rsa.pub)"

Assigning a variable and preserving the literal value can be performed using single quotes:

rpi-image-gen build -c config.yaml -- 'IGconf_ssh_pubkey_user1=$(cat /path/to/.ssh/id_rsa.pub)'

The above yield identical results in the build configuration for the value of IGconf_ssh_pubkey_user1. The difference is that the first expands the value immediately upon assignment using the calling shell (eg Bash) before engine processing. The second keeps the literal text, which is expanded during engine processing using POSIX-sh.

Expansion Context

Variables are resolved after configuration parsing and layer assembly, and then expanded. Expansion is performed with a strict policy using /bin/sh with POSIXLY_CORRECT=1 under set -eu in a 'clean room' environment. This means:

  • errexit (set -e): Exit immediately if any command returns a non-zero exit status, including when variable expansion fails or returns an error.

  • nounset (set -u): Treat unset variables as errors during expansion - attempting to expand a variable that hasn’t been set will exit with an error instead of expanding to an empty string.

  • When these are combined, variable expansion is strict and fails quickly. Any attempt to expand an undefined variable will cause the build to exit immediately.

Note

There may be a need to handle non-alphanumeric characters in config files and/or command line arguments. Input files are parsed as-is, without stripping quotes. In YAML, quote only when you need YAML to treat the value as a string (eg, URLs can be unquoted: url: https://…​;). On the command line, use shell quoting when needed to protect special characters or expansions (eg, rpi-image-gen build -c <args> — IGconf_device_user1pass='Fo0bar!!').

Cross-Reference Expansion

Variables can reference other configuration variables:

device:
  class: pi5
  hostname: ${IGconf_device_class}-server

image:
  name: ${IGconf_device_class}-custom-image

Layer Variable Dependencies

Layer dependencies can use config or command line variable assignments to create dynamic dependency resolution:

# METABEGIN
# X-Env-Layer-Name: arch-specific-tools
# X-Env-Layer-Requires: base-layer,${ARCH}-toolchain,${DISTRO}-packages
# METAEND

This allows layers to dynamically depend on other layers based on build-time set variables:

# Set variables on the command line
$ rpi-image-gen build -c <args> -- ARCH=arm64 DISTRO=debian
...
# Layer dependencies will be equivalent to:
# X-Env-Layer-Requires: base-layer, arm64-toolchain, debian-packages

This feature enables dynamic layer selection. For example, one usage could be to use the same layer across different build environments to support toolchain selection based on target architecture. By using variables in the 'parent' layer, dynamic dependency resolution pulls in an architecture specific toolchain 'child' layer.

Note

Layer discovery will fail if variables referenced in dependencies are not set.

Important

Variable dependencies are evaluated during layer discovery, not during layer variable expansion. This ensures dependencies are resolved before layer processing begins. Only variables set in the config file or on the command line can be used in layer dependencies.

All dependencies are always included, i.e. variable names result in actual layer names at build-time. For example, ${ARCH}-toolchain always results in a dependency, but the specific layer name depends on the ARCH environment variable value.

Configuration Loading

Processing Overview

The configuration system processes files using the following steps:

  1. Parse YAML configuration file

  2. Process includes recursively - Follows include directives (single or list) in order, with circular dependency detection

  3. Apply command line overrides - Processes variables specified after --

  4. Generate output - Either loads into environment or writes to file

Command Line Interface

Config Command

The config command provides direct access to configuration functionality:

# Load all sections into environment
rpi-image-gen config myconfig.yaml

# Load specific section
rpi-image-gen config myconfig.yaml --section device

# Write resolved config to file
rpi-image-gen config myconfig.yaml --write-to build.env

# Use command line overrides
rpi-image-gen config myconfig.yaml -- IGconf_device_class=pi5 IGconf_image_compression=xz

# Custom include search paths
rpi-image-gen config myconfig.yaml --path "./config:./shared:./templates"

# Disable variable expansion
rpi-image-gen config myconfig.yaml --no-expand

# Combine options with overrides
rpi-image-gen config myconfig.yaml --section device --write-to build.env -- IGconf_device_class=pi5

Config Generation

Generate an example YAML configuration:

rpi-image-gen config --gen

Integration with Layer System

Layer Variable Discovery

The configuration system integrates with the layer management system to:

  1. Discover declared variables from layer metadata

  2. Validate variable values against declared constraints

  3. Apply variable policies (immediate, lazy, force, skip)

  4. Resolve variable dependencies between layers

Variable Policies

Variables support different assignment policies:

  • immediate: Set the variable if it is currently unset (first-wins strategy). This is the default behaviour.

  • lazy: Applied after all layers are processed (last-wins strategy). Useful for defaults that can be overridden.

  • force: Always overwrite existing environment value, regardless of what was set before.

  • skip: Never set the variable.

Refer to the layer documentation for further details.

Layer Build Order

Variables are processed in layer dependency order, ensuring that:

  1. Base layer variables are available to dependent layers

  2. Variable expansion can reference previously set values

  3. Validation occurs with complete variable context

Best Practices

Configuration Organisation

  1. Use base configurations for common settings across builds

  2. Layer-specific overrides in focused configuration files

  3. Environment-specific values in separate configuration files

  4. Runtime overrides via command-line arguments after --

Variable Naming

  1. Follow prefix conventions with logical section grouping

  2. Use descriptive names that indicate purpose and scope

  3. Consistent naming patterns within configuration sections

File Structure

config/
├── base.yaml              # Common base configuration
├── environments/
│   ├── development.yaml   # Development overrides
│   ├── staging.yaml      # Staging overrides
│   └── production.yaml   # Production overrides
└── variants/
    ├── local.yaml       # Local development settings
    └── ci.yaml          # CI/CD specific settings

Variable Expansion Security

  1. Validate input in configuration files to prevent injection

  2. Limit variable scope to prevent unintended expansion

  3. Use quotes for values containing special characters

  4. Test expansion in controlled environments before production

Error Handling and Debugging

Common Configuration Issues

  • Circular includes: Detected automatically with clear error messages

  • Missing include files: Searches all configured paths before failing

  • Invalid YAML syntax: Provides line numbers and context

  • Undefined variable expansion: Fails fast with variable name

  • Section/key conflicts: Resolved by precedence rules

Debugging Configuration

  1. Use --write-to to see resolved variable values

  2. Check precedence with CFG/OVR output prefixes

  3. Validate includes by examining search path messages

  4. Run interactively to allow a pseudo 'dry-run' stepped flow

  5. Test expansion with simple configuration files

Output Interpretation

Configuration loading outputs show variable sources:

CFG IGconf_device_class=pi5         # From configuration file
OVR IGconf_device_hostname=pi5-test  # From command line override

Layer collection output shows variable policy and evaluated value:

 [SET]  IGconf_device_class=pi5 (layer: rpi5)
 [SKIP]  IGconf_device_hostname (already set)
 [LAZY]  IGconf_device_variant=none (layer: device-base)

Performance Considerations

Configuration Loading Performance

  • Lazy evaluation of variable expansion where possible

  • Caching of included files to avoid repeated parsing

  • Minimal environment manipulation during loading

  • Efficient precedence resolution without redundant checks

Summary

The rpi-image-gen configuration system provides a robust foundation for managing complex build configurations through:

  • Hierarchical variable organisation with the IGconf_ namespace

  • Multiple configuration sources with clear precedence rules

  • YAML configuration files with include support

  • Variable expansion supporting cross-references and environment integration

  • Layer system integration for variable discovery and validation

  • Command-line tools for configuration management and debugging