Table of Contents

Overview

rpi-image-gen provides a flexible, hierarchical approach to managing build variables through multiple configuration sources with clear precedence rules. The system is built around the IGconf_ prefix namespace and supports both INI and YAML file formats.

Core Architecture

Variable Naming Convention

Excluding environment keys, configuration variables in rpi-image-gen use the IGconf_ prefix followed by a structured naming pattern:

IGconf_<section>_<key>

Where:

  • section: Logical grouping (e.g., device, image)

  • key: Specific configuration parameter (e.g., class, storage_type, hostname)

Example variables:

IGconf_device_class=pi5
IGconf_device_storage_type=sd
IGconf_image_compression=zstd

Layer Variable Declaration

Variables are declared by layer YAML files using metadata headers within comment blocks:

# METABEGIN
# X-Env-Layer-Name: rpi5
# X-Env-Layer-Category: device
# X-Env-Layer-Desc: Raspberry Pi 5 specific device layer
#
# X-Env-VarPrefix: device
#
# X-Env-Var-class: pi5
# X-Env-Var-class-Desc: Device class
# X-Env-Var-class-Required: n
# X-Env-Var-class-Valid: keywords:pi5
# X-Env-Var-class-Set: y
#
# X-Env-Var-storage_type: sd
# X-Env-Var-storage_type-Desc: Pi5 storage type
# X-Env-Var-storage_type-Required: n
# X-Env-Var-storage_type-Valid: sd,nvme
# X-Env-Var-storage_type-Set: y
# METAEND

Variable Declaration Metadata

Each variable declaration includes several metadata fields:

  • X-Env-Var-<name>: Default value for the variable

  • X-Env-Var-<name>-Desc: Human-readable description

  • X-Env-Var-<name>-Required: Whether the variable is required (y/n)

  • X-Env-Var-<name>-Valid: Validation rules (keywords, patterns, etc.)

  • X-Env-Var-<name>-Set: Variable assignment policy

Variable Prefix

The X-Env-VarPrefix field specifies the section name used in the IGconf_ namespace:

X-Env-VarPrefix: device

Results in variables like:

IGconf_device_class=pi5
IGconf_device_storage_type=sd

Anchors

Anchors are declared in a variable’s metadata and are definitive remapping points for paths and other environment values. They allow locations to be described in an abstract way (eg, @IGROOT, @SRCROOT) and are resolved by the engine automatically, never leaking to the shell. Anchors enable support of different contexts (eg, host vs. container, with different root paths) without rewriting layer or config files. The resolver replaces ${@ANCHOR} with the concrete value from the anchor registry when producing the final environment. From a simplistic viewpoint, an anchor is essentially a shortcut to a variable’s value.

The engine automatically sets a small number of default anchors. These are listed below:

Anchor Scope Description

@IGROOT

Always

Root path of rpi-image-gen tree

@SRCROOT

-S only

Root path of source directory

An Anchor is declared in layer metadata as follows:

# X-Env-Var-dir: ${IGconf_sys_workroot}/${IGconf_target_context}
# X-Env-Var-dir-Anchor: @TARGETDIR
# X-Env-Var-dir-Desc: Directory containing the target and related build artefacts.

An anchor can be used in a config file as follows:

layer:
  app: example-kiosk
  extra: rpi-splash-screen

splash:
  image_path: ${@SRCROOT}/my.tga

In this case, when building sources via -S, IGconf_splash_image_path resolves to /path/to/source/dir/my.tga. This can be observed in the output:

$ rpi-image-gen build -S ./examples/webkiosk/ -c kiosk.yaml
...
FINAL ENV
-> IGconf_splash_image_path : /path/to/repos/raspberrypi/rpi-image-gen/examples/webkiosk/my.tga

Configuration File Support

The configuration system supports both INI and YAML file formats with automatic format detection based on file extension.

YAML Configuration Format

YAML provides the most flexible configuration format with native support for hierarchical data and includes.

Mapping Sections

include:
  file: base.yaml

env:
  MYVAR: UNCHANGED

device:
  class: pi5
  storage_type: sd
  hostname: ${IGconf_device_class}-$(tr -dc 'a-z' < /dev/urandom | head -c 6)

image:
  compression: zstd
  boot_part_size: 200%
  root_part_size: 300%

List Sections

In addition to mapping (key-value) sections, YAML configuration files support sequence (list) sections. List items are assigned 0-based numeric keys, producing the same IGconf_<section>_<key> variable format as mapping sections.

packages:
  - vim
  - git
  - curl

This produces the following variables:

IGconf_packages_0=vim
IGconf_packages_1=git
IGconf_packages_2=curl

This is equivalent to the explicit mapping form:

packages:
  0: vim
  1: git
  2: curl

List sections are useful when the key names are not meaningful and only ordering matters. They follow the same include merging and override rules as mapping sections.

Warning

List sections do not merge with includes in the same way as mapping sections. Because list items are internally keyed by their position index, an include merge between two files defining the same list section produces a positional override rather than an append or replacement. For predictable results, avoid defining the same list section in both a base file and a file that includes it.

INI Configuration Format

INI format provides traditional section-based configuration:

!include base.cfg

[env]
MYVAR = UNCHANGED

[device]
class = pi5
storage_type = sd
hostname = ${IGconf_device_class}-$(tr -dc 'a-z' < /dev/urandom | head -c 6)

[image]
compression = zstd
boot_part_size = 200%
root_part_size = 300%

Configuration Includes

Both formats support including other configuration files to promote reusability and modularity. The include precedence order is shown in the table below.

Priority Source Description

1 (Highest)

Parent directory of the including file

2

<src dir>/config

The source directory if specified via the command line

3 (Lowest)

<built-in>/config

Default in-tree location, eg /usr/share/rpi-image-gen/config/

YAML Includes

include:
  file: shared/base.yaml

INI Includes

!include shared/base.cfg

Therefore, included files are resolved using:

  1. Parent directory

  2. Search path

Section Names

Any section name used in a configuration file maps directly to the IGconf_<section>_* variable namespace. A layer can declare a section via X-Env-VarPrefix in its metadata. This means that the set of meaningful section names in a build is determined by the layers included. Declaring a section allows a layer to expose a group of configuration variables. Some section names carry specific functional meaning because the build system actively responds to them.

device

The device section (X-Env-VarPrefix: device) holds variables describing the target hardware. Variables in this section are declared by device-base and downstream device layers.

The device.layer key has functional meaning: it’s one way to specify the device layer to include in the build.

image

The image section (X-Env-VarPrefix: image) holds variables that control image construction. Variables in this section are declared by image-base and downstream image layers.

The image.layer key has functional meaning: it’s one way to specify the image layer to include in the build.

layer

The layer section is used to specify layers to include in the build. Each key names an extra layer:

layer:
  app: my-app-layer
  extra: rpi-splash-screen

The key names are arbitrary - only the values matter. Lists are also supported. This is the primary mechanism for composing a build from multiple optional layers. Device and image layers can be specified here, too:

layer:
  - rpi-cm5
  - image-rota

packages

The packages section triggers automatic package installation into the target rootfs during the build. Each entry is either an apt package name or a path to a local .deb file on the build host.

packages:
  - curl
  - vim
  - git

Local .deb files are specified by path and are detected by the presence of the file on the build host at the time the hook runs:

packages:
  apt: curl
  local1: /path/to/mypkg_1.0_arm64.deb
  local2: myotherpkg_2.0_arm64.deb

Both list and mapping forms are supported. Local .deb files are installed before apt packages so that apt can satisfy any dependencies they introduce.

Absolute paths are fully supported. Relative paths are resolved from the source directory and require -S to be specified, falling back to the rpi-image-gen root directory.

Variable Sources and Precedence

The configuration system supports multiple variable sources with a clear precedence hierarchy:

Priority Source Description

1 (Highest)

Command Line Overrides

Variables specified after -- on the command line

2

Configuration Files

INI or YAML configuration files

3 (Lowest)

Layer Defaults

Default values declared in layer metadata

Command Line Overrides

Variables can be set directly on the command line as overrides after --, taking the highest precedence:

rpi-image-gen build -c config.yaml -- IGconf_device_class=pi5 IGconf_image_compression=xz

Command-line overrides are handled by the calling shell before the engine sees them.

Variable Expansion

The engine supports deferred variable expansion, meaning that it’s possible to forward-declare references to other variables. Variables are resolved in the pipeline phase rather than the config file parsing phase. Provided a variable is declared, the engine will resolve it. Values assigned to all variables must adhere to POSIX‑sh syntax.

Environment Variable Expansion

Variable values may include parameter expansion or command substitution. To be expanded by the engine, they must use one of these forms:

  • ${variable}

  • $(command)

Any other $ (eg, $VAR or $6$…​) is treated as literal and will be escaped.

device:
  name: controllerbox
  rev: 2
  hostname: ${IGconf_device_name}-$(uuidgen | cut -d'-' -f1)

image:
  assetdir: ${WORKSPACE}/build/assets/${IGconf_device_name}-rev${IGconf_device_rev}

ssh:
  pubkey_user1: ${DEPLOY}/device/baseassets/id_rsa.pub
  pubkey_only: y

Care must be taken if using the command line to assign a value to a variable if the value contains spaces or newlines. Making sure the calling shell expands it into a single argument ensures the command substitution happens once and stays on one line:

rpi-image-gen build -c config.yaml -- "IGconf_ssh_pubkey_user1=$(< /path/to/.ssh/id_rsa.pub)"

Assigning a variable and preserving the literal value can be performed using single quotes:

rpi-image-gen build -c config.yaml -- 'IGconf_ssh_pubkey_user1=$(cat /path/to/.ssh/id_rsa.pub)'

The above yield identical results in the build configuration for the value of IGconf_ssh_pubkey_user1. The difference is that the first expands the value immediately upon assignment using the calling shell (eg Bash) before engine processing. The second keeps the literal text, which is expanded during engine processing using POSIX-sh.

Expansion Context

Variables are resolved after configuration parsing and layer assembly, and then expanded. Expansion is performed with a strict policy using /bin/sh with POSIXLY_CORRECT=1 under set -eu in a 'clean room' environment. This means:

  • errexit (set -e): Exit immediately if any command returns a non-zero exit status, including when variable expansion fails or returns an error.

  • nounset (set -u): Treat unset variables as errors during expansion - attempting to expand a variable that hasn’t been set will exit with an error instead of expanding to an empty string.

  • When these are combined, variable expansion is strict and fails quickly. Any attempt to expand an undefined variable will cause the build to exit immediately.

Note

There may be a need to handle non-alphanumeric characters in config files and/or command line arguments. Input files are parsed as-is, without stripping quotes. In YAML, quote only when you need YAML to treat the value as a string (eg, URLs can be unquoted: url: https://…​;). On the command line, use shell quoting when needed to protect special characters or expansions (eg, rpi-image-gen build -c <args> — IGconf_device_user1pass='Fo0bar!!').

Cross-Reference Expansion

Variables can reference other configuration variables:

device:
  class: pi5
  hostname: ${IGconf_device_class}-server

image:
  name: ${IGconf_device_class}-custom-image

Layer Variable Dependencies

Layer dependencies can use config or command line variable assignments to create dynamic dependency resolution:

# METABEGIN
# X-Env-Layer-Name: arch-specific-tools
# X-Env-Layer-Requires: base-layer,${ARCH}-toolchain,${DISTRO}-packages
# METAEND

This allows layers to dynamically depend on other layers based on build-time set variables:

# Set variables on the command line
$ rpi-image-gen build -c <args> -- ARCH=arm64 DISTRO=debian
...
# Layer dependencies will be equivalent to:
# X-Env-Layer-Requires: base-layer, arm64-toolchain, debian-packages

This feature enables dynamic layer selection. For example, one usage could be to use the same layer across different build environments to support toolchain selection based on target architecture. By using variables in the 'parent' layer, dynamic dependency resolution pulls in an architecture specific toolchain 'child' layer.

Note

Layer discovery will fail if variables referenced in dependencies are not set.

Important

Variable dependencies are evaluated during layer discovery, not during layer variable expansion. This ensures dependencies are resolved before layer processing begins. Only variables set in the config file or on the command line can be used in layer dependencies.

All dependencies are always included, i.e. variable names result in actual layer names at build-time. For example, ${ARCH}-toolchain always results in a dependency, but the specific layer name depends on the ARCH environment variable value.

Configuration Loading

Processing Overview

The configuration system processes files using the following steps:

  1. Parse main configuration file - Automatically detects INI or YAML format

  2. Process includes recursively - Follows include directives with circular dependency detection

  3. Apply command line overrides - Processes variables specified after --

  4. Generate output - Either loads into environment or writes to file

Command Line Interface

Config Command

The config command provides direct access to configuration functionality:

# Load all sections into environment
rpi-image-gen config myconfig.yaml

# Load specific section
rpi-image-gen config myconfig.yaml --section device

# Write resolved config to file
rpi-image-gen config myconfig.yaml --write-to build.env

# Use command line overrides
rpi-image-gen config myconfig.yaml -- IGconf_device_class=pi5 IGconf_image_compression=xz

# Custom include search paths
rpi-image-gen config myconfig.yaml --path "./config:./shared:./templates"

# Disable variable expansion
rpi-image-gen config myconfig.yaml --no-expand

# Combine options with overrides
rpi-image-gen config myconfig.yaml --section device --write-to build.env -- IGconf_device_class=pi5

Config Generation

Generate example configuration files:

# Generate boilerplate INI and YAML examples
rpi-image-gen config --gen

# Migrate INI to YAML format
rpi-image-gen config legacy.cfg --migrate > modern.yaml

Integration with Layer System

Layer Variable Discovery

The configuration system integrates with the layer management system to:

  1. Discover declared variables from layer metadata

  2. Validate variable values against declared constraints

  3. Apply variable policies (immediate, lazy, force, skip)

  4. Resolve variable dependencies between layers

Variable Policies

Variables support different assignment policies:

  • immediate: Set the variable if it is currently unset (first-wins strategy). This is the default behavior.

  • lazy: Applied after all layers are processed (last-wins strategy). Useful for defaults that can be overridden.

  • force: Always overwrite existing environment value, regardless of what was set before.

  • skip: Never set the variable.

Refer to the layer documentation for further details.

Layer Build Order

Variables are processed in layer dependency order, ensuring that:

  1. Base layer variables are available to dependent layers

  2. Variable expansion can reference previously set values

  3. Validation occurs with complete variable context

Best Practices

Configuration Organisation

  1. Use base configurations for common settings across builds

  2. Layer-specific overrides in focused configuration files

  3. Environment-specific values in separate configuration files

  4. Runtime overrides via command-line arguments after --

Variable Naming

  1. Follow prefix conventions with logical section grouping

  2. Use descriptive names that indicate purpose and scope

  3. Consistent naming patterns within configuration sections

File Structure

config/
├── base.yaml              # Common base configuration
├── environments/
│   ├── development.yaml   # Development overrides
│   ├── staging.yaml      # Staging overrides
│   └── production.yaml   # Production overrides
└── variants/
    ├── local.yaml       # Local development settings
    └── ci.yaml          # CI/CD specific settings

Variable Expansion Security

  1. Validate input in configuration files to prevent injection

  2. Limit variable scope to prevent unintended expansion

  3. Use quotes for values containing special characters

  4. Test expansion in controlled environments before production

Error Handling and Debugging

Common Configuration Issues

  • Circular includes: Detected automatically with clear error messages

  • Missing include files: Searches all configured paths before failing

  • Invalid YAML/INI syntax: Provides line numbers and context

  • Undefined variable expansion: Fails fast with variable name

  • Section/key conflicts: Resolved by precedence rules

Debugging Configuration

  1. Use --write-to to see resolved variable values

  2. Check precedence with CFG/OVR output prefixes

  3. Validate includes by examining search path messages

  4. Run interactively to allow a pseudo 'dry-run' stepped flow

  5. Test expansion with simple configuration files

Output Interpretation

Configuration loading outputs show variable sources:

CFG IGconf_device_class=pi5         # From configuration file
OVR IGconf_device_hostname=pi5-test  # From command line override

Layer collection output shows variable policy and evaluated value:

 [SET]  IGconf_device_class=pi5 (layer: rpi5)
 [SKIP]  IGconf_device_hostname (already set)
 [LAZY]  IGconf_device_variant=none (layer: device-base)

Migration and Compatibility

INI to YAML Migration

Use the built-in migration tool:

rpi-image-gen config legacy.cfg --migrate > modern.yaml

The migration tool:

  • Preserves include directives with updated syntax

  • Maintains section structure and variable relationships

  • Updates file extensions in include references

  • Adds migration comments for traceability

Backward Compatibility

The configuration system maintains compatibility with:

  • Existing INI files through automatic format detection

  • Legacy include syntax (!include vs include:)

  • Environment variable patterns used by previous versions

  • Command-line interfaces with extended functionality

Performance Considerations

Configuration Loading Performance

  • Lazy evaluation of variable expansion where possible

  • Caching of included files to avoid repeated parsing

  • Minimal environment manipulation during loading

  • Efficient precedence resolution without redundant checks

Summary

The rpi-image-gen configuration system provides a robust foundation for managing complex build configurations through:

  • Hierarchical variable organisation with the IGconf_ namespace

  • Multiple configuration sources with clear precedence rules

  • Flexible file formats (INI and YAML) with include support

  • Variable expansion supporting cross-references and environment integration

  • Layer system integration for variable discovery and validation

  • Command-line tools for configuration management and debugging