- Overview
- Core Architecture
- Configuration File Support
- Section Names
- Variable Sources and Precedence
- Variable Expansion
- Configuration Loading
- Command Line Interface
- Integration with Layer System
- Best Practices
- Error Handling and Debugging
- Migration and Compatibility
- Performance Considerations
- Summary
Overview
rpi-image-gen provides a flexible, hierarchical approach to managing build variables through multiple configuration sources with clear precedence rules. The system is built around the IGconf_ prefix namespace and supports both INI and YAML file formats.
Core Architecture
Variable Naming Convention
Excluding environment keys, configuration variables in rpi-image-gen use the IGconf_ prefix followed by a structured naming pattern:
IGconf_<section>_<key>
Where:
-
section: Logical grouping (e.g.,
device,image) -
key: Specific configuration parameter (e.g.,
class,storage_type,hostname)
Example variables:
IGconf_device_class=pi5
IGconf_device_storage_type=sd
IGconf_image_compression=zstd
Layer Variable Declaration
Variables are declared by layer YAML files using metadata headers within comment blocks:
# METABEGIN
# X-Env-Layer-Name: rpi5
# X-Env-Layer-Category: device
# X-Env-Layer-Desc: Raspberry Pi 5 specific device layer
#
# X-Env-VarPrefix: device
#
# X-Env-Var-class: pi5
# X-Env-Var-class-Desc: Device class
# X-Env-Var-class-Required: n
# X-Env-Var-class-Valid: keywords:pi5
# X-Env-Var-class-Set: y
#
# X-Env-Var-storage_type: sd
# X-Env-Var-storage_type-Desc: Pi5 storage type
# X-Env-Var-storage_type-Required: n
# X-Env-Var-storage_type-Valid: sd,nvme
# X-Env-Var-storage_type-Set: y
# METAEND
Variable Declaration Metadata
Each variable declaration includes several metadata fields:
-
X-Env-Var-<name>: Default value for the variable
-
X-Env-Var-<name>-Desc: Human-readable description
-
X-Env-Var-<name>-Required: Whether the variable is required (
y/n) -
X-Env-Var-<name>-Valid: Validation rules (keywords, patterns, etc.)
-
X-Env-Var-<name>-Set: Variable assignment policy
Variable Prefix
The X-Env-VarPrefix field specifies the section name used in the IGconf_ namespace:
X-Env-VarPrefix: device
Results in variables like:
IGconf_device_class=pi5
IGconf_device_storage_type=sd
Anchors
Anchors are declared in a variable’s metadata and are definitive remapping points for paths and other environment values. They allow locations to be described in an abstract way (eg, @IGROOT, @SRCROOT) and are resolved by the engine automatically, never leaking to the shell. Anchors enable support of different contexts (eg, host vs. container, with different root paths) without rewriting layer or config files. The resolver replaces ${@ANCHOR} with the concrete value from the anchor registry when producing the final environment. From a simplistic viewpoint, an anchor is essentially a shortcut to a variable’s value.
The engine automatically sets a small number of default anchors. These are listed below:
| Anchor | Scope | Description |
|---|---|---|
@IGROOT |
Always |
Root path of rpi-image-gen tree |
@SRCROOT |
-S only |
Root path of source directory |
An Anchor is declared in layer metadata as follows:
# X-Env-Var-dir: ${IGconf_sys_workroot}/${IGconf_target_context}
# X-Env-Var-dir-Anchor: @TARGETDIR
# X-Env-Var-dir-Desc: Directory containing the target and related build artefacts.
An anchor can be used in a config file as follows:
layer:
app: example-kiosk
extra: rpi-splash-screen
splash:
image_path: ${@SRCROOT}/my.tga
In this case, when building sources via -S, IGconf_splash_image_path resolves to /path/to/source/dir/my.tga. This can be observed in the output:
$ rpi-image-gen build -S ./examples/webkiosk/ -c kiosk.yaml
...
FINAL ENV
-> IGconf_splash_image_path : /path/to/repos/raspberrypi/rpi-image-gen/examples/webkiosk/my.tga
Configuration File Support
The configuration system supports both INI and YAML file formats with automatic format detection based on file extension.
YAML Configuration Format
YAML provides the most flexible configuration format with native support for hierarchical data and includes.
Mapping Sections
include:
file: base.yaml
env:
MYVAR: UNCHANGED
device:
class: pi5
storage_type: sd
hostname: ${IGconf_device_class}-$(tr -dc 'a-z' < /dev/urandom | head -c 6)
image:
compression: zstd
boot_part_size: 200%
root_part_size: 300%
List Sections
In addition to mapping (key-value) sections, YAML configuration files support sequence (list) sections. List items are assigned 0-based numeric keys, producing the same IGconf_<section>_<key> variable format as mapping sections.
packages:
- vim
- git
- curl
This produces the following variables:
IGconf_packages_0=vim
IGconf_packages_1=git
IGconf_packages_2=curl
This is equivalent to the explicit mapping form:
packages:
0: vim
1: git
2: curl
List sections are useful when the key names are not meaningful and only ordering matters. They follow the same include merging and override rules as mapping sections.
|
Warning
|
List sections do not merge with includes in the same way as mapping sections. Because list items are internally keyed by their position index, an include merge between two files defining the same list section produces a positional override rather than an append or replacement. For predictable results, avoid defining the same list section in both a base file and a file that includes it. |
INI Configuration Format
INI format provides traditional section-based configuration:
!include base.cfg
[env]
MYVAR = UNCHANGED
[device]
class = pi5
storage_type = sd
hostname = ${IGconf_device_class}-$(tr -dc 'a-z' < /dev/urandom | head -c 6)
[image]
compression = zstd
boot_part_size = 200%
root_part_size = 300%
Configuration Includes
Both formats support including other configuration files to promote reusability and modularity. The include precedence order is shown in the table below.
| Priority | Source | Description |
|---|---|---|
1 (Highest) |
Parent directory of the including file |
|
2 |
<src dir>/config |
The source directory if specified via the command line |
3 (Lowest) |
<built-in>/config |
Default in-tree location, eg |
YAML Includes
include:
file: shared/base.yaml
INI Includes
!include shared/base.cfg
Therefore, included files are resolved using:
-
Parent directory
-
Search path
Section Names
Any section name used in a configuration file maps directly to the IGconf_<section>_* variable namespace. A layer can declare a section via X-Env-VarPrefix in its metadata. This means that the set of meaningful section names in a build is determined by the layers included. Declaring a section allows a layer to expose a group of configuration variables. Some section names carry specific functional meaning because the build system actively responds to them.
device
The device section (X-Env-VarPrefix: device) holds variables describing the target hardware. Variables in this section are declared by device-base and downstream device layers.
The device.layer key has functional meaning: it’s one way to specify the device layer to include in the build.
image
The image section (X-Env-VarPrefix: image) holds variables that control image construction. Variables in this section are declared by image-base and downstream image layers.
The image.layer key has functional meaning: it’s one way to specify the image layer to include in the build.
layer
The layer section is used to specify layers to include in the build. Each key names an extra layer:
layer:
app: my-app-layer
extra: rpi-splash-screen
The key names are arbitrary - only the values matter. Lists are also supported. This is the primary mechanism for composing a build from multiple optional layers. Device and image layers can be specified here, too:
layer:
- rpi-cm5
- image-rota
packages
The packages section triggers automatic package installation into the target rootfs during the build. Each entry is either an apt package name or a path to a local .deb file on the build host.
packages:
- curl
- vim
- git
Local .deb files are specified by path and are detected by the presence of the file on the build host at the time the hook runs:
packages:
apt: curl
local1: /path/to/mypkg_1.0_arm64.deb
local2: myotherpkg_2.0_arm64.deb
Both list and mapping forms are supported. Local .deb files are installed before apt packages so that apt can satisfy any dependencies they introduce.
Absolute paths are fully supported. Relative paths are resolved from the source directory and require -S to be specified, falling back to the rpi-image-gen root directory.
Variable Sources and Precedence
The configuration system supports multiple variable sources with a clear precedence hierarchy:
| Priority | Source | Description |
|---|---|---|
1 (Highest) |
Command Line Overrides |
Variables specified after |
2 |
Configuration Files |
INI or YAML configuration files |
3 (Lowest) |
Layer Defaults |
Default values declared in layer metadata |
Command Line Overrides
Variables can be set directly on the command line as overrides after --, taking the highest precedence:
rpi-image-gen build -c config.yaml -- IGconf_device_class=pi5 IGconf_image_compression=xz
Command-line overrides are handled by the calling shell before the engine sees them.
Variable Expansion
The engine supports deferred variable expansion, meaning that it’s possible to forward-declare references to other variables. Variables are resolved in the pipeline phase rather than the config file parsing phase. Provided a variable is declared, the engine will resolve it. Values assigned to all variables must adhere to POSIX‑sh syntax.
Environment Variable Expansion
Variable values may include parameter expansion or command substitution. To be expanded by the engine, they must use one of these forms:
-
${variable} -
$(command)
Any other $ (eg, $VAR or $6$…) is treated as literal and will be escaped.
device:
name: controllerbox
rev: 2
hostname: ${IGconf_device_name}-$(uuidgen | cut -d'-' -f1)
image:
assetdir: ${WORKSPACE}/build/assets/${IGconf_device_name}-rev${IGconf_device_rev}
ssh:
pubkey_user1: ${DEPLOY}/device/baseassets/id_rsa.pub
pubkey_only: y
Care must be taken if using the command line to assign a value to a variable if the value contains spaces or newlines. Making sure the calling shell expands it into a single argument ensures the command substitution happens once and stays on one line:
rpi-image-gen build -c config.yaml -- "IGconf_ssh_pubkey_user1=$(< /path/to/.ssh/id_rsa.pub)"
Assigning a variable and preserving the literal value can be performed using single quotes:
rpi-image-gen build -c config.yaml -- 'IGconf_ssh_pubkey_user1=$(cat /path/to/.ssh/id_rsa.pub)'
The above yield identical results in the build configuration for the value of IGconf_ssh_pubkey_user1. The difference is that the first expands the value immediately upon assignment using the calling shell (eg Bash) before engine processing. The second keeps the literal text, which is expanded during engine processing using POSIX-sh.
Expansion Context
Variables are resolved after configuration parsing and layer assembly, and then expanded. Expansion is performed with a strict policy using /bin/sh with POSIXLY_CORRECT=1 under set -eu in a 'clean room' environment. This means:
-
errexit (set -e): Exit immediately if any command returns a non-zero exit status, including when variable expansion fails or returns an error.
-
nounset (set -u): Treat unset variables as errors during expansion - attempting to expand a variable that hasn’t been set will exit with an error instead of expanding to an empty string.
-
When these are combined, variable expansion is strict and fails quickly. Any attempt to expand an undefined variable will cause the build to exit immediately.
|
Note
|
There may be a need to handle non-alphanumeric characters in config files and/or command line arguments. Input files are parsed as-is, without stripping quotes. In YAML, quote only when you need YAML to treat the value as a string (eg, URLs can be unquoted: |
Cross-Reference Expansion
Variables can reference other configuration variables:
device:
class: pi5
hostname: ${IGconf_device_class}-server
image:
name: ${IGconf_device_class}-custom-image
Layer Variable Dependencies
Layer dependencies can use config or command line variable assignments to create dynamic dependency resolution:
# METABEGIN
# X-Env-Layer-Name: arch-specific-tools
# X-Env-Layer-Requires: base-layer,${ARCH}-toolchain,${DISTRO}-packages
# METAEND
This allows layers to dynamically depend on other layers based on build-time set variables:
# Set variables on the command line
$ rpi-image-gen build -c <args> -- ARCH=arm64 DISTRO=debian
...
# Layer dependencies will be equivalent to:
# X-Env-Layer-Requires: base-layer, arm64-toolchain, debian-packages
This feature enables dynamic layer selection. For example, one usage could be to use the same layer across different build environments to support toolchain selection based on target architecture. By using variables in the 'parent' layer, dynamic dependency resolution pulls in an architecture specific toolchain 'child' layer.
|
Note
|
Layer discovery will fail if variables referenced in dependencies are not set. |
|
Important
|
Variable dependencies are evaluated during layer discovery, not during layer variable expansion. This ensures dependencies are resolved before layer processing begins. Only variables set in the config file or on the command line can be used in layer dependencies. All dependencies are always included, i.e. variable names result in actual layer names at build-time. For example, |
Configuration Loading
Processing Overview
The configuration system processes files using the following steps:
-
Parse main configuration file - Automatically detects INI or YAML format
-
Process includes recursively - Follows include directives with circular dependency detection
-
Apply command line overrides - Processes variables specified after
-- -
Generate output - Either loads into environment or writes to file
Command Line Interface
Config Command
The config command provides direct access to configuration functionality:
# Load all sections into environment
rpi-image-gen config myconfig.yaml
# Load specific section
rpi-image-gen config myconfig.yaml --section device
# Write resolved config to file
rpi-image-gen config myconfig.yaml --write-to build.env
# Use command line overrides
rpi-image-gen config myconfig.yaml -- IGconf_device_class=pi5 IGconf_image_compression=xz
# Custom include search paths
rpi-image-gen config myconfig.yaml --path "./config:./shared:./templates"
# Disable variable expansion
rpi-image-gen config myconfig.yaml --no-expand
# Combine options with overrides
rpi-image-gen config myconfig.yaml --section device --write-to build.env -- IGconf_device_class=pi5
Config Generation
Generate example configuration files:
# Generate boilerplate INI and YAML examples
rpi-image-gen config --gen
# Migrate INI to YAML format
rpi-image-gen config legacy.cfg --migrate > modern.yaml
Integration with Layer System
Layer Variable Discovery
The configuration system integrates with the layer management system to:
-
Discover declared variables from layer metadata
-
Validate variable values against declared constraints
-
Apply variable policies (immediate, lazy, force, skip)
-
Resolve variable dependencies between layers
Variable Policies
Variables support different assignment policies:
-
immediate: Set the variable if it is currently unset (first-wins strategy). This is the default behavior.
-
lazy: Applied after all layers are processed (last-wins strategy). Useful for defaults that can be overridden.
-
force: Always overwrite existing environment value, regardless of what was set before.
-
skip: Never set the variable.
Refer to the layer documentation for further details.
Layer Build Order
Variables are processed in layer dependency order, ensuring that:
-
Base layer variables are available to dependent layers
-
Variable expansion can reference previously set values
-
Validation occurs with complete variable context
Best Practices
Configuration Organisation
-
Use base configurations for common settings across builds
-
Layer-specific overrides in focused configuration files
-
Environment-specific values in separate configuration files
-
Runtime overrides via command-line arguments after
--
Variable Naming
-
Follow prefix conventions with logical section grouping
-
Use descriptive names that indicate purpose and scope
-
Consistent naming patterns within configuration sections
File Structure
config/
├── base.yaml # Common base configuration
├── environments/
│ ├── development.yaml # Development overrides
│ ├── staging.yaml # Staging overrides
│ └── production.yaml # Production overrides
└── variants/
├── local.yaml # Local development settings
└── ci.yaml # CI/CD specific settings
Variable Expansion Security
-
Validate input in configuration files to prevent injection
-
Limit variable scope to prevent unintended expansion
-
Use quotes for values containing special characters
-
Test expansion in controlled environments before production
Error Handling and Debugging
Common Configuration Issues
-
Circular includes: Detected automatically with clear error messages
-
Missing include files: Searches all configured paths before failing
-
Invalid YAML/INI syntax: Provides line numbers and context
-
Undefined variable expansion: Fails fast with variable name
-
Section/key conflicts: Resolved by precedence rules
Debugging Configuration
-
Use
--write-toto see resolved variable values -
Check precedence with CFG/OVR output prefixes
-
Validate includes by examining search path messages
-
Run interactively to allow a pseudo 'dry-run' stepped flow
-
Test expansion with simple configuration files
Output Interpretation
Configuration loading outputs show variable sources:
CFG IGconf_device_class=pi5 # From configuration file
OVR IGconf_device_hostname=pi5-test # From command line override
Layer collection output shows variable policy and evaluated value:
[SET] IGconf_device_class=pi5 (layer: rpi5)
[SKIP] IGconf_device_hostname (already set)
[LAZY] IGconf_device_variant=none (layer: device-base)
Migration and Compatibility
INI to YAML Migration
Use the built-in migration tool:
rpi-image-gen config legacy.cfg --migrate > modern.yaml
The migration tool:
-
Preserves include directives with updated syntax
-
Maintains section structure and variable relationships
-
Updates file extensions in include references
-
Adds migration comments for traceability
Backward Compatibility
The configuration system maintains compatibility with:
-
Existing INI files through automatic format detection
-
Legacy include syntax (
!includevsinclude:) -
Environment variable patterns used by previous versions
-
Command-line interfaces with extended functionality
Performance Considerations
Configuration Loading Performance
-
Lazy evaluation of variable expansion where possible
-
Caching of included files to avoid repeated parsing
-
Minimal environment manipulation during loading
-
Efficient precedence resolution without redundant checks
Summary
The rpi-image-gen configuration system provides a robust foundation for managing complex build configurations through:
-
Hierarchical variable organisation with the
IGconf_namespace -
Multiple configuration sources with clear precedence rules
-
Flexible file formats (INI and YAML) with include support
-
Variable expansion supporting cross-references and environment integration
-
Layer system integration for variable discovery and validation
-
Command-line tools for configuration management and debugging