Test packages in Buildroot
When writing packages for Buildroot there are several conditions that you have to test your package against.
This includes different toolchains, architectures, C-libraries, thread-implementations and more. To help you with that, Buildroot provides the utils/test-pkg script.
Nothing describes the script better than its own help text [1]:
test-pkg: test-build a package against various toolchains and architectures The supplied config snippet is appended to each toolchain config, the resulting configuration is checked to ensure it still contains all options specified in the snippet; if any is missing, the build is skipped, on the assumption that the package under test requires a toolchain or architecture feature that is missing. In case failures are noticed, you can fix the package and just re-run the same command again; it will re-run the test where it failed. If you did specify a package (with -p), the package build dir will be removed first. The list of toolchains is retrieved from support/config-fragments/autobuild/toolchain-configs.csv. Only the external toolchains are tried, because building a Buildroot toolchain would take too long. An alternative toolchains CSV file can be specified with the -t option. This file should have lines consisting of the path to the toolchain config fragment and the required host architecture, separated by a comma. The config fragments should contain only the toolchain and architecture settings. By default, a useful subset of toolchains is tested. If needed, all toolchains can be tested (-a), an arbitrary number of toolchains (-n in order, -r for random).
Hands on
In these examples I'm going to build the CRIU package that i recently worked on.
First I will create a config snippet that contains the necessary options to enable my packet. In my case it's CRIU and HOST_PYTHON3:
1 cat > criu.config <<EOF
2 BR2_PACKAGE_HOST_PYTHON3=y
3 BR2_PACKAGE_CRIU=y
4 EOFI can now start test-pkg and provide the config snippet:
1 utils/test-pkg -c criu.config -p criu
2 bootlin-armv5-uclibc [1/6]: FAILED
3 bootlin-armv7-glibc [2/6]: FAILED
4 bootlin-armv7m-uclibc [3/6]: SKIPPED
5 bootlin-x86-64-musl [4/6]: OK
6 br-arm-full-static [5/6]: SKIPPED
7 sourcery-arm [6/6]: FAILED
8 6 builds, 2 skipped, 3 build failed, 0 legal-info failed, 0
9show-info failedSome of them fails, the end of the build log is available at ~/br-test-pkg/*/logfile.
Now start read the logfiles and fix the errors for each failed test.
Once the test-pkg (without -a) will have no failures, it will be a good test to retry it with the -a (run all tests) option:
1 utils/test-pkg -a -c criu.config -p criuAs soon as more and more tests passes, it takes unnecessary amount of time to run the same (successful) test again, so it's better to hand-pick those tests that actually fails.
To do this you may provide a specific list of toolchain configurations:
1 cp support/config-fragments/autobuild/toolchain-configs.csv
2criu-toolchains.csv
3 # edit criu-toolchains.csv to keep your toolchains of interest.
4 utils/test-pkg -a -t criu-toolchains.csv -c criu.config -p criuThis will retest only the toolchains kept in the csv.