GNU Guix continuous integration - NLNet grant

January 19, 2021

I am glad to let you know that NLNet will support my project of improving GNU Guix continuous integration system. The project will be funded through the NGI0 PET Fund, a fund established by NLnet with financial support from the European Commission's Next Generation, as explained here.


GNU Guix is using Cuirass as continuous integration software. It's a general purpose build automation server written in GNU Guile that checks out sources from VCS repositories, execute build jobs and store build results in a database. Cuirass also provides a web interface to monitor the build results.

Cuirass is supposed to build and provide binary substitutes for every package or system image definition. Without those substitutes available in time, the user experience is really degraded, as the user has to allocate important time and computation power resources into package building.

Cuirass has been supported by the Google summer of code project (2016, 2017 and 2018). It is right now developed voluntarily mostly by the GNU Guix community. This grant will allow me to dedicate 6 months of my time to work on improving Cuirass.

I plan to achieve the following tasks.

Add dynamic offloading support

Implement a dynamic offloading mode. This mode will allow an offloading server to distribute work among different build machines.

The network discovery between the offloading server and the build machines will be done using the Guile-Avahi library.

The build machines must be able to come an go dynamically. They will pick build tasks from the offloading server using a work stealing scheduling strategy.

The communication between the server and the build machines will be implemented using Guile-ZMQ library.

Improve the test suite

Implement different dynamic offloading scenarios in the Cuirass test suite to secure the development and prevent the risks of regressions.

Improve Cuirass build monitoring interface

Add build monitoring tools to Cuirass so that system administrators and users can closely monitor the health of the whole continuous integration system and promptly report any issue or slowdown.

Improve Cuirass build reporting interface

Create a user account section to setup customized monitoring dashboards, subscribe to build failures notifications and get notified of the availability of new substitutes.

Starting to work!

As a first step, I have already migrated Cuirass from SQLite to PostgreSQL. I might write something about that later on. I also started working on the dynamic offloading mechanism with some encouraging results.

I will of course report the results periodically on this website and on the GNU Guix blog.

Don't hesitate to contact me by email or on #guix to discuss your impressions as well as the new features you would like to see in Cuirass.