Last month I stumbled upon guile-avahi, writen by Ludovic Courtès, the creator of GNU Guix. Strangely, this package never made its way to Guix repository. With Ludovic help, I released the 0.4 version and added it to Guix.
Having a fully declarative system mechanism in Guix is great. However, every now and then, having some auto-magical configuration can also be appreciable.
In this blog
Ludovic suggested to advertise
guix publish servers on the local network
using Avahi. As you may know, Guix supports fetching substitutes from several
When fetching substitutes from an unauthorized substitute server, the
substitute signature is checked against an authorized substitute server,
ci.guix.gnu.org, if it matches then the substitute can be
downloaded and installed.
The interesting part is that the unauthorized substitute server can be a mirror or a local machine allowing faster download speed than the official substitute server.
The only missing part to this puzzle is then, to automatically discover
guix publish servers on the local network using Avahi and add them the list of
As a first step, I created
(guix avahi) module, providing two procedures.
avahi-publish-service-thread ;advertise a service. avahi-browse-service-thread ;discover services.
Then, I added an
--advertise option to
guix publish using the first
procedure defined above,
Usage: guix publish [OPTION]... Publish /gnu/store over HTTP. [...] -a, --advertise advertise on the local network
and its counterpart in
(guix-publish-configuration (host "0.0.0.0") (port 3000) (advertise? #t) ;advertise using Avahi. (cache #f) (ttl #f) (compression-level 9))
avahi-browse, we can now discover kind users providing substitutes
on the local network.
mathieu@cervin:~$ avahi-browse -alr = wlp3s0 IPv4 guix-publish-elbruz _guix_publish._tcp local hostname = [elbruz.local] address = [192.168.1.51] port =  txt = 
Finally, I added a
--discover option to the
guix-daemon, so that it can
guix discover process that uses
discover and report the available substitute servers.
Usage: guix-daemon [OPTION...] guix-daemon -- perform derivation builds and store accesses [...] --discover[=yes/no] use substitute servers discovered on the local network
The discovered substitute servers are then automatically added to the list of
substitute servers. There's of course, a related field in the
(guix-configuration (authorize-key? #f) (discover? #t) ;enable substitute server discovery. (substitute-urls '("https://ci.guix.gnu.org")) (log-compression 'gzip) (build-accounts (* 4 16)) (extra-options (list "--max-jobs" "16" "--cores" "4")))
Then, running any Guix command, substitutes will be downloaded from local substitute server if possible:
mathieu@cervin:~$ guix pull Updating channel 'guix' from Git repository at 'https://git.savannah.gnu.org/git/guix.git'... Authenticating channel 'guix', commits 9edb3f6 to 770fb65 (2 new commits)... Building from this channel: guix https://git.savannah.gnu.org/git/guix.git 770fb65 Computing Guix derivation for 'x86_64-linux'... | substitute: updating substitutes from 'http://192.168.1.51:3000'... 100.0% substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0% The following derivations will be built: [...] substitute: updating substitutes from 'http://192.168.1.51:3000'... 100.0% downloading from http://192.168.1.51:3000/nar/gzip/kq69y9h1k3mv1dn289aayfikqizzgmgx-guix-manual guix-manual 23.2MiB/s 00:00 | 4.7MiB transferred downloading from http://192.168.1.51:3000/nar/gzip/8f4qppkg0qqmmi1rhzis9hxz6dmwrj3l-guix-cli guix-cli 968KiB/s 00:02 | 1.4MiB transferred
I have also added a
herd command so that discovery can be enabled and
disabled at run-time when using Guix System, this way:
herd discover guix-daemon on herd discover guix-daemon off
Finally, a patch adding this mechanism to the installer is on its way.
This development made me realize that improving substitute download speed was a big deal. I hope to find some time to work on restoring a CDN and maybe work on the IPFS branch.
Until then, many thanks to my sponsors and nice end-of-year festivities!