This year, I was lucky to be sponsored by the FreeBSD Foundation to improve libvirt's FreeBSD integration. One of the areas of focus was enabling CI. Actually, I had wanted to have some form of CI for a while, as manually testing things is quite time-consuming. Quite often, I don’t have enough time to test RC releases, so sometimes the releases ship with issues on FreeBSD. I knew about the libvirt TCK project (TCK stands for "Technology Compatibility Kit"), but I wasn’t sure whether it would be easier to implement my own testing routine or port the existing one to run on FreeBSD.
When I started this project, I decided to re-evaluate libvirt-tck, because it makes much more sense to re-use the existing tests—even if the initial porting requires more effort—than to implement everything from scratch.
The first steps of getting libvirt-tck to run were a bit tricky, because it starts with image generation using virt-builder. The virt-builder tool is part of libguestfs, which has not been ported to FreeBSD. I imagine porting libguestfs is not a trivial task, and it has been listed on the FreeBSD Wanted Ports list for a while. I didn’t want to dig into porting libguestfs, so I tried a few different workarounds—for example, creating a wrapper script to run virt-builder on a Linux host via SSH, then copying the resulting image with scp or using a shared partition. I also tried running it with Linuxulator. These methods had mixed success, but I later realized that once the image is generated, it can be reused for subsequent runs. It’s not ideal, but it works for now. With the image available, I could move on to the next phase: adjusting the tests themselves. This mainly involved handling cases where bhyve doesn’t support certain features and making a few patches to libvirt’s driver.
Some of that work was not very exciting, but it did result in new features being implemented, such as virtio-rnd support. Sometimes it was simply easier to add a missing feature than to handle its absence. It took a few merge requests to libvirt-tck and a few libvirt releases to get things working, but eventually the main set of tests I wanted to run worked fine. These tests include:
- domain
- networks
- storage
The main tests I wanted to run were successful: domain, networks, and storage. However, there are still a few test groups that I don’t run, either because they aren’t relevant or aren’t supported on FreeBSD:
- hooks
- nwfilter
- qemu
- selinux
I think I need to take a closer look at the hooks tests. In the future, libvirt on FreeBSD might gain nwfilter support, so those tests could become relevant, but it’s unlikely to happen soon. I had no plans to support the qemu driver on FreeBSD (why, when bhyve is available?), but with the vmm(4) accelerator support for QEMU, it might become useful. And selinux is obviously not applicable.
The final part was Jenkins configuration. Frankly, I don’t have much experience configuring Jenkins, though I do use it fairly often at work.
My "pipeline" (which is not really a pipeline in the strict sense) consists of two steps:
- Build a "-devel" version of the libvirt port (pointing to a Git repository) using poudriere(8), and publish the results to a package repo.
- Spin up a BastilleBSD container, install the libvirt packages from the previous step, check out libvirt-tck from Git, run the tests, and publish the results.
I use the "Job DSL" plugin to store job configuration, which is available in the Git repo. While the setup works, there are still several areas I’d like to improve:
- I’d like to figure out how to build an arbitrary Git revision of the port, without updating DISTVERSION in Makefile or the checksums in distinfo. Probably it would be easier to just build libvirt manually. That would also simplify the pipeline.
- I need to see if I can store not only job configurations in plain text, but also a complete Jenkins configuration, so I don’t have to manually install plugins and so on.
- I definitely need to script the test image creation process.
No comments:
Post a Comment