REST service on top of Marcuscom Tinderbox

Like many people who work on FreeBSD Ports I've been using great Marcuscom Tinderbox to verify my changes. Some time ago I started to realise that it's quite a boring task to schedule a lot of builds using web interface again and again and finally I've managed to make myself start making the process a little bit easier.

It seems that building a REST service on top on the existing is a nice way to go, so that's what I've been doing last few days and finally I have a prototype now.

As I didn't want to patch original tinderbox's webui sources, I just created a sub-directory in it called 'api' and started there. To my surprise, I wasn't able to find any ready to use solution to implement REST service in PHP (maybe I've missed something though), so ended up with a few php scripts and rewrite rules to make URLs feel RESTy. It was a quite hard part of the task since I haven't been coding in PHP for years and don't feel comfortable with it even after few days of practice.

Simultaneously I've started coding a client for which I choose Python.

Workflow

The goal was to support minimum but most important (for me at least) scenario -- testing an update for a port. Let's assume the port being tested is security/gnutls-devel. The process looks like that:

1. Produce a patch and apply it to the tree (can be done via ssh, or nfs, whatever, and could be easily scripted up)
2. Check builds we have:


(21:33) novel@fsol:~ %> tbc build
id name status current port updated
1 8.x-FreeBSD IDLE None 2011-01-30 14:05:25
(21:40) novel@fsol:~ %>


So, we see an idle build here with id 1. Not so much choices, so let's use it, but first let's see what's in the queue:


(21:40) novel@fsol:~ %> tbc queue
id username port build pri status enqueued completed
10 novel security/gnutls 8.x-FreeBSD 10 SUCCESS 2011-01-30 13:54:43 2011-01-30 14:05:27
(21:45) novel@fsol:~ %>


Now adding a port to the queue:


(21:46) novel@fsol:~ %> tbc queue add -b 1 security/gnutls-devel
(21:46) novel@fsol:~ %>


Let's check if new port build added successfully:


(21:46) novel@fsol:~ %> tbc queue
id username port build pri status enqueued completed
10 novel security/gnutls 8.x-FreeBSD 10 SUCCESS 2011-01-30 13:54:43 2011-01-30 14:05:27
11 novel security/gnutls-devel 8.x-FreeBSD 10 ENQUEUED 2011-01-30 21:46:20 None
(21:46) novel@fsol:~ %>


Cool, it's here! Now we can see that our build turned into PREPARE state:


(21:48) novel@fsol:~ %> tbc build
id name status current port updated
1 8.x-FreeBSD PREPARE None 2011-01-30 21:48:15
(21:48) novel@fsol:~ %>


All we have to do now is wait and poll for changes. Now we can see it's building:


(21:59) novel@fsol:~ %> tbc build
id name status current port updated
1 8.x-FreeBSD PORTBUILD gnutls-devel-2.11.5 2011-01-30 22:00:25
(22:00) novel@fsol:~ %>


And finally we see it's done:

(22:07) novel@fsol:~ %> tbc queue
id username port build pri status enqueued completed
11 novel security/gnutls 8.x-FreeBSD 10 SUCCESS 2011-01-30 21:46:20 2011-01-30 21:57:18
12 novel security/gnutls-devel 8.x-FreeBSD 10 SUCCESS 2011-01-30 21:50:54 2011-01-30 22:09:22
(22:15) novel@fsol:~ %> tbc queue 11


For everyone who's interested I've uploaded sources on github:

• service (README, API Reference)
  


• client (README)
  

NOTICE: it's an early work in progress! The code is unstable and lacks a lot of features. Anyway, any feedback about an idea and implementation is welcome.