[PD-dev] buildbot on Macs and from git

[Hans-Christoph Steiner]

On Wed, 2010-09-15 at 22:11 +0200, András Murányi wrote:
> 
> 
> 2010/9/15 Hans-Christoph Steiner <hans at at.or.at>
>         
>         On Sep 14, 2010, at 5:08 PM, András Murányi wrote:
>         
>         
>         
>                 
>                 At the moment buildbot fails with
>                 "exceptions.RuntimeError: Couldn't find executable for
>                 'svn'"
>                 
>                 + the same for git
>                 
>                 I added ~buildbot/.bashrc to hopefully add Fink stuff
>                 to buildbot's environment
>                 
>                 .hc
>                 
>                 
>                 Hmm, didn't work out, and
>                 net.sourceforge.buildbot.plist has sw/bin in the path
>                 too, and svn is there, so i really don't know...
>                 
>                 Andras
>         
>         
>         Ok, I was getting some builds from macosx104-i386, but then it
>         disappeared, donno what happened there.  macosx104-powerpc
>         seems to be running still tho.
>         
> 
> Yea the process died somehow. I have restarted it with the command:
>  /sw/bin/buildbot restart /Users/buildbot/macosx104-i386
> 
> The output had some complaints:
> /sw/lib/python2.6/site-packages/twisted/persisted/sob.py:12:
> DeprecationWarning: the md5 module is deprecated; use hashlib instead
>   import os, md5, sys
> /sw/lib/python2.6/site-packages/twisted/python/filepath.py:12:
> DeprecationWarning: the sha module is deprecated; use the hashlib
> module instead
>   import sha
> /sw/lib/python2.6/site-packages/twisted/internet/_sslverify.py:5:
> DeprecationWarning: the md5 module is deprecated; use hashlib instead
>   import itertools, md5
> Following twistd.log until startup finished..
> /sw/lib/python2.6/site-packages/buildbot/scripts/logwatcher.py:52:
> PotentialZombieWarning: spawnProcess called, but the SIGCHLD handler
> is not installed. This probably means you have not yet called
> reactor.run, or called reactor.run(installSignalHandler=0). You will
> probably never see this process finish, and it may become a zombie
> process.
>   env=os.environ,
> Removing stale pidfile /Users/buildbot/macosx104-i386/twistd.pid
>  
>         I worked on the pd-master/master.cfg a bit, including changing
>         some of the names to be more consistent.  I also got pure-data
>         building from Miller's git.
>         
>         http://128.238.56.50:8010/builders/pure-data%20Linux%
>         20debian-stable-i386/builds/6
>         
>         .hc
>         
> 
> Good! I saw you stared experimenting with a builder for the externals
> too - do i understand right that at the end we will have every
> external built separately? I was thinking about breaking them out to a
> separate master, but then we'd need to duplicate every slave setup, so
> finally i think they could stay in the main master, and we could have
> each of their have their own "category" name, which allows for some
> selection at the web page.
> Also note that for the builders, you can define an array with
> "slavenames:" instead of a single string "slavename", so you can test
> the same builder on multiple slaves at the same time. At the end they
> have to broken down to one slave per builder, otherwise the diag
> output is not easy to understand.
> 
> Changing descriptionDone values to past tense like "compiled" may not
> make sense when the step fails and the output goes like "compiled
> failed". Also there are things like "autogen" which don't have a
> proper past tense... :)
> Another thing i noticed an "svn update" by itself, i think we shall
> have the sources explicitly in master.cfg otherwise it will fail where
> the slave got reset. Also you told before we wanted "clobber" (tabula
> rasa) checkout not an update...
> I saw the "make install", "make uninstall" steps in the output -
> having these would make much sense, fyi tests can be called with
> Test() which has some advantages over ShellCommand() like it doesn't
> make the whole build halt on failure.
> BTW switching the sources to git is easy, what we have to work out is
> Git polling. It's built into 0.8.1 but needs to triggered from git for
> 0.7.12. And then we have this thing with the poller to explain it
> which builder to start upon updates... a.k.a. the cake :)
> 
> Andras

One approach would be to use multiple PBChangeSource things and have
commit hooks report to buildbot that they should build:

http://stackoverflow.com/questions/2795386/support-for-multiple-repositories-using-buildbot

Another promising thing is loki, a web interface for easily setting up
master/slaves:

https://fedorahosted.org/loki/

.hc