[PD-dev] buildbot on Macs and from git

[Hans-Christoph Steiner]

On Thu, 2010-09-16 at 02:31 +0200, András Murányi wrote:
> 
>         >
>         >                 At the moment buildbot fails with
>         >                 "exceptions.RuntimeError: Couldn't find
>         executable for
>         >                 'svn'"
>         >
>         >                 + the same for git
>         >
>         >                 I added ~buildbot/.bashrc to hopefully add
>         Fink stuff
>         >                 to buildbot's environment
>         >
>         >                 .hc
>         >
>         >
>         >                 Hmm, didn't work out, and
>         >                 net.sourceforge.buildbot.plist has sw/bin in
>         the path
>         >                 too, and svn is there, so i really don't
>         know...
>         >
>         >                 Andras
>         >
>         >
>         >         Ok, I was getting some builds from macosx104-i386,
>         but then it
>         >         disappeared, donno what happened there.
>          macosx104-powerpc
>         >         seems to be running still tho.
>         >
>         >
>         > Yea the process died somehow. I have restarted it with the
>         command:
>         >  /sw/bin/buildbot restart /Users/buildbot/macosx104-i386
>         >
>         > The output had some complaints:
>         > /sw/lib/python2.6/site-packages/twisted/persisted/sob.py:12:
>         > DeprecationWarning: the md5 module is deprecated; use
>         hashlib instead
>         >   import os, md5, sys
>         > /sw/lib/python2.6/site-packages/twisted/python/filepath.py:12:
>         > DeprecationWarning: the sha module is deprecated; use the
>         hashlib
>         > module instead
>         >   import sha
>         > /sw/lib/python2.6/site-packages/twisted/internet/_sslverify.py:5:
>         > DeprecationWarning: the md5 module is deprecated; use
>         hashlib instead
>         >   import itertools, md5
>         > Following twistd.log until startup finished..
>         > /sw/lib/python2.6/site-packages/buildbot/scripts/logwatcher.py:52:
>         > PotentialZombieWarning: spawnProcess called, but the SIGCHLD
>         handler
>         > is not installed. This probably means you have not yet
>         called
>         > reactor.run, or called reactor.run(installSignalHandler=0).
>         You will
>         > probably never see this process finish, and it may become a
>         zombie
>         > process.
>         >   env=os.environ,
>         > Removing stale
>         pidfile /Users/buildbot/macosx104-i386/twistd.pid
>         >
>         >         I worked on the pd-master/master.cfg a bit,
>         including changing
>         >         some of the names to be more consistent.  I also got
>         pure-data
>         >         building from Miller's git.
>         >
>         >         http://128.238.56.50:8010/builders/pure-data%20Linux
>         %
>         >         20debian-stable-i386/builds/6
>         >
>         >         .hc
>         >
>         >
>         > Good! I saw you stared experimenting with a builder for the
>         externals
>         > too - do i understand right that at the end we will have
>         every
>         > external built separately? I was thinking about breaking
>         them out to a
>         > separate master, but then we'd need to duplicate every slave
>         setup, so
>         > finally i think they could stay in the main master, and we
>         could have
>         > each of their have their own "category" name, which allows
>         for some
>         > selection at the web page.
>         > Also note that for the builders, you can define an array
>         with
>         > "slavenames:" instead of a single string "slavename", so you
>         can test
>         > the same builder on multiple slaves at the same time. At the
>         end they
>         > have to broken down to one slave per builder, otherwise the
>         diag
>         > output is not easy to understand.
>         >
>         > Changing descriptionDone values to past tense like
>         "compiled" may not
>         > make sense when the step fails and the output goes like
>         "compiled
>         > failed". Also there are things like "autogen" which don't
>         have a
>         > proper past tense... :)
>         > Another thing i noticed an "svn update" by itself, i think
>         we shall
>         > have the sources explicitly in master.cfg otherwise it will
>         fail where
>         > the slave got reset. Also you told before we wanted
>         "clobber" (tabula
>         > rasa) checkout not an update...
>         > I saw the "make install", "make uninstall" steps in the
>         output -
>         > having these would make much sense, fyi tests can be called
>         with
>         > Test() which has some advantages over ShellCommand() like it
>         doesn't
>         > make the whole build halt on failure.
>         > BTW switching the sources to git is easy, what we have to
>         work out is
>         > Git polling. It's built into 0.8.1 but needs to triggered
>         from git for
>         > 0.7.12. And then we have this thing with the poller to
>         explain it
>         > which builder to start upon updates... a.k.a. the cake :)
>         >
>         > Andras
>         
>         
>         One approach would be to use multiple PBChangeSource things
>         and have
>         commit hooks report to buildbot that they should build:
>         
> 
> Actually there is only one PBChangeSource needed/possible because it's
> a listener, and multiple commit hooks can communicate with it from the
> repos.
>  
>         Another promising thing is loki, a web interface for easily
>         setting up
>         master/slaves:
>         
>         https://fedorahosted.org/loki/
>         
> 
> Hm. Seems a bit young to me...
> 
> I see that at the moment every factory is "under construcion" so i
> won't touch the config file for a while so that you can edit around.
> (Which makes me think about putting it in svn later...)
> http://128.238.56.50:8010/one_box_per_builder
> 
> Andras

I'm actually not going to have any time to look at this for a couple
weeks, so please edit away.  About the 'externals' collection, one idea
I had was to set up a different step for each library, then have a
common collection of steps for easy reuse.  For the Makefile-based libs
the steps could be: make, make install, make dist.  For autotools, we
could have ./configure, make, make install.  If it uses automake, then
these should also be available: make uninstall, make dist.

Perhaps that would be manageable, but it seems there might be a better
way.

.hc