[PD-dev] win32 build farm 'msys.exe' (WAS: symlinks render pdstring unbuildable on MinGW/Windows)

Hans-Christoph Steiner hans at at.or.at
Mon Apr 20 04:39:58 CEST 2009

On Apr 19, 2009, at 5:22 AM, Bryan Jurish wrote:

> moin Hans,
> On 2009-04-18 06:57:53, Hans-Christoph Steiner <hans at eds.org>  
> appears to
> have written:
>> The bad news is that it seems that my bad diagnosis led you on a wide
>> goose chase thru the pain of Windows development.
> It may or may not have been a bad diagnosis (although it certainly  
> was a
> pain ;-) -- I still haven't verified the former, since I don't use (or
> understand) all of the auto-build scripts.  I'm still kinda hoping
> IOhannes might make some headway there...

You don't really need to understand the auto-build scripts, all that  
needs to work is this:

cd pure-data/externals
make moocow
make moocow_install

But we really should replace my scripts with something like hudson or  


>> Apparently,
>> string2any and friends are still not getting built.  In fact all of  
>> the
>> 'moocow' is empty on Windows.  Here's the bug report:
>> https://sourceforge.net/tracker/?func=detail&aid=2138593&group_id=55736&atid=478070
> I noticed this too, and am currently trying to figure out exactly why.
> Actually, I didn't really expect any of my externals to build on  
> windows
> currently, since I was missing test clauses for win32 in
> ax_pd_external.m4.  I've added the test clauses now, and at least  
> win32
> on the build farm machine is getting properly recognized, but I'm
> getting a whole mess of other (stupid) errors which are preventing
> manual compilation:
> 'msys.bat' doesn't run on the autobuild machine from the cygwin  
> shell I
> get when I ssh in: is this (easily) fixable?  I can run
> "/cygdrive/c/msys/1.0/bin/sh" manually, but it's not really useable as
> an interactive shell.  So far I've been explicitly calling "$MSYS_SH
> ./configure" to test, for MSYS_SH=/cygdrive/c/msys/1.0/bin/sh.exe
> I've tried manually prepending
> "/cygdrive/c/msys/1.0/bin:/cygdrive/c/MinGW/bin" to PATH, and that  
> gets
> me all the mingw programs, but their compiled-in paths (e.g. include
> paths for gcc) are foobared, since cygwin's pseudo-filesystem doesn't
> jive with the msys one.  In particular, I'm getting:
> pddev at windowsxp-i386$ sh ./configure \
> 	--with-pd-include="/home/pd/auto-build/pd-extended/pd/src"
> ...
> checking for string.h... no
> configure: WARNING: could not find standard C headers -- things may  
> get ugly
> ... similar problems with linker paths.  I've hacked this for now by
> setting CPPFLAGS and LDFLAGS, and the updated version of pdstring  
> seems
> to build ok.

I just documented a way to get the same MSYS shell used by the  
builds.  You need rdesktop, for better or worse:


But then I succeeded in locking all accounts out from network  
logins... DOH!  I'll try to fix it tomorrow.

>> Part of the problem might be MinGW's very old version of autotools:
>> automake 1.7 and libtool 1.4, which is a big bummer.
> This shouldn't be a problem, since I've been storing autoconf- 
> generated
> stuff in svn anyways, and calling AM_MAINTAINER_MODE to ensure that it
> doesn't get implicitly rebuilt.  See the thread "[PD-dev] moocow: svn
> and compilation issues" for more details.
> In a related question, should the auto-build process for the win32
> build-farm machine be trying to build everything from the LIB_TARGETS
> variable in externals/Makefile?  If so, then I must be being pretty
> dense, because I can't see where my builds are getting called (I see
> install calls, but no configuration or compilation), much less where
> they might be failing...

You are correct.  Search for 'moocow_install:' in externals/Makefile.   
That's where its being called.


> marmosets,
> 	Bryan
>> On Mar 7, 2009, at 5:14 PM, Bryan Jurish wrote:
>>> moin Hans,
>>> Short version: should be fixed now.  package-local links to the  
>>> common
>>> code are now no longer tracked by SVN, but kludged into place by an
>>> "./svn-prepare.sh" (symlink ;-) in each package directory,  
>>> alternately
>>> by the DIR.autogen_stamp target in extended/Makefile.
>>> More comments: see below.
>>> On 2009-03-05 23:44:58, Hans-Christoph Steiner <hans at eds.org>  
>>> appears to
>>> have written:
>>>> On Mar 5, 2009, at 5:22 PM, Bryan Jurish wrote:
>>>>> On 2009-03-05 21:33:36, Hans-Christoph Steiner <hans at eds.org>
>>>>> appears to
>>>>> have written:
>>>>>> On Mar 4, 2009, at 4:23 PM, Bryan Jurish wrote:
>>>>>>> On 2009-03-04 05:37:40, Hans-Christoph Steiner <hans at eds.org>
>>>>>>> appears to
>>>>>>> have written:
>>>>>>>> I was just trying to build string2any and friends on Windows  
>>>>>>>> for a
>>>>>>>> student, but the symlinks used in moocow are throwing a huge  
>>>>>>>> wrench
>>>>>>>> in
>>>>>>>> the process.
>>>>>>>> They show up at .lnk files, and are not links at all.
>>>>>>>> That's because cygwin translates symlinks into Windows  
>>>>>>>> shortcuts,
>>>>>>>> aka .lnk.
>>> [snip]
>>>>> the rsync "-L" flag ("--copy-links") works for me here, even  
>>>>> with a
>>>>> preceding "-a" ("--archive")... does that not work on cygwin?   
>>>>> The only
>>>>> times I've ever had problems with "-L" were symlink cycles (./ 
>>>>> dir ->
>>>>> .),
>>>>> which I certainly am not inserting into the repository.
>>>> Sorry, I had no intention to insult or demean, I guess I was just
>>>> terse.
>>> No worries; the whole mess really arises from my not having the  
>>> time to
>>> dig around in the automake internals (bad programmer, no biscuit):  
>>> I'm
>>> sure there's a way to get the common code into its own automake 
>>> +autoconf
>>> package, but I haven't figured it out yet.  Ideally, I'd like to  
>>> have an
>>> family of automake "_PDEXTERNALS" targets (analagous to "_PROGRAMS",
>>> "_LIBRARIES", "_DATA", etc.), but that's not happening yet; hence  
>>> the
>>> intermediate solution "../common", which goes pretty far towards
>>> eliminating the headaches necessary to roll up a new external  
>>> package or
>>> add new functionality to an existing package.
>>>> The bad news is that its not that simple.  I added "--no-l
>>>> --copy-links" to cygwin rsync and it still doesn't work.
>>> Curioser and curioser.  I just tried an rsync from linux (with  
>>> symlinks)
>>> to cygwin with "-a --no-l --copy-links" here, and I got copies of  
>>> the
>>> symlinks rather than windoof shortcuts.  Can you point me at the  
>>> (script
>>> containining the) full rsync call so I can test that here?
>>>> There is
>>>> nothing you have to do here, I just thought it'd be nice to have  
>>>> those
>>>> externals included in Windows.  Here are the logs, it seems it  
>>>> doesn't
>>>> find the compiler properly:
>>>> http://bxmc.poly.edu/pdlab/moocow_log.txt
>>>> http://bxmc.poly.edu/pdlab/config.log
>>> 403 forbidden
>>> not really important though, since the AC_CONFIG_LINKS() symlinks  
>>> aren't
>>> the problem.
>>>> What I don't understand is why this code needs such a complicated  
>>>> build
>>>> system? As far as I can tell, it is mostly pretty standard C code.
>>> If you're asking that now, it's probably a good thing I didn't  
>>> delegate
>>> external building to libtool ;-)
>>> It's (mostly) not the *code* which needs automake, it's *me* who  
>>> wants
>>> it.  Having 'distcheck' and 'dist' targets generated auto- 
>>> magically is
>>> really outrageously spiffy.  And with the shared code, building a  
>>> new
>>> external in an existing package is as easy as setting a couple of  
>>> make
>>> variables:
>>> pdexterns_PROGRAMS += myexternal
>>> myexternal_SOURCES = myexternal.c mycommon.c mycommon.h
>>> ... and automake takes care of the rest (make, make install, make
>>> uninstall, ...).  Creating a new external package using this  
>>> system is
>>> pretty easy too: see the SVN directory externals/moocow/hello for a
>>> complete working tutorial, or its README
>>> here:
>>> http://pure-data.svn.sourceforge.net/viewvc/pure-data/trunk/externals/moocow/hello/README.txt?view=markup
>>> The ./configure options are there for debugging, support for  
>>> multiple
>>> installations of Pd on the same machine, as well as support for
>>> pd-extended (e.g. --enable-object-externals).
>>> autotools support for use environment variables (CFLAGS, CPPFLAGS,  
>>> etc.)
>>> is really handy too, e.g. for messing with optimization & debugging
>>> flags.  I wish every external build system would support these.
>>> That said, in some cases the code actually *does* require some
>>> platform-dependent initialization in ./configure.  [locale] for  
>>> instance
>>> checks for definitions of all of the "LC_*" variables it supports,
>>> [flite] needs to know where to find the libflite include files, as  
>>> well
>>> as how to link to libflite: this is exactly the kind of thing  
>>> autoconf
>>> was made to handle, and which it does quite elegantly.
>>>> I
>>>> find that in the long run, simple Makefiles are the least work  
>>>> overall.
>>>> To each his own, I suppose, or maybe I'm missing something.
>>> If I were maintaining only 1 or 2 external packages (or a single  
>>> global
>>> build system ;-), I think I might tend to agree.  As it is, I  
>>> think the
>>> autotools beat copy+paste Makefiles (which I still use a lot, e.g.  
>>> for
>>> my LaTeX documents, which rarely need to be synchronized or  
>>> updated once
>>> the paper has been written) hands down.  Are
>>> automake+autoconf+autoheader overkill for my externals?  Sure they  
>>> are:
>>> I think this is demonstrated pretty well by the fact that  
>>> [sprinkler]
>>> and [pdstring] built for years in what became flatspace/ with 2- 
>>> liner C
>>> files to the tune of:
>>> #define PACKAGE_VERISON "cvs"
>>> #include "../../moocow/sprinkler/sprinkler.c"
>>> Did flatspace builds get me 'make dist'?  Nope.  Did they get me  
>>> 'make
>>> distcheck'?  No way.  Could I configure them for target systems to  
>>> which
>>> I myself have no access simply by querying the availability of the
>>> relevant C libraries and includes?  Uh... yeah, right...  enough;  
>>> I love
>>> externals/build rsp. flatspace and everything it's done for me,  
>>> and I
>>> really don't want to start a flame war... I hope the windoof  
>>> builds work
>>> now (though I am still curious about that apparent rsync bug)...
>>> marmosets,
>>>   Bryan
>>> --Bryan Jurish                           "There is *always* one more
>>> bug."
>>> jurish at ling.uni-potsdam.de      -Lubarsky's Law of Cybernetic  
>>> Entomology
>> ----------------------------------------------------------------------------
>> I spent 33 years and four months in active military service and  
>> during
>> that period I spent most of my time as a high class muscle man for  
>> Big
>> Business, for Wall Street and the bankers.      - General Smedley  
>> Butler
> -- 
> Bryan Jurish                           "There is *always* one more  
> bug."
> jurish at ling.uni-potsdam.de      -Lubarsky's Law of Cybernetic  
> Entomology


"It is convenient to imagine a power beyond us because that means we  
don't have to examine our own lives.", from "The Idols of  
Environmentalism", by Curtis White

More information about the Pd-dev mailing list