[PD-dev] win32 build farm 'msys.exe' (WAS: symlinks render pdstring unbuildable on MinGW/Windows)

[Bryan Jurish]

moin Hans,

On 2009-04-18 06:57:53, Hans-Christoph Steiner <hans at eds.org> appears to
have written:
> 
> The bad news is that it seems that my bad diagnosis led you on a wide
> goose chase thru the pain of Windows development. 

It may or may not have been a bad diagnosis (although it certainly was a
pain ;-) -- I still haven't verified the former, since I don't use (or
understand) all of the auto-build scripts.  I'm still kinda hoping
IOhannes might make some headway there...

> Apparently,
> string2any and friends are still not getting built.  In fact all of the
> 'moocow' is empty on Windows.  Here's the bug report:
> 
> https://sourceforge.net/tracker/?func=detail&aid=2138593&group_id=55736&atid=478070

I noticed this too, and am currently trying to figure out exactly why.
Actually, I didn't really expect any of my externals to build on windows
currently, since I was missing test clauses for win32 in
ax_pd_external.m4.  I've added the test clauses now, and at least win32
on the build farm machine is getting properly recognized, but I'm
getting a whole mess of other (stupid) errors which are preventing
manual compilation:

'msys.bat' doesn't run on the autobuild machine from the cygwin shell I
get when I ssh in: is this (easily) fixable?  I can run
"/cygdrive/c/msys/1.0/bin/sh" manually, but it's not really useable as
an interactive shell.  So far I've been explicitly calling "$MSYS_SH
./configure" to test, for MSYS_SH=/cygdrive/c/msys/1.0/bin/sh.exe

I've tried manually prepending
"/cygdrive/c/msys/1.0/bin:/cygdrive/c/MinGW/bin" to PATH, and that gets
me all the mingw programs, but their compiled-in paths (e.g. include
paths for gcc) are foobared, since cygwin's pseudo-filesystem doesn't
jive with the msys one.  In particular, I'm getting:

pddev at windowsxp-i386$ sh ./configure \
	--with-pd-include="/home/pd/auto-build/pd-extended/pd/src"
...
checking for string.h... no
configure: WARNING: could not find standard C headers -- things may get ugly

... similar problems with linker paths.  I've hacked this for now by
setting CPPFLAGS and LDFLAGS, and the updated version of pdstring seems
to build ok.

> Part of the problem might be MinGW's very old version of autotools:
> automake 1.7 and libtool 1.4, which is a big bummer.

This shouldn't be a problem, since I've been storing autoconf-generated
stuff in svn anyways, and calling AM_MAINTAINER_MODE to ensure that it
doesn't get implicitly rebuilt.  See the thread "[PD-dev] moocow: svn
and compilation issues" for more details.

In a related question, should the auto-build process for the win32
build-farm machine be trying to build everything from the LIB_TARGETS
variable in externals/Makefile?  If so, then I must be being pretty
dense, because I can't see where my builds are getting called (I see
install calls, but no configuration or compilation), much less where
they might be failing...

marmosets,
	Bryan

> On Mar 7, 2009, at 5:14 PM, Bryan Jurish wrote:
> 
>> moin Hans,
>>
>> Short version: should be fixed now.  package-local links to the common
>> code are now no longer tracked by SVN, but kludged into place by an
>> "./svn-prepare.sh" (symlink ;-) in each package directory, alternately
>> by the DIR.autogen_stamp target in extended/Makefile.
>>
>> More comments: see below.
>>
>> On 2009-03-05 23:44:58, Hans-Christoph Steiner <hans at eds.org> appears to
>> have written:
>>> On Mar 5, 2009, at 5:22 PM, Bryan Jurish wrote:
>>>> On 2009-03-05 21:33:36, Hans-Christoph Steiner <hans at eds.org>
>>>> appears to
>>>> have written:
>>>>> On Mar 4, 2009, at 4:23 PM, Bryan Jurish wrote:
>>>>>> On 2009-03-04 05:37:40, Hans-Christoph Steiner <hans at eds.org>
>>>>>> appears to
>>>>>> have written:
>>>>>>> I was just trying to build string2any and friends on Windows for a
>>>>>>> student, but the symlinks used in moocow are throwing a huge wrench
>>>>>>> in
>>>>>>> the process.
>>>>>>> They show up at .lnk files, and are not links at all.
>>>>>>> That's because cygwin translates symlinks into Windows shortcuts,
>>>>>>> aka .lnk.
>> [snip]
>>>> the rsync "-L" flag ("--copy-links") works for me here, even with a
>>>> preceding "-a" ("--archive")... does that not work on cygwin?  The only
>>>> times I've ever had problems with "-L" were symlink cycles (./dir ->
>>>> .),
>>>> which I certainly am not inserting into the repository.
>>
>>> Sorry, I had no intention to insult or demean, I guess I was just
>>> terse.
>>
>> No worries; the whole mess really arises from my not having the time to
>> dig around in the automake internals (bad programmer, no biscuit): I'm
>> sure there's a way to get the common code into its own automake+autoconf
>> package, but I haven't figured it out yet.  Ideally, I'd like to have an
>> family of automake "_PDEXTERNALS" targets (analagous to "_PROGRAMS",
>> "_LIBRARIES", "_DATA", etc.), but that's not happening yet; hence the
>> intermediate solution "../common", which goes pretty far towards
>> eliminating the headaches necessary to roll up a new external package or
>> add new functionality to an existing package.
>>
>>> The bad news is that its not that simple.  I added "--no-l
>>> --copy-links" to cygwin rsync and it still doesn't work.
>>
>> Curioser and curioser.  I just tried an rsync from linux (with symlinks)
>> to cygwin with "-a --no-l --copy-links" here, and I got copies of the
>> symlinks rather than windoof shortcuts.  Can you point me at the (script
>> containining the) full rsync call so I can test that here?
>>
>>> There is
>>> nothing you have to do here, I just thought it'd be nice to have those
>>> externals included in Windows.  Here are the logs, it seems it doesn't
>>> find the compiler properly:
>>>
>>> http://bxmc.poly.edu/pdlab/moocow_log.txt
>>> http://bxmc.poly.edu/pdlab/config.log
>>
>> 403 forbidden
>>
>> not really important though, since the AC_CONFIG_LINKS() symlinks aren't
>> the problem.
>>
>>> What I don't understand is why this code needs such a complicated build
>>> system? As far as I can tell, it is mostly pretty standard C code.
>>
>> If you're asking that now, it's probably a good thing I didn't delegate
>> external building to libtool ;-)
>>
>> It's (mostly) not the *code* which needs automake, it's *me* who wants
>> it.  Having 'distcheck' and 'dist' targets generated auto-magically is
>> really outrageously spiffy.  And with the shared code, building a new
>> external in an existing package is as easy as setting a couple of make
>> variables:
>>
>> pdexterns_PROGRAMS += myexternal
>> myexternal_SOURCES = myexternal.c mycommon.c mycommon.h
>>
>> ... and automake takes care of the rest (make, make install, make
>> uninstall, ...).  Creating a new external package using this system is
>> pretty easy too: see the SVN directory externals/moocow/hello for a
>> complete working tutorial, or its README
>> here:
>> http://pure-data.svn.sourceforge.net/viewvc/pure-data/trunk/externals/moocow/hello/README.txt?view=markup
>>
>>
>> The ./configure options are there for debugging, support for multiple
>> installations of Pd on the same machine, as well as support for
>> pd-extended (e.g. --enable-object-externals).
>>
>> autotools support for use environment variables (CFLAGS, CPPFLAGS, etc.)
>> is really handy too, e.g. for messing with optimization & debugging
>> flags.  I wish every external build system would support these.
>>
>> That said, in some cases the code actually *does* require some
>> platform-dependent initialization in ./configure.  [locale] for instance
>> checks for definitions of all of the "LC_*" variables it supports,
>> [flite] needs to know where to find the libflite include files, as well
>> as how to link to libflite: this is exactly the kind of thing autoconf
>> was made to handle, and which it does quite elegantly.
>>
>>> I
>>> find that in the long run, simple Makefiles are the least work overall.
>>> To each his own, I suppose, or maybe I'm missing something.
>>
>> If I were maintaining only 1 or 2 external packages (or a single global
>> build system ;-), I think I might tend to agree.  As it is, I think the
>> autotools beat copy+paste Makefiles (which I still use a lot, e.g. for
>> my LaTeX documents, which rarely need to be synchronized or updated once
>> the paper has been written) hands down.  Are
>> automake+autoconf+autoheader overkill for my externals?  Sure they are:
>> I think this is demonstrated pretty well by the fact that [sprinkler]
>> and [pdstring] built for years in what became flatspace/ with 2-liner C
>> files to the tune of:
>>
>>  #define PACKAGE_VERISON "cvs"
>>  #include "../../moocow/sprinkler/sprinkler.c"
>>
>> Did flatspace builds get me 'make dist'?  Nope.  Did they get me 'make
>> distcheck'?  No way.  Could I configure them for target systems to which
>> I myself have no access simply by querying the availability of the
>> relevant C libraries and includes?  Uh... yeah, right...  enough; I love
>> externals/build rsp. flatspace and everything it's done for me, and I
>> really don't want to start a flame war... I hope the windoof builds work
>> now (though I am still curious about that apparent rsync bug)...
>>
>> marmosets,
>>     Bryan
>>
>> --Bryan Jurish                           "There is *always* one more
>> bug."
>> jurish at ling.uni-potsdam.de      -Lubarsky's Law of Cybernetic Entomology
> 
> 
> 
> ----------------------------------------------------------------------------
> 
> 
> I spent 33 years and four months in active military service and during
> that period I spent most of my time as a high class muscle man for Big
> Business, for Wall Street and the bankers.      - General Smedley Butler
> 
> 
> 
> 

-- 
Bryan Jurish                           "There is *always* one more bug."
jurish at ling.uni-potsdam.de      -Lubarsky's Law of Cybernetic Entomology