[PD-cvs] externals/pdp/doc/misc devdoc.html, 1.2, 1.3 layers.txt, 1.2, 1.3 overview.html, 1.2, 1.3 todo.jme, 1.2, 1.3

Fri Dec 16 01:51:07 CET 2005

Update of /cvsroot/pure-data/externals/pdp/doc/misc
In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv3209/doc/misc

Added Files:
	devdoc.html layers.txt overview.html todo.jme 
Log Message:
checking in pdp 0.12.4 from http://zwizwa.fartit.com/pd/pdp/pdp-0.12.4.tar.gz

--- NEW FILE: todo.jme ---
todo list of jme at off.net
------------------------
- a packet to trigger packet generator instead of bang
  o the created packet has the same format as the incoming packet

--- NEW FILE: overview.html ---
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html><head><title>Pure Data Packet</title></head>
<body>

<h1>Pure Data Packet</h1>

<h2>Introduction</h2>

Pure Data Packet (PDP) is an extension library for the computer music 
program <a href="http://www.pure-data.org">Pure Data</a> (PD), by <a href = 
"http://www-crca.ucsd.edu/~msp/software.html">Miller Puckette</a> and 
others. Its goal is to provide a way to use arbitrary data types (data 
packets) as messages that can be passed around inside PD, along side the 
standard PD numbers and symbol types. In short it puts any data object on 
the same level as a float or a symbol.

PDP runs on Linux and OSX. The OSX version depends on <a 
href="http://fink.sourceforge.net/">Fink</a>, which is not in the "point & 
click" stage yet, so setting it up will require some efford. There is no 
windows version. The reason for this is simple: i don't use windows myself. 
Porting would require writing code for input/output and getting the 
libraries PDP depends on to work. If anyone is willing to do this, just let 
me know. PDP can run without X Window, using SDL.

Currently, PDP's focus is on images and video, but there is no reason it 
should stay like that. There is limited support for matrix processing 
included in the main library (like Jitter or Gridflow). There is an 
extension library for 1D and 2D binary cellular automata, opengl rendering 
(like Gem). Some plans include audio buffers (like Vasp), ascii packets, 
text buffers, ... Finally there's a library that enables you to connect a 
scheme interpreter (guile) to PD/PDP. For more image processing objects,
have a look at Yves Degoyon's <a 
href="http://ydegoyon.free.fr/pidip.html">PiDiP</a> library.

<h2>Getting Started</h2>

If you're used to working with PD, the the documentation and example 
patches should be enough to get you started. Have a look at the README file 
in the distribution to find out how to compile and setup. The file 
doc/reference.txt contains a list of objects. If you have installed PDP 
properly, you can just press the right mouse button on an object and select 
help to get a help patch. If this doesn't work, look in the directory 
doc/objects for a collection of help patches. The directory doc/examples 
contains some more demos. The directory doc/objects contains two 
abstractions that are used to setup the input and output in the help 
patches. You might want to cut and connect some wires to use the 
input/output setup that works for you.

<h2>Packets and Types</h2>

PDP is centered around the concept of packets and operations on 
packets. There are several types of packets. The default type for most 
objects is <code>image/YCrCb/320x240</code>. This is a single video 
frame, encoded in the internal 16bit YUV format, measuring 320 by 240 
pixels. Another image type is the grayscale image 
<code>image/grey/320x240</code>. Important notes: All image processing objects that
combine two or more packets need to be fed with the same packet types, i.e.
encoding (YCrCb/grey) and dimensions need to be the same. Image dimensions need to be a 
multiple of <code>8x8</code>.

The 
<code>bitmap/*/*</code> type is another image representation type 
supporting several encodings. I.e. <code>bitmap/rgb/*</code>, 
<code>bitmap/rgba/*</code>, <code>bitmap/yv12/*</code>, ...

This type cannot be processed directly by most of the image processing 
objects, but it can be used to store in delay lines, or to send over the 
network. It's main use is to support all kinds of input/output devices, and 
opengl textures, without introducing too many conversions, but it can serve 
as a space and bandwidth saver too (especially 
<code>bitmap/yv12/*</code>).

One of the interesting
features in PD is the possibility of connecting everything with everything. 
If you want to generalize this to all kinds of media objects, the complexity 
of managing the different types starts to grow quite fast. Therefore PDP has 
a type conversion system that can take care of most of the conversions 
using the <code>[pdp_convert]</code> object. You can manually convert 
packets to a certain type by specifying a type template as a creation 
argument. I.e. <code>[pdp_convert image/grey/*]</code> will convert 
any packet to a greyscale image. Most of the conversion will become 
automatic later on.

An example: You can use the basic PDP library together with the 
cellular automata library and the opengl rendering library to use a cellular 
automaton as an input to a video processing chain. You can convert the 
processed image to a texture that can be applied to a 3d object, which then 
can be drawn to the screen, captured as a texture, converted back to an 
image, which can then be converted to a sound, processed and converted back 
to an image, etc... You get the point. The possibilities are endless.

    <hr>
    <address><a href="mailto:pdp at zzz.kotnet.org">Tom Schouten</a></address>
<!-- Created: Thu Apr 24 22:21:03 CEST 2003 -->
<!-- hhmts start -->
Last modified: Thu Sep 25 20:51:44 CEST 2003
<!-- hhmts end -->
  </body>
</html>

--- NEW FILE: devdoc.html ---
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
  <head>
    <title>PDP Developer Documentation</title>
  </head>

  <body>
    <h1>PDP Developer Documentation</h1>

    <h2>Introduction</h2>

    <p>There is not yet much developer information, partly because pdp is not that big and since the goals are
      not completely clear yet, a lot will probably change on the inside in the future. I believe it is
      not too hard to figure out how it works, once you get started somewhere. This document is a minimalistic 
      attempt to provide that starting point.  For full prototypes see the header files. I suggest you have a look at the pdp_base base class, and some simple
      modules: pdp_add, pdp_noise and pdp_gain for examples.

    <h2> PDP architecture </h2>
    <p> Architecture is a big word, but pdp is organized as modules. A packet pool module (a reuse pool memory manager), 
    a packet class, a processing queue module, a high level type conversion module, an image packet class, and some
    low level modules for image type conversion, image resampling and all sorts of other image processing. Besides that
    there are 2 extension libraries: pdp_scaf, a cellular automata extension and pdp_opengl, a 3d rendering extension.
    These are separate because of portability issues. The different pdp_* externs in the main pdp library use the 
    core modules' functionality to minimize code duplication. I'm relatively happy with how it fits together,
    but some things need to change for future plans. Most objects are written in the object oriented c style of pd.
    To prevent namespace conflicts, (almost) all routines start with the pdp_ prefix. The second name is the name of the
    object or module they belong to. The first argument is always a pointer to an object or an integer (for packets).

    <h2> PD ties </h2>
    <p> PDP is written as an extension for PD. One of the goals of pdp is to evolve to a separate library that can
      be reused in other software. The architecture will be split into two parts. A pd-independent part (the packet classes,
      the packet pool, the type conversion system and the forth system) and a part with pd specific stuff (the process queue and interfaces to the
      pd system like the base classes and the pd communication protocol). In order to do this the packet class will probably
      evolve to a proper object model, supporting run time attribute binding (inspired by the python object model).

    <p>There are some things that put a stamp on the current pdp design. Most importantly pd's processor object model and
      communication protocol. (i.e. the fact that pd only supports unidirectional messaging creates the awkward concept
      of a "passing packet" to eliminate excessive data copying.)

    <p> In pd, the pdp messaging protocol is implemented as pd messages. The protocol is however 3 phase. 
      With a read only register phase, a read/write register phase and a process phase. This functionality
      is part of the base class or the forth processor object. The dpd protocol is entirely different, 
      and is used in the opengl library. It is
      not based on parallel dataflow but serial context passing.

      <h2> Packets </h2>
    <p> PDP introduces a new atom: the data packet. This can contain all kinds of data. Images (16bit/8bit), cellular
      automata (1bit), matrices (real/complex float/double), opengl textures and 3d rendering contexts. Packets
      are stored in a pool to ensure fast reuse, and to enable sharing. The paradigm is centered around a
      combination of an object oriented approach and a dataflow approach. 
    <p>The methods operating on packets 
      (pdp_packet_*) are mainly for administrative purposes: memory management (construction, registering, copying)
      and getting or setting info. 
    <p>All processing is done in the pd modules. Processors can be defined using
      the forth scripting language, but this is still experimental. The forth system can be accessed
      from the guile library.
    <p> There is a central mechanism for packet type conversion. This is to facilitate the combination of different
      media types. Whenever a packet class is constructed (i.e. in an extension library), a number of conversion
      routines should be defined to convert the added type to one or some of the main pdp types.

    <h2>PDP API Overview</h2>

    The pdp public api contains only a single class: the packet. (The internal api has more classes, that can be used
    too if necessary, but i won't document them.) A packet is a class in pdp. The table below lists the supported methods. 
    The first argument of a call is a packet id. 

    <TABLE border = "1">
	<TR><TH colspan = "2">pdp_packet_*
	<TR><TD>new                <TD>construct a raw packet (depreciated)
	<TR><TD>new_*              <TD>construct packet of specific type/subtype/...
	<TR><TD>mark_unused        <TD>release
	<TR><TD>mark_passing       <TD>conditional release (release on first copy ro/rw)
	<TR><TD>copy_ro            <TD>readonly (shared) copy
	<TR><TD>copy_rw            <TD>private copy
	<TR><TD>clone_rw           <TD>private copy (copies only meta data, not the content)
	<TR><TD>header             <TD>get the raw header (t_pdp *)
	<TR><TD>data               <TD>get the raw data (void *)
	<TR><TD>pass_if_valid      <TD>send a packet to pd outlet, if it is valid, and mark unused
	<TR><TD>replace_if_valid   <TD>delete packet and replace with new one, if new is valid
	<TR><TD>copy_ro_or_drop    <TD>copy readonly, or don't copy if dest slot is full + send drop notify
	<TR><TD>copy_rw_or_drop    <TD>same, but private copy
	<TR><TD>get_description    <TD>retrieve type info
	<TR><TD>convert_ro         <TD>same as copy_ro, but with an automatic conversion matching a type template
	<TR><TD>convert_rw         <TD>same as convert_ro, but producing a private copy
    </TABLE>

    <p>The pool object methods. All the packets are stored in a central packet pool.

    <TABLE border = "1">
	<TR><TH colspan = "2">pdp_pool_*
	<TR><TD>collect_garbage    <TD>manually free all unused resources in packet pool
    </TABLE>

    <p>The process queue object methods. PDP supports a separate processing thread.

    <TABLE border = "1">
	<TR><TH colspan = "2"> pdp_queue_*
	<TR><TD>add                <TD>add a process method + callback
	<TR><TD>finish             <TD>wait until a specific task is done
	<TR><TD>wait               <TD>wait until processing queue is done
    </TABLE>

    <p>The control methods. General pdp control messages.

    <TABLE border = "1">
	<TR><TH colspan = "2"> pdp_control_*
	<TR><TD>notify_drop        <TD>notify that a packet has been dropped
    </TABLE>

    <p> The type mediator methods.
    <TABLE border = "1">
      <TR><TH colspan = "2"> pdp_type_* 
	<TR><TD>description_match   <TD>check if two type templates match
	<TR><TD>register_conversion <TD>register a type conversion program

</TABLE>

   <p>NOTE: it is advised to derive your module from the pdp base class defined in pdp_base.h
         instead of communicating directly with the pdp core

    <h2>pdp_base class</h2>
    If you want to write a pdp extern, you can derive it from the pdp_base class, instead of t_object.
    This class abstracts a lot of the hassle of writing ordinary (inplace) packet processors. The base
    allows you to register process callbacks. There are 3 kinds of callbacks: preproc, process and postproc.
    The preproc method is called inside the pd thread. This can be used to setup some things that can only
    be done inside the pd thread. The process method should do most of the work, and is called from the
    pdp processing thread if it is enabled, after the preproc method is finished. You can't use most
    of pd's calls in this method. The postproc method is called
    from the pd thread after the process method is finished, and can be used to send data to pd outlets. Simple
    packet processors only need the process method (packet input/output is handled by the pdp_base class).

    <h2>pdp_imageproc_* modules</h2>
    Most of the image processing code is organized as planar 16 bit signed processors. 
    This is crude and oversimplified, but it helps to keep the code size small and fast
    at the same time (platform dependent assembly code is reduced to a bare minimum). These
    routines can be used to build higher level image processing objects that are more (cache)
    efficient than an abstraction using separate pdp modules. If you plan to write your own image
    processing routines, you can use the pdp_imageproc_dispatch_ routine to support all 16bit image
    types at once (greyscale, subsampled YCrCb, multichannel planar). This requires you write the
    image processing routine as a planar (greyscale) processor using the pdp_imageproc_
    interface. (see pdp_imageproc.h)

    <h2>pdp_llconv call</h2>
    Low level image conversion routines. (operating on raw data buffers). You probably won't need this,
    since the high level type conversion (pdp_packet_convert_ro/rw) covers most of its functionality.

    <hr>
    <address><a href="mailto:pdp at zzz.kotnet.org">Tom Schouten</a></address>
<!-- Created: Mon Apr 28 15:35:12 CEST 2003 -->
<!-- hhmts start -->
Last modified: Fri Sep 19 04:52:12 CEST 2003
<!-- hhmts end -->
  </body>
</html>

--- NEW FILE: layers.txt ---
pdp 0.13 design layers + components
-----------------------------------

from version 0.13 onwards, pdp is no longer just a pd plugin but a 
standalone unix library (libpdp). this documents is an attempt to 
describe the design layers.

A. PD INTERFACE
---------------

on the top level, libpdp is interfaced to pd using a glue layer which 
consists of

1. pdp/dpd protocols for pd
2. process queue
3. base classes for pdp/dpd
4. some small utility pd objects
5. pd specific interfaces to part of pdp core
6. pdp_console
7. pd object interface to packet forth (pdp object)

1. is the same as previous versions to ensure backwards compatibility in 
pd with previous pdp modules and extensions that are written as pd 
externs or external libs. this includes parts of pdp that are not yet 
migrated to libpdp (some of them are very pd specific and will not be 
moved to libpdp), and pidip. if you intend to write new modules, it is 
encouraged to use the new forth based api, so your code can be part of 
libpdp to use it in other image processing applications.

2. is considered a pd part. it implements multithreading of pdp inside 
pd. multithreading is considered a host interface part, since it usually  
requires special code.

3. the base classes (pd objects) for pdp image processing remain part of 
the pd<->pdp layer. the reason is the same as 1. a lot of the original 
pd style pdp is written as subclasses of the pdp_base, pdp_image_base, 
dpd_base and pdp_3dp_base classes. if you need to write pd specific 
code, it is still encouraged to use these apis, since they eliminate a 
lot of red tape involving the pdp protocol. a disadvantage is that this 
api is badly documented, and the basic api (1.) is a lot simpler to 
learn and documented. 3dp is supposed to merge to the new forth api, 
along with the image/video processing code.

4. is small enough to be ignored here

5. includes interfaces to thread system and type conversion system + 
some pd specific stuff using 1. or 3.

6. the console interface to the pdp core, basicly a console for a 
forth like language called packet forth which is pdp's main scripting 
language. it's inteded for develloping and testing pdp but it can be 
used to write controllers for pd/pdp/... too. this is based on 1.

7. is the main link between the new libpdp and pd. it is used to 
instantiate pdp processors in pd which are written in the packet forth. 
i.e. to create a mixer, you instantiate a [pdp mix] object, which would 
be the same as the previous [pdp_mix] object. each [pdp] object creates 
a forth environment, which is initialized by calling a forth init 
method. [pdp mix] would call the forth word init_mix to create the local 
environment for the mix object. wrappers will be included for backward 
compatibility when the image processing code is moved to libpdp.

B. PDP SYSTEM CODE
------------------

1. basic building blocks: symbol, list, memory manager
2. packet implementation (packet class and reuse queue)
3. packet type conversion system
4. os interface (X11, net, png, ...)
5. packet forth
6. additional libraries

1. pdp makes intensive use of symbols and lists (trees, stacks, queues). 
pdp's namespace is built on the symbol implementation. a lot of other 
code uses the list

2. the pdp packet model is very simple. basicly nothing more than 
constructors (new, copy, clone), destructors (mark_unused (for reuse 
later), delete). there is no real object model for processors. this is a 
conscious decision. processor objects are implemented as packet forth 
processes with object state stored in process data space. this is enough 
to interface the functionality present in packet forth code to any 
arbitrary object oriented language or system.

3. each packet type can register conversion methods to other types. the 
type conversion system does the casting. this is not completely finished 
yet (no automatic multistage casting yet) but most of it is in place and 
usable. no types without casts.

4. os specific modules for input/output. not much fun here..

5. All of pdp is glued together with a scripting language called packet 
forth. This is a "high level" forth dialect that can operate on floats, 
ints, symbols, lists, trees and packets. It is a "fool proof" forth, 
which is polymorphic and relatively robust to user errors (read: it 
should not crash or cause memory leaks when experimenting). It is 
intended to serve as a packet level glue language, so it is not very 
efficient for scalar code. This is usually not a problem, since packet 
operations (esp. image processing) are much more expensive than a this 
thin layer of glue connecting them.

All packet operations can be accessed in the forth. If you've ever 
worked with HP's RPN calculators, you can use packet forth. The basic 
idea is to write objects in packet forth that can be used in pd or in 
other image processing applications. For more information on packet 
forth, see the code (pdp_forth.h, pdp_forth.c and words.def)

6. opengl lib, based on dpd (3.) which will be moved to packet forth 
words and the cellular automata lib, which will be moved to 
vector/slice forth later.

C. LOW LEVEL CODE
-----------------

All the packet processing code is (will be) exported as packet forth 
words. This section is about how the code exported by those words is 
structured.

C.1 IMAGE PROESSING: VECTOR FORTH

Eventually, image operations need to be implemented, and in order 
to do this efficiently, both codewize (good modularity) as execution speed 
wize, i've opted for another forth. DSP and forth seem to mix well, once 
you get the risc pipeline issues out of the way. And, as a less rational 
explanation, forth has this magic kind of feel, something like art.. 
well, whatever :)

As opposed to the packet forth, this is a "real" lowlevel forth 
optimized for performance. Its reason of being is the solution of 3 
problems: image processing code factoring, quasi optimal processor 
pipeline & instruction usage, and locality of reference for maximum 
cache performance. Expect crashes when you start experimenting with 
this. It's really nothing more than a fancy macro assembler. It has no 
safety belts. Chuck Moore doctrine.. 

The distinction between the two forths is at first sight not a good 
example of minimizing glue layers. I.e. both systems (packet script 
forth and low level slice forth) are forths in essence, requiring 
(partial) design duplication. Both implementations are however 
significantly different which justified this design duplication.

Apart from the implementation differences, the purpose of both languages 
is not the same. This requires the designs of both languages to be 
different in some respect. So, with the rule of "do everything right 
once" in mind, this small remark tries to justify the fact that forth != 
forth.

The only place where apparent design correspondence (the language model)
is actually used is in the interface between the packet forth and the 
slice forth. 

The base forth is an ordinary minimal 32bit (or machine word 
lenght) subroutine threaded forth, with a native code kernel for 
i386/mmx, a portable c code kernel and room for more native code kernels 
(i.e i386/sse/sse2/3dnow, altivec, dsp, ...) Besides support for native 
machine words bit ints and pointers, no floats, since they clash with 
mmx, are not needed for the fixed point image type, and can be 
implemented using other vector instructions when needed), support for 
slices and a separate "vector stack". 

Vectors are the native machine vectors, i.e. 64bit for mmx/3dnow, 
128bit for sse/sse2, or anything else. The architecture is supposed to 
be open. (I've been thinking to add a block forth, operating on 256bit 
blocks to eliminate pipeline issues). Blocks are just blocks of vectors 
which are used as a basic loop unrolling data size grain for solving 
pipeline operations in slice processing words.

Slices are just arrays of blocks. In the mmx forth kernel, they can 
represent 4 scanlines or a 4 colour component scanline, depending on how 
they are fed from packet data. Slices can be anything, but right now, 
they're just scanlines. The forth kernel has a very simple and efficient
(branchless) reference count based memory allocator for slices. This 
slice allocator is also stack based which ensures locality of reference: 
a new allocated slice is the last deallocated slice.

The reason for this obsession with slices is that a lot of video 
effects can be chained on the slice level (scanline or bunch of 
scanlines), which improves speed through more locality of reference. In 
concreto intermediates are not flushed to slower memory. The same 
principles can be used to do audio dsp, but that's for later.

The mmx forth kernel is further factored down following another 
virtual machine paradigm. After doing some profiling, i came to the 
conclusion that the only, single paradigm way of writing efficient 
vector code on today's machines is multiple accumulators to avoid 
pipeline stalls. The nice thing about image processing is that it 
parallellizes easily. Hence the slice/block thing. This leads to the 
1-operand virtual machine concept for the mmx slice operations. The 
basic data size is one 4x4 pixel block (16bit pixels), which is 
implemented as asm macros in mmx-sliceops-macro.s and used in 
mmx-sliceops-code.s to build slice operations. The slice operations are 
built out of macro instructions for this 256bit or 512bit, 2 or 1 
register virtual machine which has practically no pipeline delays 
between its instructions.

Since the base of sliceforth is just another forth, it could be that 
(part of) 3dp will be implemented in this lowlevel forth too, if 
performance dictates it. It's probably simpler to do it in the lowlevel 
forth than the packet forth anyway, in the form of cwords.

C.2: MATRIX PROCESSING: LIBGSL

All matrix processing packet forth words are (will be) basicly wrappers 
around gsl library calls. Very straightforward.

C.3: OPENGL STUFF

The same goes for opengl. The difference here is that it uses the dpd 
protocol in pdp, which is more like the Gem way of doing things. The 
reason for this is that, although i've tried hard to avoid it, opengl 
seems to dictate a drawing context based, instead of an object based way 
of working. So processing is context (accumulator) based. Packet forth 
will probably get some object oriented, or context oriented feel when 
this is implemented.