[PD-dev] CUDA discussion

[Charles Henry]

Dear list,

I'd like to start a conversation about CUDA and Pd.

For those of you who don't know, CUDA is a minimal instruction set for
doing single precision floating point calculations on NVIDIA GPUs.
It's a C-based coding paradigm in which blocks of data are copied to
GPU device memory and operations are performed on that data with
thread blocks in increments of 32 threads.  Complete sets of floating
point math functions are available for CUDA.  The CUDA compiler nvcc
works very well alongside gcc.

I've been studying it at work, but have not coded anything for Pd yet.
 There's a whole lot of performance issues based on tiny details in
the documentation--the implementation of cuda based externals could be
made fairly simple for developers if a complete set of CUDA<->Pd
extensions could be coded from the beginning.

Any project worth doing is worth doing right.  So, I want to figure
out if:  a) it's worth doing and b) how to do it right.

I've got a first draft of top-down design issues, and I'd like to make
a list of incremental milestones that would prove the concept is
sound.

top-down design issues:
1.  The essential CUDA<->Pd functions should be made separate from
CUDA based Pd externals, with a separate header file, and compilable
to shared and static libraries.
2.  The set of CUDA<->Pd extensions needs to be able to manage
multiple devices, including device query, initialization and setting
global parameter sets per GPU.  Most likely, this means a custom data
structure and object based method system.
3.  Compilation--how to create the build system and handle
dependencies for a library of CUDA based externals.  Management of
CUDA libraries, CUDA-rt and CUDA-BLAS especially.
4.  Testing and initialization.  At setup time, a CUDA based external
should be able to find out if it is legal and ready to run.
5.  Abstraction of major device and memory operations.  What makes up
a sufficient and complete set of operations?  This is a list that is
most likely to be grown through experimentation, but a good
preliminary list of operations will help get things started on the
right footing.
6.  Performance.  How to profile or benchmark and make comparisons
between implementations?  The single greatest performance issue that I
have identified is caching on GPU.  host<->device memory transfers can
be eliminated in some cases, allowing CUDA based externals to follow
one another in the DSP tree with faster scheduling and runtime
performance.

(proposed) incremental milestones:
1.  Create an external that checks GPUs and hands back error messages to Pd.
2.  Create an external that initializes GPUs.
3.  Create an external that performs host<->device memory transfer and
runs an operation.
4.  Create an external that performs an operation and compares the
time it takes against the same operation on CPU.  At this point, it
should be possible to identify and hopefully quantify the potential
speedup on GPU, and decide whether or not it is worth it.

That's enough for now.  I'd like to know if anyone else has been
thinking along similar lines (CUDA has been out for, like, 2 years or
so now, so I bet that many people know about it), and if you have any
input on the design issues.

Chuck