[PD] Windows and Debian packages for PuREST JSON

Mon Jan 30 23:38:40 CET 2012

I was just thinking about the data format and how to represent the nested lists of json.  I guess those are two different problems.

First thing, about the raw json data coming out of [rest-json] and [json-encode], I think those should be lists of byte values of the characters, like the pdstring library does (i.e. [moocow/bytes2any], etc.).  Then Pd will never hang up on any characters, like {} and it won't generate symbols for the big blocks of data, and the subchunks.  Then [json-decode] would accept lists of byte values.

Using byte values also means that the raw json text could be parsed and broken into lists by Pd, using the byte values of {}[], etc.  Even better would be if we came up with some kind of format that made sense in terms of Pd lists.  Here are some ideas:

* the key side of the "key":"value" pair gets turned into a symbol and is used as the selector of a message, i.e. [key value( or [name Thomas Mayer(.

* the "value" side of the pair gets converted into the Pd type that makes the most sense, i.e:
   :"Thomas Mayer" --> list of 2 symbols
   :"residuum"     --> single symbol
   :1239           --> single float

* [ ] used to bracket nested lists, like in JSON (all 1 list):
 author [  [ name Thomas Mayer ] [ login residuum ] [ email thomas at residuum.org ]

* [nestlevel] object that breaks out nested lists based on level of nesting

Just some ideas to get the discussion going.  I attached a simple sketch of json parsing by bytes I did a while back for a simple example.

.hc
-------------- next part --------------
A non-text attachment was scrubbed...
Name: json-test.pd
Type: application/octet-stream
Size: 771 bytes
Desc: not available
URL: <http://lists.puredata.info/pipermail/pd-list/attachments/20120130/7da0bcdb/attachment.obj>
-------------- next part --------------

On Jan 30, 2012, at 4:38 PM, Hans-Christoph Steiner wrote:

> 
> About strange characters, Pd should handle those fine.  The ones that Pd has trouble with are the ones with meaning to Pd: , ; \ and space.  $ with a number after it will be interpretted by Pd as a dollar arg, but $ASD will just be a symbol.  Then there are the Tcl braces {} that we haven't gotten properly escaped so they are currently not usable. I think that's it.  Now that Pd is UTF-8 internally, it really should handle any character.
> 
> So that is , ; \ { } space and sometimes $.  {}; are used by the json format, so people likely already avoid them in their json data.
> 
> .hc
> 
> On Jan 30, 2012, at 4:11 PM, m.e.grimm wrote:
> 
>> great! thanks for all the work... good to have this in pd for data
>> visualization, etc...
>> 
>> anyway, a quick test patch is attached. the third message doesn't
>> work. might you guys know why? is it the "&" in the request? request
>> looks right...
>> 
>> which brings up the question. how to handle complicated requests with
>> strange characters in message boxes....
>> 
>>> For anyone wanting to build it themselves on Mac OS X, I updated the Fink package.  It is now called 'json-c'
>>> since there were name conflicts for 'libjson'.  It should also be in 10.7 now.
>> 
>> yeah i dont know why but its still not in 10.7.
>> 
>> Thanks for the builds though!!! work good!
>> 
>> m
>> 
>> 
>> 
>> On Mon, Jan 30, 2012 at 3:29 PM, Hans-Christoph Steiner <hans at at.or.at> wrote:
>>> 
>>> On Jan 30, 2012, at 2:20 PM, Thomas Mayer wrote:
>>> 
>>>> Hi,
>>>> 
>>>> 
>>>> On 30.01.2012 14:52, m.e.grimm wrote:
>>>>> great! Thanks!
>>>>> 
>>>>> works! ... i just get a pd crash when message
>>>>> "http://search.twitter.com/search.json?q=%23puredata" is sent to
>>>>> [rest-json]
>>>> 
>>>> Oops, that was a bug that I reintroduced during some code cleanup. I have fixed it in git with commit cee81ff:
>>>> https://github.com/residuum/PuRestJson/commit/cee81ff94d044caec1c722cb47b54bdd416ef592
>>>> 
>>>> Feel free to post any bugs at https://github.com/residuum/PuRestJson/issues
>>>> 
>>>>> i will be nice when the oauth is implemented. i was just looking at
>>>>> this recently to send twitter updates... any idea when this might
>>>>> work?
>>>> 
>>>> Right now, I am merging the helpful patches from Hans-Christoph Steiner and writing compilation docs in the Github wiki, cleaning out edges etc. When that is done, I will tag it 0.6, and make some binary packages.
>>>> 
>>>> After that, I will start with OAuth. The problem with OAuth is not so much a programming problem, but more of a design problem. What should be done in the Pd object, what is the responsibility of the user, how to deal with using one object to get data from different servers, how is OAuth really implemented at each provider, etc.
>>>> 
>>>> I will start off with some ideas and then post some stuff in the Github wiki.
>>> 
>>> I updated my Mac OS X 32-bit Intel i386 build:
>>> 
>>> http://autobuild.puredata.info/pdlab/purest_json.zip
>>> 
>>> I also posted some bug reports.  If you want ssh access to the PdLab mac machines, please post a request to pd-dev which includes your ssh key, and I can add you.
>>> 
>>> For anyone wanting to build it themselves on Mac OS X, I updated the Fink package.  It is now called 'json-c' since there were name conflicts for 'libjson'.  It should also be in 10.7 now.
>>> 
>>>>> great job... really cool stuff...
>>> 
>>> I agree :-D
>>> 
>>> 
>>> .hc
>>> 
>>> ----------------------------------------------------------------------------
>>> 
>>> Mistrust authority - promote decentralization.  - the hacker ethic
>>> 
>>> 
>> 
>> 
>> 
>> -- 
>> ____________________
>> m.e.grimm | m.f.a | ed.m.
>> megrimm at gmail.com
>> _________________________________
>> <nyt-json-test.pd>
> 
> 
> 
> ----------------------------------------------------------------------------
> 
> There is no way to peace, peace is the way.       -A.J. Muste
> 
> 

----------------------------------------------------------------------------

"It is convenient to imagine a power beyond us because that means we don't have to examine our own lives.", from "The Idols of Environmentalism", by Curtis White