Re: YAJL Parser... getting a pointer to the data and length of the data?

Cool.. Thanks, Scott. I hope to get to test these soon. They look like
what I was going to do but the message with the warning had me a little
worried. :)

Just one issue with the 2nd one.. will the %str() function limit the max
size of the data to the max string size for the OS? In researching this I
found this was an issue with the YAJL function that wrote the data to a
stream file. On V5Rx it was limited to 64k even if the data was larger.

Here's the dialog I was referring to:
http://www.code400.com/forum/showthread.php/13803-Large-string-in-YAJL

Brad
www.bvstools.com

On Wed, Apr 13, 2016 at 10:15 AM, Scott Klement <rpg400-l@xxxxxxxxxxxxxxxx>
wrote:

Hi Brad,

You're referring to getting the value of a string node, right? (Not the
underlying buffer for the whole document.)

Each yajl_val (node pointer) is a pointer to a data structure that
contains the data type, and has a 'union' (overlaid fields) for the raw
underlying data, with a separate field for each data type.

I would not recommend dereferencing the pointer in your own program
(because if the structure ever changes, your programs will fail) but
instead, I would add a new routine to YAJLR4 that gets this data.

Are you looking for the raw UTF-8 string? OR do you want it translated to
EBCDIC for you? Getting the raw UTF-8 would be very easy.

This is off the top of my head, untested:


*+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
* YAJL_GET_STRING_UTF8(): Retrieve a pointer to the raw UTF-8
* data for a YAJL string
*
* node = (input) YAJL tree node
*
* returns a pointer to a C-style, zero-terminated string
* or *NULL if the node is not a string
*+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
P YAJL_GET_STRING_UTF8...
P B export
D PI *
D node like(yajl_val) value

D nv ds likeds(yajl_val_t)
D based(node)

/free
if YAJL_IS_STRING(node);
return nv.string;
else;
return *null;
endif;
/end-free
P E


If you want the data to be translated to EBCDIC, that is more complicated
because the length of the data can vary so much, you don't necessarily know
up front how big the data will be. You could dynamically allocate the
memory, but then the caller would have to fuss with making sure it's freed
up, which I prefer to avoid.

So what I would do is let the caller provide the buffer. Instead of
returning a pointer, just put data into the caller's buffer. Then if it
wants to use dynamic allocation, it can... but it could also just use a
large(-er) variable or user space or whatever makes sense.

Again, I have not tested this, I'll leave that to you, but the code would
look something like this:


*+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
* YAJL_GET_STRING_BUF(): Convert JSON string to EBCDIC and place
* it into a buffer
*
* node = (input) YAJL tree node
* buf = (input) pointer to buffer to load
* bufsize = (input) size of buffer to load
*
* returns *ON if successful, *OFF otherwise
*+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
P YAJL_GET_STRING_BUF...
P B export
D PI 1n
D node like(yajl_val) value
D buf * value
D bufSize 10u 0 value

D memcpy pr * extproc('memcpy')
D dst * value
D src * value
D len 10u 0 value

D nv ds likeds(yajl_val_t)
D based(node)
D str s *
D len s 10u 0
/free
if YAJL_IS_STRING(node) = *OFF;
return *OFF;
endif;

init_gconv();

str = yajl_iconv_toLocal( gConv: nv.string );

len = %len(%str(str));
if len > bufSize;
len = bufSize;
endif;
memcpy(buf: str: len);

yajl_iconv_string_free(str);

return *ON;
/end-free
P E

Let me know how that works out for you. If these work nicely, I'll add
them to my YAJL distro.

-SK




On 4/12/2016 7:38 PM, Bradley Stone wrote:

I was working on a project the other day where the JSON contains a large
amount of data.

I had previously used my own JSON parser that I wrote a few years ago. I
have one function that returns a pointer to the JSON data as well as the
length of that data. It can be any length, which is nice (no 64k limit).

So I can do something like:

rc = #json_setValue('tag':'fileData); // the tag to retrieve data for
rc = #json_getData(data@);

rc will contain the length of the data, and data@ will be a pointer to
it.

From there if it's base64 I can decode it and then write it all to a
stream
file or do whatever I want.

I want to use YAJL for this now because it SO much faster than my home
grown parser. But I didn't see such a beast in the docs.

I did see, in the source, a reference to something that looks like what
I'm
looking for, but there was a stern warning there as well saying "not
recommended for RPG". :)

Just wondering if there's anything similar before I attempt to "roll my
own" and modify the YAJL parser.

Thanks!

Brad
www.bvstools.com

--

This is the RPG programming on the IBM i (AS/400 and iSeries) (RPG400-L)
mailing list
To post a message email: RPG400-L@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/rpg400-l
or email: RPG400-L-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/rpg400-l.

Please contact support@xxxxxxxxxxxx for any subscription related
questions.