Re: YAJL Parser... getting a pointer to the data and length of the data?

Brad,

Apologies for the confusion, I misread your code. I thought you were doing strlen() on the string passed from the RPG program, but instead, you're doing strlen() on the JSON value. If anything, that's even worse!!

yajl_get_string_buf() returns a string from a JSON document. In order to do that with pointers, it needs to know how much data (at maximum) it can copy into the resulting memory.

Your version does this:

str = yajl_iconv_toLocal( gConv: nv.string );
len = strlen(str);
memcpy(buf:str:len);
yajl_iconv_string_free(str);

So let's take the example you gave {"name": "Brad"} as a trivial example. The caller would do something like this, right?

dcl-s myName char(4);

node = yajl_object_find(docNode: 'name');
len = yajl_get_string_buf(node: %addr(myName));

So that works as long as the string ("Brad" in this example) is 4 characters or less. what if the JSON document looked like this?

{"name": "Darpankumar"}

Now your len = strlen(str) returns 11 instead of 4. When it does memcpy() to put the value into 'buf', it copies 11 bytes. Only problem is, buf points to myName which is CHAR(4). Ooops! You just overwrote 7 bytes of memory that didn't belong to you!

Of course, when YAJLR4 is built, it has no way to know that someone will come along and pass a shorter buffer than the length of the data in the string.

My proposal (which has already been coded on my box) is to pass an additional parameter to yajl_get_string_buf() that contains the length of the 'buf' parameter. That makes the prototype look like this:

P YAJL_GET_STRING_BUF...
P B export
D PI 10u 0
D node like(yajl_val) value
D buf * value
D bufSize 10u 0 value

With this added, the caller will provide the size of the buffer in the 3rd parameter, like this:

dcl-s myName char(4);

node = yajl_object_find(docNode: 'name');
len = yajl_get_string_buf(node: %addr(myName): %size(myName));

Inside yajl_get_string_buf(), it now has bufSize that contains the maximum size of the buf parameter. So it won't overwrite memory, it'll cap the size of the result to that length.

len = strlen(str);
if len > bufSize;
len = bufSize;
endif;
memcpy(buf: str: len);

See what I mean? Now when the value is "Darpankumar" it will only copy "Darp" instead of overwriting extra memory.

Knowing this length also makes it possible to do something like this to blank out the buffer at the start, which might be useful (or wasteful?) too:

D memset pr * extproc('memset')
D dst * value
D char 10i 0 value
D size 10u 0 value

D char ds qualified
D val 3u 0

char = *blanks;
memset(buf: char.val: bufSize);

That way your traditional RPGer who expects string operations to always pad the result with blanks will be a bit happier?

Sound good?

-SK


On 5/23/2016 7:42 AM, Bradley Stone wrote:

Scott,

I think I see what you're saying.

I'm returning the length of the data, not the length of the actual buffer.

Here's what I mean..

{"name" : "Brad"}

This would return 4, correct? What would the buffer size be in this case?

For my needs I want the length of the data and the pointer to that said
data (and it seems to be working great for all lengths of data).

So how would a buffer length differ from the data or string length? Is it
the term "buffer" that is actually incorrect in this case then?

Thanks for looking into it!

Brad
www.bvstools.com

On Sun, May 22, 2016 at 6:11 PM, Scott Klement <rpg400-l@xxxxxxxxxxxxxxxx>
wrote:

Hi Brad,

So it looks like you changed yajl_get_string_buf() to calculate the buffer
size based on looking for a zero-terminator in 'buf'. This is not,
normally, now C strings work, though... Normally the x'00' terminator is
the end of the string, but the buffer size may be larger. For example, I
could declare a string as char data[256], and put just "hello" in the
string. The x'00' would be found after 5 characters, buf the buffer is
still 256 characters long. That is why my approach passed the buffer size
as a parameter.

I'm thinking the best approach is to create a hybrid of your solution and
mine... basically, use my solution, but return a length instead of just
on/off.

Thoughts?

-SK


On 5/14/2016 9:07 AM, Bradley Stone wrote:

Just for the record I did end up adding a couple subprocedures to YAJL. I
only ended up using the yajl_get_string_buf (with some slight
modifications
from your suggestion) though, not the UTF8 one. Although the
yajl_get_string_utf8 does seem to work as well.

Here they are if you want to add them to your distro:

http://www.fieldexit.com/forum/display?threadid=321

I put the changes in an article so the formatting would be better. If you
want me to email you the changed source files just let me know and I'd be
happy to do so.

Brad
www.bvstools.com

--

This is the RPG programming on the IBM i (AS/400 and iSeries) (RPG400-L)
mailing list
To post a message email: RPG400-L@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/rpg400-l
or email: RPG400-L-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/rpg400-l.

Please contact support@xxxxxxxxxxxx for any subscription related
questions.