Re: YAJL Parser... getting a pointer to the data and length of the data? -- RPG400-L

Scott,

I see what you're saying.

In my case it's working because I'm not processing the data unless it's
under the MAX Size allowed for the application.

But if someone were to pass in a reference variable then you're right, it
would be trouble if the variable wasn't large enough to handle the size of
the buffer. sort of like using Varying options on RPG procedures
incorrectly.

I like your changes though and they do make sense. I will implement them
unless you have your changes already available online for download.

One other thing that I am finding would be useful is the opposite. I want
to be able to when creating JSON pass in a pointer and a length of data for
it to add to the tree. Similar to the one that writes a node value from a
stream file, but instead pass in a pointer and length.

This makes it easier to do something like:

1. Read data from a stream file to a pointer (not variable based)
2. Convert the data to base64
3. Write the data to the JSON tree (using a pointer and length since we
should have that from previous steps)

Right I am doing the following:

1. Read data from a stream file
2. Convert data to base64
3. Write data to a new stream file
4. Write the stream file data to the JSON tree

Make sense? I was planning on digging in as it can't be very difficult.
Just trying to find the time. :)

Brad
www.bvstools.com

On Wed, May 25, 2016 at 4:49 PM, Scott Klement <rpg400-l@xxxxxxxxxxxxxxxx>
wrote:

Brad,

Apologies for the confusion, I misread your code. I thought you were doing
strlen() on the string passed from the RPG program, but instead, you're
doing strlen() on the JSON value. If anything, that's even worse!!

yajl_get_string_buf() returns a string from a JSON document. In order to
do that with pointers, it needs to know how much data (at maximum) it can
copy into the resulting memory.

Your version does this:

str = yajl_iconv_toLocal( gConv: nv.string );
len = strlen(str);
memcpy(buf:str:len);
yajl_iconv_string_free(str);

So let's take the example you gave {"name": "Brad"} as a trivial example.
The caller would do something like this, right?

dcl-s myName char(4);

node = yajl_object_find(docNode: 'name');
len = yajl_get_string_buf(node: %addr(myName));

So that works as long as the string ("Brad" in this example) is 4
characters or less. what if the JSON document looked like this?

{"name": "Darpankumar"}

Now your len = strlen(str) returns 11 instead of 4. When it does memcpy()
to put the value into 'buf', it copies 11 bytes. Only problem is, buf
points to myName which is CHAR(4). Ooops! You just overwrote 7 bytes of
memory that didn't belong to you!

Of course, when YAJLR4 is built, it has no way to know that someone will
come along and pass a shorter buffer than the length of the data in the
string.

My proposal (which has already been coded on my box) is to pass an
additional parameter to yajl_get_string_buf() that contains the length of
the 'buf' parameter. That makes the prototype look like this:

P YAJL_GET_STRING_BUF...
P B export
D PI 10u 0
D node like(yajl_val) value
D buf * value
D bufSize 10u 0 value

With this added, the caller will provide the size of the buffer in the 3rd
parameter, like this:

dcl-s myName char(4);

node = yajl_object_find(docNode: 'name');
len = yajl_get_string_buf(node: %addr(myName): %size(myName));

Inside yajl_get_string_buf(), it now has bufSize that contains the maximum
size of the buf parameter. So it won't overwrite memory, it'll cap the size
of the result to that length.

len = strlen(str);
if len > bufSize;
len = bufSize;
endif;
memcpy(buf: str: len);

See what I mean? Now when the value is "Darpankumar" it will only copy
"Darp" instead of overwriting extra memory.

Knowing this length also makes it possible to do something like this to
blank out the buffer at the start, which might be useful (or wasteful?) too:

D memset pr * extproc('memset')
D dst * value
D char 10i 0 value
D size 10u 0 value

D char ds qualified
D val 3u 0

char = *blanks;
memset(buf: char.val: bufSize);

That way your traditional RPGer who expects string operations to always
pad the result with blanks will be a bit happier?

Sound good?

-SK

On 5/23/2016 7:42 AM, Bradley Stone wrote:

Scott,

I think I see what you're saying.

I'm returning the length of the data, not the length of the actual buffer.

Here's what I mean..

{"name" : "Brad"}

This would return 4, correct? What would the buffer size be in this case?

For my needs I want the length of the data and the pointer to that said
data (and it seems to be working great for all lengths of data).

So how would a buffer length differ from the data or string length? Is it
the term "buffer" that is actually incorrect in this case then?

Thanks for looking into it!

Brad
www.bvstools.com

On Sun, May 22, 2016 at 6:11 PM, Scott Klement <rpg400-l@xxxxxxxxxxxxxxxx

wrote:

Hi Brad,

So it looks like you changed yajl_get_string_buf() to calculate the
buffer
size based on looking for a zero-terminator in 'buf'. This is not,
normally, now C strings work, though... Normally the x'00' terminator
is
the end of the string, but the buffer size may be larger. For example, I
could declare a string as char data[256], and put just "hello" in the
string. The x'00' would be found after 5 characters, buf the buffer is
still 256 characters long. That is why my approach passed the buffer
size
as a parameter.

I'm thinking the best approach is to create a hybrid of your solution and
mine... basically, use my solution, but return a length instead of just
on/off.

Thoughts?

-SK

On 5/14/2016 9:07 AM, Bradley Stone wrote:

Just for the record I did end up adding a couple subprocedures to YAJL.

I
only ended up using the yajl_get_string_buf (with some slight
modifications
from your suggestion) though, not the UTF8 one. Although the
yajl_get_string_utf8 does seem to work as well.

Here they are if you want to add them to your distro:

http://www.fieldexit.com/forum/display?threadid=321

I put the changes in an article so the formatting would be better. If
you
want me to email you the changed source files just let me know and I'd
be
happy to do so.

Brad
www.bvstools.com

--

This is the RPG programming on the IBM i (AS/400 and iSeries) (RPG400-L)
mailing list
To post a message email: RPG400-L@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/rpg400-l
or email: RPG400-L-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/rpg400-l.

Please contact support@xxxxxxxxxxxx for any subscription related
questions.

--

This is the RPG programming on the IBM i (AS/400 and iSeries) (RPG400-L)
mailing list
To post a message email: RPG400-L@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/rpg400-l
or email: RPG400-L-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/rpg400-l.

Please contact support@xxxxxxxxxxxx for any subscription related
questions.