× The internal search function is temporarily non-functional. The current search engine is no longer viable and we are researching alternatives.
As a stop gap measure, we are using Google's custom search engine service.
If you know of an easy to use, open source, search engine ... please contact support@midrange.com.



Scott,

Well, in the real world I support clients on many levels.

We are discontinuing support for V5R4 soon for our canned software (but not
our consulting which this applies to more) but even in V6R1 there is a
limit (16m, right?)

So it's not the V5R4 limit of 64k I'm worried about.. it's the limit
itself.

This application sends actual files in JSON. These files can be over 16m.

But, I have a feeling from the info you provided already I can figure that
out. If the string is null terminated, and I have a pointer to the
beginning, I should be able to find the length.

The data in this case is Base64 encoded so I really don't need it converted
to EBCDIC either. I just want to decode it and dump it into a file. :)

Brad
www.bvstools.com

On Thu, Apr 14, 2016 at 2:43 PM, Scott Klement <rpg400-l@xxxxxxxxxxxxxxxx>
wrote:

Brad,

That would only be a problem if you're compiling the code to V5R4 or
older. Since even 6.1 is no longer supported, I don't feel that there's
any point in supporting V5R4, do you?

-SK





On 4/13/2016 11:12 AM, Bradley Stone wrote:

Cool.. Thanks, Scott. I hope to get to test these soon. They look like
what I was going to do but the message with the warning had me a little
worried. :)

Just one issue with the 2nd one.. will the %str() function limit the max
size of the data to the max string size for the OS? In researching this I
found this was an issue with the YAJL function that wrote the data to a
stream file. On V5Rx it was limited to 64k even if the data was larger.

Here's the dialog I was referring to:
http://www.code400.com/forum/showthread.php/13803-Large-string-in-YAJL

Brad
www.bvstools.com

On Wed, Apr 13, 2016 at 10:15 AM, Scott Klement <
rpg400-l@xxxxxxxxxxxxxxxx>
wrote:

Hi Brad,

You're referring to getting the value of a string node, right? (Not the
underlying buffer for the whole document.)

Each yajl_val (node pointer) is a pointer to a data structure that
contains the data type, and has a 'union' (overlaid fields) for the raw
underlying data, with a separate field for each data type.

I would not recommend dereferencing the pointer in your own program
(because if the structure ever changes, your programs will fail) but
instead, I would add a new routine to YAJLR4 that gets this data.

Are you looking for the raw UTF-8 string? OR do you want it translated
to
EBCDIC for you? Getting the raw UTF-8 would be very easy.

This is off the top of my head, untested:


*+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
* YAJL_GET_STRING_UTF8(): Retrieve a pointer to the raw UTF-8
* data for a YAJL string
*
* node = (input) YAJL tree node
*
* returns a pointer to a C-style, zero-terminated string
* or *NULL if the node is not a string
*+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
P YAJL_GET_STRING_UTF8...
P B export
D PI *
D node like(yajl_val) value

D nv ds likeds(yajl_val_t)
D based(node)

/free
if YAJL_IS_STRING(node);
return nv.string;
else;
return *null;
endif;
/end-free
P E


If you want the data to be translated to EBCDIC, that is more complicated
because the length of the data can vary so much, you don't necessarily
know
up front how big the data will be. You could dynamically allocate the
memory, but then the caller would have to fuss with making sure it's
freed
up, which I prefer to avoid.

So what I would do is let the caller provide the buffer. Instead of
returning a pointer, just put data into the caller's buffer. Then if it
wants to use dynamic allocation, it can... but it could also just use a
large(-er) variable or user space or whatever makes sense.

Again, I have not tested this, I'll leave that to you, but the code would
look something like this:


*+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
* YAJL_GET_STRING_BUF(): Convert JSON string to EBCDIC and place
* it into a buffer
*
* node = (input) YAJL tree node
* buf = (input) pointer to buffer to load
* bufsize = (input) size of buffer to load
*
* returns *ON if successful, *OFF otherwise
*+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
P YAJL_GET_STRING_BUF...
P B export
D PI 1n
D node like(yajl_val) value
D buf * value
D bufSize 10u 0 value

D memcpy pr * extproc('memcpy')
D dst * value
D src * value
D len 10u 0 value

D nv ds likeds(yajl_val_t)
D based(node)
D str s *
D len s 10u 0
/free
if YAJL_IS_STRING(node) = *OFF;
return *OFF;
endif;

init_gconv();

str = yajl_iconv_toLocal( gConv: nv.string );

len = %len(%str(str));
if len > bufSize;
len = bufSize;
endif;
memcpy(buf: str: len);

yajl_iconv_string_free(str);

return *ON;
/end-free
P E

Let me know how that works out for you. If these work nicely, I'll add
them to my YAJL distro.

-SK




On 4/12/2016 7:38 PM, Bradley Stone wrote:

I was working on a project the other day where the JSON contains a large
amount of data.

I had previously used my own JSON parser that I wrote a few years ago.
I
have one function that returns a pointer to the JSON data as well as the
length of that data. It can be any length, which is nice (no 64k
limit).

So I can do something like:

rc = #json_setValue('tag':'fileData); // the tag to retrieve data for
rc = #json_getData(data@);

rc will contain the length of the data, and data@ will be a pointer to
it.

From there if it's base64 I can decode it and then write it all to a
stream
file or do whatever I want.

I want to use YAJL for this now because it SO much faster than my home
grown parser. But I didn't see such a beast in the docs.

I did see, in the source, a reference to something that looks like what
I'm
looking for, but there was a stern warning there as well saying "not
recommended for RPG". :)

Just wondering if there's anything similar before I attempt to "roll my
own" and modify the YAJL parser.

Thanks!

Brad
www.bvstools.com

--

This is the RPG programming on the IBM i (AS/400 and iSeries) (RPG400-L)
mailing list
To post a message email: RPG400-L@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/rpg400-l
or email: RPG400-L-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/rpg400-l.

Please contact support@xxxxxxxxxxxx for any subscription related
questions.

--
This is the RPG programming on the IBM i (AS/400 and iSeries) (RPG400-L)
mailing list
To post a message email: RPG400-L@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/rpg400-l
or email: RPG400-L-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/rpg400-l.

Please contact support@xxxxxxxxxxxx for any subscription related
questions.


As an Amazon Associate we earn from qualifying purchases.

This thread ...

Follow-Ups:
Replies:

Follow On AppleNews
Return to Archive home page | Return to MIDRANGE.COM home page

This mailing list archive is Copyright 1997-2024 by midrange.com and David Gibbs as a compilation work. Use of the archive is restricted to research of a business or technical nature. Any other uses are prohibited. Full details are available on our policy page. If you have questions about this, please contact [javascript protected email address].

Operating expenses for this site are earned using the Amazon Associate program and Google Adsense.