Re: YAJL implementation / suggestions

[Scott Klement <midrange-l@xxxxxxxxxxxxxxxx>]

Henrik,

I have posted my thoughts on the first issue here:
http://www.code400.com/forum/showthread.php/13743-JSON-and-YAJL-problem

It would make things simpler if we had this discussion in only one forum 
instead of having discussions in many places.  At least, for me, I find 
it confusing when part of a conversation is on Midrange.com, part is on 
Code400.com, etc...

For the 2nd issue, yes... I could add something like that to YAJLR4, 
that would be easy to do.

-SK



On 7/4/2015 7:18 AM, Henrik Rützou wrote:
> Hi Scott,
>
>
>
> I have debugged the error gio describes in another tread and there seems to
> be a storage allocation problem in YAJL.
>
>
>
> When I reach approx. 9-10 MB in the JSON build YAJL fails and doesn’t
> extend its buffer. It would be nice if you were able to specify a standard
> heap size of the output buffer in yajl_genOpen(*ON:100000); that then was
> used as the initial heap size and afterwards was used to extend the
> allocated storage for the generation instead of having to modifying your
> YAJL distribution that would affect all YAJL generations and thereby
> overloading allocated storage even if the program only produces small JSON
> objects.
>
>
>
> Another thing is the yajl_copyBuf(); that returns data to a local
> field/allocated storage and a length of the converted result from UTF-8 to
> EBCDIC. This is not only an unnecessary conversion (I know you can pass
> 1208 as the CCSID) but it also requires that you allocate a field/storage
> beforehand. In other words this has to be big enough to hold the unknown
> length of the result.
>
>
>
> My suggestion is a sub-procedure like yajl_getBuf(addr:size); that calls
> yajl_gen_get_buf(); and just returns the UTF-8 address and size to the main
> program. This information can then be used to pass the generated UTF-8
> result with either text/… charset=utf-8 or any other non “text/…” mime
> header so Apache doesn’t convert back from EBCDIC to UTF-8 and thereby
> speeding thing up directly or used to allocate local storage that fits the
> JSON result.
>
>
>
> The first I don’t know how to do, the second I could do myself but that
> would require that I modify your YAJL implementation in my local copy or
> you implement the simple sub-procedure (yajl_getBuf) in yours.