× The internal search function is temporarily non-functional. The current search engine is no longer viable and we are researching alternatives.
As a stop gap measure, we are using Google's custom search engine service.
If you know of an easy to use, open source, search engine ... please contact support@midrange.com.


  1.  Home
  2. WEB400
  3. October 2015

Re: JSON performance

I too am blown away at the performance by the YAJL port from Scott.

We had written our own JSON parser (I needed one before anything else was
available). It was a fun programming project that reminded my of my
computer science days, traversing trees with recursion, etc... but the
performance of it was blown away by YAJL.

I have since switched all new development to YAJL as well. I was able to
put a front end on it that matched our JSON parser in case that's needed by
any customers already using ours and not willing to switch. :)

With ours we would use string notation to locate the node/data...

customerList[n]:customerName:firstname

Retrieve the nth customer first name from the customerList array.

Brad
www.bvstools.com

On Sun, Oct 18, 2015 at 6:03 AM, Henrik Rützou <hr@xxxxxxxxxxxx> wrote:

Kevin

I will do it a little different way when I comes to it, at the moment it is
just possible to use yajl 'as is' in powerEXT Core.

I have two JSON node inline subprocedures jsonNode() and jsonEndNode().

These are primarly used in my REST/CRUD web services while my javascript
program generator uses templates that contains the JSON based code that
is used to define various EXT JS UI components.

For the latter I will leave them 'as is' since they through API's are
completely
bound to the underlying CGIDEV2 output buffer and btw doesn't use the
jsonNode()/jsonEndNode() sub-procedures.

For my REST/CRUD services I will introduce a jsonMode(*ORIGIN/*YAJL) sub-
procedure that internally calls yajl instead of using the original method
that
writes JSON to the CGIDEV2 buffer.

So one statement is all needed to shift from original to yajl without
having to
rewrite a lot of code and still customized code will work and if it doesn't
yajl
can then just be turn off again.



On Sun, Oct 18, 2015 at 12:32 PM, Henrik Rützou <hr@xxxxxxxxxxxx> wrote:

Kevin

Your performance problems of course inspired me to the decision, but
first
when Scott
added som more sub-procedures (that I needed) in july 2015 I integrated
it. I actually
started to look at it in 2014.

These new sub-procedures are:

I - export symbol(yajl_writeStdout)
I - export symbol(yajl_stdin_load_tree)
I - export symbol(yajl_addCharEx)
I - export symbol(yajl_addCharStmf)
I - export symbol(yajl_exbuf_new)
I - export symbol(yajl_exbuf_concat_ptr)
I - export symbol(yajl_exbuf_concat)
I - export symbol(yajl_exbuf_free)
I - export symbol(yajl_save_string_stmf)
I - export symbol(yajl_getBuf)
I - export symbol(yajl_tree_free_rpg)



Especially the yajl_getBuf(addr,size); without copying the storage helped
a lot.



On Sun, Oct 18, 2015 at 12:14 PM, Kevin Turner <
kevin.turner@xxxxxxxxxxxxxxx> wrote:

Presumably you made this decision following that email trail I started a
few months ago comparing yajl to the offering from rpgnextgen (simply
called JSON)?

I switched to yajl back then when we discovered the bottleneck that was
caused by the rpgnextgen parser - it's a shame because it is actually a
little bit more intuitive to use (procedure name-wise). All I did was
to
put a more intuitive wrapper around yajl and the performance
improvements
were significant.

On 18 Oct 2015, at 10:51, Henrik Rützou <hr@xxxxxxxxxxxx> wrote:

In my newly uploaded October 2015 version (5.03) of powerEXT Core I
have
included Scott Klement’s port of C based YAJL (Yet Another JSON
Library) a
port many seems to have overlooked.



Before I have written some sub-procedures in RPGLE (JSON node support)
but
in comparison to YAJL I get 2.5-5x better performance with YAJL thus
reducing an average 20KB JSON REST/CRUD total request time in the
browser
from 300ms to approx. 150ms or a performance gain 2x.



Now, it could be funny to compare performance in other techniques such
as
node.js and .NET. So I have built a little demo program that should be
easy
to replicate in another language or even other native JSON wrappers.



The program uses SQL to read a table we all have (QIWS/QCUSTCDT) but
instead of making a single JSON object per row it loops 1,000 times
and
creates 1,000 row objects for each physical row in the table resulting
in
132.000 JSON nodes being generated (31,500 per second) in a 3,6 MB
JSON
object.



Using my RPG JSON node support it took 24 sec to generate the object,
with
YAJL it took 4,2 sec - with smaller objects the perfomance gain is
only
2.5x.



Resources:



Scott Klements YAJL port: http://www.scottklement.com/yajl/



powerEXT Core including YAJL: http://powerext.com



My test program: http://powerext.com/rpgyajl.txt




--
Regards,
Henrik Rützou

http://powerEXT.com <http://powerext.com/>
--
This is the Web Enabling the IBM i (AS/400 and iSeries) (WEB400)
mailing list
To post a message email: WEB400@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/web400
or email: WEB400-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/web400.


___________________________________________
This email has been scanned by iomartcloud.
http://www.iomartcloud.com/


________________________________

NOTICE: The information in this electronic mail transmission is intended
by CoralTree Systems Ltd for the use of the named individuals or entity
to
which it is directed and may contain information that is privileged or
otherwise confidential. If you have received this electronic mail
transmission in error, please delete it from your system without
copying or
forwarding it, and notify the sender of the error by reply email or by
telephone, so that the sender's address records can be corrected.





--------------------------------------------------------------------------------


CoralTree Systems Limited
25 Barnes Wallis Road
Segensworth East, Fareham
PO15 5TT

Company Registration Number 5021022.
Registered Office:
12-14 Carlton Place
Southampton, UK
SO15 2EA
VAT Registration Number 834 1020 74.
--
This is the Web Enabling the IBM i (AS/400 and iSeries) (WEB400) mailing
list
To post a message email: WEB400@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/web400
or email: WEB400-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/web400.




--
Regards,
Henrik Rützou

http://powerEXT.com <http://powerext.com/>





--
Regards,
Henrik Rützou

http://powerEXT.com <http://powerext.com/>
--
This is the Web Enabling the IBM i (AS/400 and iSeries) (WEB400) mailing
list
To post a message email: WEB400@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/web400
or email: WEB400-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/web400.



As an Amazon Associate we earn from qualifying purchases.

This thread ...
Follow-Ups:
Replies:
Follow On AppleNews
Return to Archive home page | Return to MIDRANGE.COM home page

This mailing list archive is Copyright 1997-2024 by midrange.com and David Gibbs as a compilation work. Use of the archive is restricted to research of a business or technical nature. Any other uses are prohibited. Full details are available on our policy page. If you have questions about this, please contact [javascript protected email address].

Operating expenses for this site are earned using the Amazon Associate program and Google Adsense.