Re: JSON performance

Bradley

the interesting part is - can we beat Java, PHP, Node on IBM I in
native ILE to deliver JSON ?

On Sun, Oct 18, 2015 at 7:43 PM, Bradley Stone <bvstone@xxxxxxxxx> wrote:

I too am blown away at the performance by the YAJL port from Scott.

We had written our own JSON parser (I needed one before anything else was
available). It was a fun programming project that reminded my of my
computer science days, traversing trees with recursion, etc... but the
performance of it was blown away by YAJL.

I have since switched all new development to YAJL as well. I was able to
put a front end on it that matched our JSON parser in case that's needed

by

any customers already using ours and not willing to switch. :)

With ours we would use string notation to locate the node/data...

customerList[n]:customerName:firstname

Retrieve the nth customer first name from the customerList array.

Brad
www.bvstools.com

On Sun, Oct 18, 2015 at 6:03 AM, Henrik Rützou <hr@xxxxxxxxxxxx> wrote:

Kevin

I will do it a little different way when I comes to it, at the moment

it

is

just possible to use yajl 'as is' in powerEXT Core.

I have two JSON node inline subprocedures jsonNode() and jsonEndNode().

These are primarly used in my REST/CRUD web services while my

javascript

program generator uses templates that contains the JSON based code that
is used to define various EXT JS UI components.

For the latter I will leave them 'as is' since they through API's are
completely
bound to the underlying CGIDEV2 output buffer and btw doesn't use the
jsonNode()/jsonEndNode() sub-procedures.

For my REST/CRUD services I will introduce a jsonMode(*ORIGIN/*YAJL)

sub-

procedure that internally calls yajl instead of using the original

method

that
writes JSON to the CGIDEV2 buffer.

So one statement is all needed to shift from original to yajl without
having to
rewrite a lot of code and still customized code will work and if it

doesn't

yajl
can then just be turn off again.



On Sun, Oct 18, 2015 at 12:32 PM, Henrik Rützou <hr@xxxxxxxxxxxx>

wrote:

Kevin

Your performance problems of course inspired me to the decision, but

first

when Scott
added som more sub-procedures (that I needed) in july 2015 I

integrated

it. I actually
started to look at it in 2014.

These new sub-procedures are:

I - export symbol(yajl_writeStdout)
I - export symbol(yajl_stdin_load_tree)
I - export symbol(yajl_addCharEx)
I - export symbol(yajl_addCharStmf)
I - export symbol(yajl_exbuf_new)
I - export symbol(yajl_exbuf_concat_ptr)
I - export symbol(yajl_exbuf_concat)
I - export symbol(yajl_exbuf_free)
I - export symbol(yajl_save_string_stmf)
I - export symbol(yajl_getBuf)
I - export symbol(yajl_tree_free_rpg)



Especially the yajl_getBuf(addr,size); without copying the storage

helped

a lot.



On Sun, Oct 18, 2015 at 12:14 PM, Kevin Turner <
kevin.turner@xxxxxxxxxxxxxxx> wrote:

Presumably you made this decision following that email trail I

started a

few months ago comparing yajl to the offering from rpgnextgen

(simply

called JSON)?

I switched to yajl back then when we discovered the bottleneck that

was

caused by the rpgnextgen parser - it's a shame because it is

actually

a

little bit more intuitive to use (procedure name-wise). All I did

was

to

put a more intuitive wrapper around yajl and the performance

improvements

were significant.

On 18 Oct 2015, at 10:51, Henrik Rützou <hr@xxxxxxxxxxxx> wrote:

In my newly uploaded October 2015 version (5.03) of powerEXT Core

I

have

included Scott Klement’s port of C based YAJL (Yet Another JSON

Library) a

port many seems to have overlooked.



Before I have written some sub-procedures in RPGLE (JSON node

support)

but

in comparison to YAJL I get 2.5-5x better performance with YAJL

thus

reducing an average 20KB JSON REST/CRUD total request time in the

browser

from 300ms to approx. 150ms or a performance gain 2x.



Now, it could be funny to compare performance in other techniques

such

as

node.js and .NET. So I have built a little demo program that

should

be

easy

to replicate in another language or even other native JSON

wrappers.

The program uses SQL to read a table we all have (QIWS/QCUSTCDT)

but

instead of making a single JSON object per row it loops 1,000

times

and

creates 1,000 row objects for each physical row in the table

resulting

in

132.000 JSON nodes being generated (31,500 per second) in a 3,6 MB

JSON

object.



Using my RPG JSON node support it took 24 sec to generate the

object,

with

YAJL it took 4,2 sec - with smaller objects the perfomance gain is

only

2.5x.



Resources:



Scott Klements YAJL port: http://www.scottklement.com/yajl/



powerEXT Core including YAJL: http://powerext.com



My test program: http://powerext.com/rpgyajl.txt




--
Regards,
Henrik Rützou

http://powerEXT.com <http://powerext.com/>
--
This is the Web Enabling the IBM i (AS/400 and iSeries) (WEB400)

mailing list

To post a message email: WEB400@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/web400
or email: WEB400-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/web400.


___________________________________________
This email has been scanned by iomartcloud.
http://www.iomartcloud.com/

________________________________

NOTICE: The information in this electronic mail transmission is

intended

by CoralTree Systems Ltd for the use of the named individuals or

entity

to

which it is directed and may contain information that is privileged

or

otherwise confidential. If you have received this electronic mail
transmission in error, please delete it from your system without

copying or

forwarding it, and notify the sender of the error by reply email or

by

telephone, so that the sender's address records can be corrected.

--------------------------------------------------------------------------------

CoralTree Systems Limited
25 Barnes Wallis Road
Segensworth East, Fareham
PO15 5TT

Company Registration Number 5021022.
Registered Office:
12-14 Carlton Place
Southampton, UK
SO15 2EA
VAT Registration Number 834 1020 74.
--
This is the Web Enabling the IBM i (AS/400 and iSeries) (WEB400)

mailing

list
To post a message email: WEB400@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/web400
or email: WEB400-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/web400.

--
Regards,
Henrik Rützou

http://powerEXT.com <http://powerext.com/>

--
Regards,
Henrik Rützou

http://powerEXT.com <http://powerext.com/>
--
This is the Web Enabling the IBM i (AS/400 and iSeries) (WEB400)

mailing

list
To post a message email: WEB400@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/web400
or email: WEB400-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/web400.

--
This is the Web Enabling the IBM i (AS/400 and iSeries) (WEB400) mailing
list
To post a message email: WEB400@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/web400
or email: WEB400-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/web400.

--
Regards,
Henrik Rützou

http://powerEXT.com <http://powerext.com/>
--
This is the Web Enabling the IBM i (AS/400 and iSeries) (WEB400) mailing
list
To post a message email: WEB400@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/web400
or email: WEB400-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/web400.