Re: XML performance issue

64k of data should take no time, Even at 4gb of data, burn some dvd's of
your files and fedex them it will be faster if you have many files.




From:
Charles Wilt <charles.wilt@xxxxxxxxx>
To:
"RPG programming on the IBM i / System i" <rpg400-l@xxxxxxxxxxxx>,
Date:
09/14/2012 08:32 AM
Subject:
Re: XML performance issue
Sent by:
rpg400-l-bounces@xxxxxxxxxxxx



So 30-60KB * 64K...

Assuming my math is right, that 2-4GB of data....6hrs doesn't seem
unreasonable...but it depends on how fast your connection to the
remote site is...

Have you benchmark how long it takes to just generate the messages?

Are you using VARYING character fields, or fixed?

Are you using MQs journalling, ie. persistent messages?

What version of OS and MQ?

Charles

On Fri, Sep 14, 2012 at 7:59 AM, Jan Grove Vejlstrup <jgv@xxxxxxxx> wrote:

Hej Åke

Have you looked at cgidev2: http://cgidev2.easy400.net/

Best regards

Jan

Am 14.09.2012 13:45, schrieb Åke Olsson:

We are about to roll out an integration to a remote system where XML is

the decided way to go. We create messages with all the information for one
product and send via MQ.

We have the process in place messages are created, syntactically

correct and sent.

That would seem all right would it not?

Not quite. It is all going pretty slow.

Some metrics:

Each XML message is from 15Kto 30K characters in length. Characters not

bytes - since the messages are in UTF-16 (CCSID-1200) the message length
in bytes is double that.

Generating and sending messages for 64K products took just over 6 and

half hours. This to me seems like a lot.

I have tried using blocked SQL fetch to speed up the reading of all the

involved files - no significant change.

All possible indexes have been applied.

As for the XML message creation this is done by regular string

handling, building the MQ message step by step. I have a hard time finding
anything horribly wrong there either.

Question: Is there any high performance tool out there that could

significantly speed up the generation of XML?

Med vänlig hälsning / Best regards

Åke H Olsson
[Description: cid:image001.png@01CA1FE6.387A03A0]
Box 433 SE 551 16 Jönköping Sweden visit: Brunnsgatan 11
phone: +46 (0)36 342976 mobile: +46 (0)705 482976 fax: +46 (0)36 34

29 29

ake.olsson@xxxxxx<mailto:ake.olsson@xxxxxx> www.pdb.se

--
This is the RPG programming on the IBM i / System i (RPG400-L) mailing

list

To post a message email: RPG400-L@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/rpg400-l
or email: RPG400-L-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/rpg400-l.