Re: XML performance issue

[Rory Hewitt <rory.hewitt@xxxxxxxxx>]

Åke,

I suspect Nathan may be onto something here - is there any way in which you
can serialize (or should that be unserialize?) the processing, so that
there are no 'dead spots' in the processing.

Also, was this processing (using MQ) in place before the decision to use
XML? If so, what was the performance like when not using XML, but using
(presumably much smaller) non-XML messages?

Also, as others have pointed out, some MQ optimization can be done by
changing the logging and by ensuring that any Java processing is reusing an
existing JVM (or equivalent).

Finally, whilst XML has huge benefits, the size of the messages and the
'overhead' of all the non-data bit of the XML means that you can hit some
pretty major performance issues that you wouldn't hit if you were just
sending 'binary' messages. Is there any way you could have the ability to
send/receive messages in non-XML format and compare the performance?

Rory

On Fri, Sep 14, 2012 at 4:45 AM, Åke Olsson <ake.olsson@xxxxxx> wrote:

> We are about to roll out an integration to a remote system where XML is
> the decided way to go. We create messages with all the information for one
> product and send via MQ.
>
> We have the process in place messages are created, syntactically correct
> and sent.
>
> That would seem all right would it not?
>
> Not quite. It is all going pretty slow.
>
> Some metrics:
>
> Each XML message is from 15Kto 30K characters in length. Characters not
> bytes - since the messages are in UTF-16 (CCSID-1200) the message length in
> bytes is double that.
>
> Generating and sending messages for 64K products took just over 6 and half
> hours. This to me seems like a lot.
>
> I have tried using blocked SQL fetch to speed up the reading of all the
> involved files - no  significant change.
> All possible indexes have been applied.
>
> As for the XML message creation this is done by regular string handling,
> building the MQ message step by step. I have a hard time finding anything
> horribly wrong there either.
>
> Question: Is there any high performance tool out there that could
> significantly speed up the generation of XML?
>
>
> Med vänlig hälsning / Best regards
>
> Åke H Olsson