× The internal search function is temporarily non-functional. The current search engine is no longer viable and we are researching alternatives.
As a stop gap measure, we are using Google's custom search engine service.
If you know of an easy to use, open source, search engine ... please contact support@midrange.com.



On 9/14/2012 6:45 AM, Åke Olsson wrote:
Each XML message is from 15Kto 30K characters in length. Characters not bytes - since the messages are in UTF-16 (CCSID-1200) the message length in bytes is double that.

Generating and sending messages for 64K products took just over 6 and half hours. This to me seems like a lot.

I just spent the last several months putting together an infrastructure for a very similar project. Unfortunately, your numbers aren't that far off.

First, the generation. You can generate XML into a local buffer and send that buffer to MQ, or you can write the XML to an IFS file and then send the IFS file. The second part adds the overhead of reading the file back from the IFS, but it allows you to look at what you sent, which is pretty important for debugging. I ended up with a compromise: I always sent from an internal buffer but I had a switch that would optionally write the message to an IFS file. It's saved me countless hours of debugging.

Next, the transmission. A lot of your overhead will depend on your MQ configuration. I am NOT an MQ expert by any means, but one thing to look into is your logging level. If your queues are logged, then MQ will be writing the messages to a disk file in addition to sending them on. In a multi-platform environment, that means writing to the IFS for each platform. I found that once we added logging (required for the package I was installing), message processing jumped into the tens or even hundreds of milliseconds per message. If I do the back of the envelope math for you, you're getting throughput of about 300ms per message. Unfortunately, at least with logging, that's not unexpected.

The first thing I would do would be to segregate the generation and the sending. Write code to just generate the messages without sending to MQ and see how long it takes. Compare that time to your total time; any additional time is overhead from MQ.

Joe


As an Amazon Associate we earn from qualifying purchases.

This thread ...

Follow-Ups:
Replies:

Follow On AppleNews
Return to Archive home page | Return to MIDRANGE.COM home page

This mailing list archive is Copyright 1997-2024 by midrange.com and David Gibbs as a compilation work. Use of the archive is restricted to research of a business or technical nature. Any other uses are prohibited. Full details are available on our policy page. If you have questions about this, please contact [javascript protected email address].

Operating expenses for this site are earned using the Amazon Associate program and Google Adsense.