RE: Avoiding disk arms with CLOB

So, your web service is processed on another platform, which then tries to perform the business logic on IBM Power using stored procedures...

Is there a reason you don't write your web service in RPG? Doing this would greatly simplify your design and performance issues.

I have been using Aaron Bartel's product (RPG-XML Suite) from Krengel Tech to build our RPG based web services. It is really very easy to do with the proper tooling.

Hth,
-Eric DeLong

-----Original Message-----
From: rpg400-l-bounces@xxxxxxxxxxxx [mailto:rpg400-l-bounces@xxxxxxxxxxxx] On Behalf Of RPGLIST
Sent: Thursday, September 19, 2013 9:56 AM
To: RPG programming on the IBM i (AS/400 and iSeries)
Subject: Re: Avoiding disk arms with CLOB

It would be coming in through a SOAP transaction to web server which in
turn will be sending it to me, right now via a stored procedure but I'm
open to another methodology.

the 12k is not the total limit, that's just one line of data, this could
very well exceed 200k or more.

How are you receiving? Socket? HTTPAPI? 12k into a stream file doesn't
sound like much.


On Thu, Sep 19, 2013 at 8:41 AM, RPGLIST <rpglist@xxxxxxxxxxx> wrote:

I'm not even trying to touch performance tuning for that same reason :)
and its outside my area :)

That said, I'm looking at a single load line having potentially a length
of over 12k and with a limit of 64512 on a queue I'm limited in the
number
of lines I can handle. They will all come through as one stream if you
will.

On 9/19/2013 9:21 AM, RPGLIST wrote:

I'm facing the need to process a very large amount of data, which is
more
than likely going to come in as a SOAP or XML stream but I need to
process
this quickly and without losing time with the disk arms if at all
possible.

If you're going to store this information, you're going to have to

write

it to disk at some point.

I know that a clob will allow me to store a larger set of data but

from

what I'm reading, you can't just access and read it like a normal
variable.

Correct.

http://www.ibmsystemsmag.com/ibmi/developer/general/BLOBs,-CLOBs-and-RPG/

I assume, and if I'm wrong feel free to slap with a lead pipe, that

like

with regular PF's writes are going to cost me time. I know some
individuals don't care about time or say its not important but I need

to

return a response in a matter of seconds like 5-8 or lower if

possible

and
I'm not going to argue that point. It is what it is.

What's the application like? Is it a simple matter of receive, store,
acknowledge and process later? Or is it a case of receive a line,
process a line, store the data, acknowledge the transaction? The

advise

you get will depend a lot on the way you need to handle things.

The data coming in if you put it in a PF would have a record length

of

somewhere in the 12k range, so you can imagine it'll be larger with

XML

tags and that's a single record, whereas we could receive up to 99

lines

with each line in the 12k+ range.

Offhand, 120k sounds like no big deal in terms of disk I/O. Have you
mocked something up and found a performance issue somewhere, or is the
question a matter of being proactive? After 30+ years of tuning
applications, I feel confident stating that performance tuning without
measurement is the road to madness. Ask my therapist :-)

Now, if someone has a suggestion I'm all ears :)

If you suspect disk I/O will be the bottleneck (and you know your
machine better than I do!) then consider creating an asynchronous
process to do the writes. One job will receive the data from your
trading partner and then say, dump it into a data queue and keep
reading. Another job will read entries off the queue and write them

to

disk.
--buck
--
This is the RPG programming on the IBM i (AS/400 and iSeries)

(RPG400-L)

mailing list
To post a message email: RPG400-L@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/rpg400-l
or email: RPG400-L-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/rpg400-l.

--
This is the RPG programming on the IBM i (AS/400 and iSeries) (RPG400-L)
mailing list
To post a message email: RPG400-L@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/rpg400-l
or email: RPG400-L-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/rpg400-l.

--
This is the RPG programming on the IBM i (AS/400 and iSeries) (RPG400-L)
mailing list
To post a message email: RPG400-L@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/rpg400-l
or email: RPG400-L-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/rpg400-l.