Re: Database I/O throughput speeds

If you are concerned about IO, then the number of disk arms could be the critical factor. If your new system significantly decreases the number of disk arms, you could see a significant decrease in performance. However, the SSD's that you are adding will likely give you a significant increase in performance. Way too many variables to make a prediction.

Mark Murphy
STAR BASE Consulting, Inc.
mmurphy@xxxxxxxxxxxxxxx

-----midrange-l-bounces@xxxxxxxxxxxx wrote: -----
To: Midrange Systems Technical Discussion <midrange-l@xxxxxxxxxxxx>
From: Vinay Gavankar
Sent by: midrange-l-bounces@xxxxxxxxxxxx
Date: 01/18/2011 07:58PM
Subject: Re: Database I/O throughput speeds

Thanks everyone for your input.

I mentioned 16 CPUs just to iterate that it is a high-end system. It
probably has no bearing on I/O speed. What I should have mentioned is it
will have 0.5 TB of RAM and I heard that they are installing solid state
drives.

As for the application, it will be single threaded. Currently it writes a
transaction to a data queue, which is picked up by another program (there
are multiple copies of that program running), and waits for the transaction
to be processed completely before reading and submitting next transaction to
the data queue. There will be multiple batch jobs running doing the same
thing at the same time, but I am trying to figure the impact on one of the
jobs.

My change involves reading and updating a couple of records after each
transaction is processed. Currently it processes about 35 to 50,000
transactions per hour depending on system usage (the transaction processing
is complex involving scores of I/O operations, though I am not sure how
many) and the new system is supposed to be at least twice as powerful (in
RAM and processor technology).

I have another program, which I was told is reading a file, writing the
records to 2 different work files based on one of the fields is positive or
negative, then reading both the files to compare and delete matching
positive and negative records. Then the remaining records are read again and
written to 2 different files. When this program was run on the current
system with about 1.5 million input records, it finished processing and
created the 2 final files with about 300K records under 3 minutes.

So it did 1.5 million reads, 1.5 million writes, then 1.5 million reads
again, about 1.2 million deletes and 0.3 million writes under 3 minutes? Is
that possible? Or do I need to check for myself what this program is
actually doing (I took someone else's word for it).

The files in above program had only about 20 fields. Does number of fields
or number of records in a file have an impact on I/O over the file?

Given the new system will be faster, should my 4 extra I/Os have
any significant impact?

On Tue, Jan 18, 2011 at 4:56 PM, Nathan Andelin <nandelin@xxxxxxxxx> wrote:

Unless you designed your batch job to run multiple parallel threads, then
it
will probably just use 1 of the CPUs, even on a 16 CPU server. It's known a
CPU
affinity. So your question probably comes down to how much faster the new
CPUs
are.

-Nathan.

----- Original Message ----
From: Vinay Gavankar <vinaygav@xxxxxxxxx>
To: Midrange Systems Technical Discussion <midrange-l@xxxxxxxxxxxx>
Sent: Tue, January 18, 2011 2:34:48 PM
Subject: Database I/O throughput speeds

 Hi,

Can any one give me some ball-park figures for number of I/O operations
(chain, update, write, delete) that can be achieved (per second or per
minute) in an RPG program running in batch mode (priority 50) on a high end
newer model? I am not sure what model my company is going for, but it is
supposed to have 16 CPUs (if I heard it right). I realize that the
throughput may vary based on the other usage of the system, but I would
have
something to go by.

I am trying to gauge the impact of adding a few I/O operations to every
transaction when processing a batch of 1-2 million transactions. Our
development machine has nowhere near that power, so running a sample
program
on it would not tell me much.

Thanks
Vinay
 --
This is the Midrange Systems Technical Discussion (MIDRANGE-L) mailing list
To post a message email: MIDRANGE-L@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/midrange-l
or email: MIDRANGE-L-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/midrange-l.



--
This is the Midrange Systems Technical Discussion (MIDRANGE-L) mailing list
To post a message email: MIDRANGE-L@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/midrange-l
or email: MIDRANGE-L-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/midrange-l.