Re: Database I/O throughput speeds

Out of sheer curiosity, and awe... how much does a machine like this cost?
We have a 2 way system with 32GB ram and usually are processing in the
thousands not millions of transactions. I love big systems, they get me
excited. It gives me that old Muscle Car feeling of raw, Tim the Tool Man
Taylor, power. How man SSD's? Total storage size?


Thanks
Bryce Martin
Programmer/Analyst I
570-546-4777



Vinay Gavankar <vinaygav@xxxxxxxxx>
Sent by: midrange-l-bounces@xxxxxxxxxxxx
01/18/2011 07:54 PM
Please respond to
Midrange Systems Technical Discussion <midrange-l@xxxxxxxxxxxx>


To
Midrange Systems Technical Discussion <midrange-l@xxxxxxxxxxxx>
cc

Subject
Re: Database I/O throughput speeds






Thanks everyone for your input.

I mentioned 16 CPUs just to iterate that it is a high-end system. It
probably has no bearing on I/O speed. What I should have mentioned is it
will have 0.5 TB of RAM and I heard that they are installing solid state
drives.

As for the application, it will be single threaded. Currently it writes a
transaction to a data queue, which is picked up by another program (there
are multiple copies of that program running), and waits for the
transaction
to be processed completely before reading and submitting next transaction
to
the data queue. There will be multiple batch jobs running doing the same
thing at the same time, but I am trying to figure the impact on one of the
jobs.

My change involves reading and updating a couple of records after each
transaction is processed. Currently it processes about 35 to 50,000
transactions per hour depending on system usage (the transaction
processing
is complex involving scores of I/O operations, though I am not sure how
many) and the new system is supposed to be at least twice as powerful (in
RAM and processor technology).

I have another program, which I was told is reading a file, writing the
records to 2 different work files based on one of the fields is positive
or
negative, then reading both the files to compare and delete matching
positive and negative records. Then the remaining records are read again
and
written to 2 different files. When this program was run on the current
system with about 1.5 million input records, it finished processing and
created the 2 final files with about 300K records under 3 minutes.

So it did 1.5 million reads, 1.5 million writes, then 1.5 million reads
again, about 1.2 million deletes and 0.3 million writes under 3 minutes?
Is
that possible? Or do I need to check for myself what this program is
actually doing (I took someone else's word for it).

The files in above program had only about 20 fields. Does number of fields
or number of records in a file have an impact on I/O over the file?

Given the new system will be faster, should my 4 extra I/Os have
any significant impact?

On Tue, Jan 18, 2011 at 4:56 PM, Nathan Andelin <nandelin@xxxxxxxxx>
wrote:

Unless you designed your batch job to run multiple parallel threads,

then

it
will probably just use 1 of the CPUs, even on a 16 CPU server. It's

known a

CPU
affinity. So your question probably comes down to how much faster the

new

CPUs
are.

-Nathan.

----- Original Message ----
From: Vinay Gavankar <vinaygav@xxxxxxxxx>
To: Midrange Systems Technical Discussion <midrange-l@xxxxxxxxxxxx>
Sent: Tue, January 18, 2011 2:34:48 PM
Subject: Database I/O throughput speeds

Hi,

Can any one give me some ball-park figures for number of I/O operations
(chain, update, write, delete) that can be achieved (per second or per
minute) in an RPG program running in batch mode (priority 50) on a high

end

newer model? I am not sure what model my company is going for, but it is
supposed to have 16 CPUs (if I heard it right). I realize that the
throughput may vary based on the other usage of the system, but I would
have
something to go by.

I am trying to gauge the impact of adding a few I/O operations to every
transaction when processing a batch of 1-2 million transactions. Our
development machine has nowhere near that power, so running a sample
program
on it would not tell me much.

Thanks
Vinay
--
This is the Midrange Systems Technical Discussion (MIDRANGE-L) mailing

list

To post a message email: MIDRANGE-L@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/midrange-l
or email: MIDRANGE-L-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/midrange-l.



--
This is the Midrange Systems Technical Discussion (MIDRANGE-L) mailing

list

To post a message email: MIDRANGE-L@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/midrange-l
or email: MIDRANGE-L-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/midrange-l.