RE: disk arms (was RE: Tips for user ASP)

Charly,

The method of determining whether disk arms are a problem hasn't really
changed through the life of the AS/400-iSeries.  Most performance
characteristics follow a logarithmic curve.  The knee of the curve, that
place where performance ceases a gradual decline and starts to decline
dramatically, is the threshold at which red flags should go up.  This
number has stayed constant at forty per cent through all of the advances
in I/O speed, caching, and disk characteristics.  It has to do with what
is called a queuing multiplier.  What is the likelihood that an I/O
request can be fulfilled immediately when that request is issued to a
disk drive?  If the drive is busy then the request must be queued and
reissued later.  At forty per cent, this multiplier effect creates the
disk arm equivalent of main storage thrashing.

For disk activity, this occurs at about forty per cent disk arm
utilization as viewed on the WRKDSKSTS display or in many of the
performance reports.  If someone thinks that the number of disk arms is
a problem, then they can use this command to view per cent utilization
during a sample interval.  If all drives are close to (or above) this
threshold, then you have a problem.  If a single (or very few drives)
are pegged, then the system could be balanced using the ASPBAL commands
with the *USAGE parameter.  It is really one of the easier performance
characteristics to diagnose.  Generally speaking, if you have maxed out
activity on all drives, your only alternative to fix the problem
involves hardware.

In your list below, I believe that you may have omitted one of the more
important hardware balancing factors.  All of the disk and tape
controllers (IOA's) are controlled by input/output processors (IOP's).
These IOP's do have maximum throughput characteristics so putting
multiple high-speed adapters on the same I/O processor can lead to the
type of performance bottleneck you describe.  If you had two RAID
controllers and a high-speed tape controller, probably the most
important hardware step you could take to optimize performance would be
to have each adapter controlled by a different processor.  In many
cases, this can be accomplished without adding additional buses.

I have found discussions of buses to be more of theoretical importance
than practical importance (I confess to being fascinated by system
throughput issues).  The reason being, when you start to design a system
to maximize throughput, all of a sudden you start to hear a "Ka-ching!
Ka-ching!" coming from the direction of Rochester.  Certainly on a
system with multiple buses, high throughput devices should be spread
out; but in most cases the idea of adding buses purely for performance
reasons is financially out of reach for most companies.  On the newer
hardware, your disk enclosures tend to come with their own buses.  For
instance the forty-five unit disk expansion towers will allow for three
RAID controllers in each enclosure.  Generally you would fill the unit
and have six RAID sets.  Few installations could afford to not fully
utilize that resource.  This becomes particularly important when you
consider that the addition of a chassis comes with monthly maintenance
charges.

A second factor which could be added to your list would be the number of
high-speed loops (HSL).  The 8xx hardware attaches the disk expansion
towers via HSL and multiple towers can be daisy-chained in a single
loop.  If I recall correctly, we can also attach some of the high-speed
tape drives via HSL also.  Because these loops do have maximum
throughput limits, it would be beneficial to install as many loops as
possible and to avoid putting tape drives and disk enclosures on the
same loop.  Even though the HSL cables are expensive, additional loops
can be added without adding buses or enclosures which would drive up
monthly maintenance costs.

Regards,
Andy Nolen-Parkhouse

> On Behalf Of Charly Jones
> Subject: RE: disk arms (was RE: Tips for user ASP)
>
>
> Chris + Scott --
>
> How do you know that your disk arms are the problem?  What
> tools do you use?  Which reports (or screens) do you look at?
> Would you be willing to talk about how many disk arms are
> attached to what disk controllers and whether you have
> separate disk pools.  I would like to continue this
> discussion if anyone is interested.
>
> Actually the bus is sometimes important as well.  If you
> have a fast tape drive on the same bus with disk drives, the
> disk activity can reduce the maximum throughput of the tape
> drive.  I can't help thinking that tape operations
> would similarly affect disk activity.  So to get the full
> picture, is anyone willing to post the following?
>
> 1)  How many system buses you have.
> 2)  How many disk controllers you have.
> 3)  The distribution of disk controllers on your buses.
> 3)  What other high speed devices share a bus with a disk controller.
> 4)  How many disk arms are attached to each disk controller.
> 5)  How many disk controllers + arms are in each disk ASP.
> 6)  Whether you think you have a disk arm problem.
> 7)  How you can tell you have a disk arm problem.
>
> Thanks a bunch,
>
> --  Charly