Re: AS/400 backup solutions questions -- MIDRANGE-L

In this post I hopefully both answer Brad's questions & also raise a number
of other points worth some discussion.  There are some potential challenges
above & beyond those that Brad raised.

Does this thread really belong on midrange_L or is it more non-technical
AS/400, focused more on the PC connectivity of non-tech list?

There was a recent thread on BPCS_L about backup/recovery more frequent than
nitely backup & someone said the closer you come to real time recovery the
cost goes up logarithmically.

In recent years we have needed to recover SOMETHING from backup on the
average of once every two months ... usually one file, one member of a file,
or a query definition.  In several cases we have not discovered right away
that we needed this, so dig back several backup tapes to figure out if it is
recoverable.  This means that our backup tapes policy is not as good as it
could be, but discussion with management about having extra tapes or
different rotation basically concludes that the rate at which we have
problems does not justify the cost of the extra tapes.

Doing a restore of one file takes excessive time, plus each time I have to
study the documentation to remember the right thing to do, like when we have
a bunch of logicals over a physical file, what all needs restoration in what
sequence.

If you doing piecemeal restore, you have to restore physical files before the
logicals that are over them.  Now we have some physicals in one library &
logicals over them in another library & the name of the library with the
logicals is alphabetically named before the library with the physicals.  I
wish I had learned about this nuance before we did the library naming.
Restores typically are in library sequence unless you doing piecemeal restore.

The terminology that I am using might not be quite correct.  I know that it
is possible to do backups of only the data that has changed since the last
backup, so that one could do a complete backup once a week & a bunch of
whatever you call them each nite, then in the restore you first restore the
once a week & then handle the intermediate with different rules ... the whole
thing replaces what's on line & the intermediate only replaces what has
changed with what's on line.  This to me sounds like some kind of nightmare
to manage.

A few months ago I was starting the backup & one of my co-workers said OOPS I
forgot to do ....... which is normally done before the backup & he wanted to
do that thing now ..... can we kill the backup?  I offered to have him spell
out instructions for me how to do whatever it is, but after the backup gets
finished, but that can be difficult to do when you not have easy access to
screen prints of the applications.

We learned, yes you can kill a backup & I took this opportunity to show my
backup person (who does this kind of stuff when I am sick or on vacation) how
to get at second session at main console when system in restricted state, but
the time lost in the process was all out of proportion to what we were trying
to accomplish ... moral from story ... do not do this on a whim.

Our tape drive went belly up this week & IBM did replace it less than 24
hours after we gave up on resolving the problem with IBM PTF downloads, which
is a small miracle considering that they had to fly in the replacement drive
from another city & air service to Evansville Indiana was not that great
before the national crisis.

We will be doing our end-month this Wed right before Thanksgiving.
There was some doubt that the IBM replacement drive would get here in time.
Management wanted to bull through without any backup but BPCS has some jobs
associated with archiving old history in which it was inconceivable to me
that we could go without running say INV900 especially since we are also
doing a physical inventory this week, so I was exploring options like ... if
we jack up ITH retention from 365 days to say 450 days there will be no
ancient records to go to tape so will the software even find out that the
drive is dead when there are no records to save, or will it go through the
process of trying to write a file with no records?

> OVRDBF tapefile-name TOFILE(DBF-with-same-record-length).
> Tape drive is hosed because of hardware?

But the drive was replaced in time so my panic what to do about INV900 is
gone.

Also the tape drive on our Microsoft network has been particularly flaky in
recent weeks.

This has led to some discussion of alternatives with management & the people
who handle backup for the PCs & me who manages the AS/400 & I also bounced
some ideas off of the CE when he visited, with respect to tape drive life &
backup reliability.

My home PC tape backup went belly up & this is the second tape drive in 3
years to die on me & I don't use it at home that much ... while backups at
work are daily, at home they perhaps once a month, if that often.  My
shopping around for replacement asked about PC tape drive with cleaning tape
system similar to what IBM uses, instead of the instructions to clean it
manually with cotton swab & alcohol ... the PC market place people telling me
that the technology has improved to the point that we do not need that any
more, which they told me when I got the one that just died & I do not believe
it.  Dust gets into everything & it needs a system of removal & blowing air &
vacuum does not cut it all places.

So at home I starting an experiment.  Backup to CD Rom Write.

The cost of CD Rom Write drive on PC is about the same as a Tape Drive Backup
& the only variable is how fast you want it.  It sounds like a CD Rom Write
drive on an AS/400 is extremely inexpensive ... IBM calls it OPTICAL Platter.
 The market uses this for archiving history files & you can put spool file
reports there indexed according to key information in the reports ... what CD
files contain reports on this or that item customer date-range etc.

Just as the PC market place has no effective software for backups to CD Rom
Write (my PC parts supplier mentioned some brand names that allegedly do this
but they are forever doing service calls to customers using those brand
names), AS/400 in same boat, unless you go BRMS which I have heard of & know
to be expensive out of our budget.

These tapes cost big bucks ... $35 - $40 on home PC, $50 & up for AS/400.
CDs cost 35 cents each for write only, $2 for write & write again ... can be
purchased in packs of 50-100 ... now the CDs cannot hold anything like as
much as one tape, so you have to do multi-volume so this could be a hassle,
reminiscent of, but not as bad as, diskettes.

On my home PC I plan to have several equivalents of GO BACKUP in rotation ...
one backup for my DOS games, one for my WORD documents, one for my e-mail,
etc.  I will need to do some analysis to see if everything in one document
folder can fit on one CD Rom ... basically there is no such animal as backup
software for CD, what you do is copy from folder on C drive to folder on CD
drive like you move anything else around on PC desk top & that's how you work
the "backup."

I have become rather disgusted with the retention reliability life span of
tapes.
When we were on diskette, I used to archive off-line the stuff we hardly ever
use, like say the source code for software ... we have thousands upon
thousands of programs ... in the next year I might need to access 100, I just
do not know which 100 ... it makes sense to store that off-line & only upload
what I need to acess.

When our backups were diskettes, I did exactly that, with 2-3 sets which was
adequate insurance against occasional diskette going bad.  I knew from
experience that stuff saved to 8 inch diskette was good for at least 10
years, if not 25 years.

These tapes I do not trust much longer than 6 months.

It remains to be seen how long CD Rom can be trusted ... I suspect the
technology closer to diskettes than tapes on this topic.

A friend was telling me about this removable hard drive system they had on
their PC that sounded much like something we had on IBM in some old S/3 days
... they were using the system for purposes of backup & archiving & different
sets of drives on-line for different applications.  I came to the conclusion
that was the right system for my home PC just about the time the company that
made that stuff went bankrupt.  I did not get the system.

The nature of our business is that we do not need 24x7 access for the people,
so it is practical to have some downtime for the mob of users in the
evenings, when we run various BPCS tasks that it is counter productive to be
doing them when lots other people on the system.

I work swing shift.  I show up in afternoon before the mob of users leave, so
I can find out what kinds of needs my co-workers might have with respect to
evolving situations, then after they have gone I am doing a variety of tasks
... operations & software development.

We now have a model 170 which we converted to Dec 1999 from an AS/436.
We now run SAVE-21 every week nite, which takes about 1 1/2 hours.

On the AS/436 we used to use GO BACKUP week nites with GO SAVE 21 once or
twice a month, because the GO BACKUP saved the volatile libraries only & took
about 2 hours, while the GO SAVE 21 took 5-6 hours.

When we got the model 170, the GO SAVE 21 went to 1 1/2 hours, while the GO
BACKUP went to 4 hours so it was a no brainer what to do.  I suspect changing
tape drive technology & think that perhaps GO BACKUP might save a library,
pause, save a library, pause, while GO SAVE is more productive.

We are not using save while active.
We shut everything down for backup.
We have found that if printers are active during backup, there can be some
error message hung on a session that causes error messages associated with
the save & you know you could spend hours analysing the log of a save to
figure out what is Ok or needed to fix it ... much simpler to prevent this
scenario in the first place.

I know that if we needed 24x7 access, and if we were rich in hard disk, you
can get everyone off the system for like 5 minutes & do check pointed save to
a SAV file on hard disk, which is really fast, then while people are using
the system like normal, be copying that SAV file from hard disk to your
backup media & theoretically could do more than one backup.

While testing the replacement drive last nite, I used both the MLRI tapes
that we use in our nitely backups & some old QIC that we use for selected
things like archiving history files & it was obvious big difference in speed
because the QIC does not have the servo track.

We use Mon Tue Wed Thu Fri tapes in rotation ... when there is some question
& we unsure if the problem is with the tape, the drive, the operator, other
players, we try again using another tape OTHER THAN the last good backup.
The last backup run before End Month is rotated off-site.
We have two off-site tapes so that we NEVER have all our backups in one place
at one time.

We got 16 Gig Tapes on various reccommendations of what made sense Dec 1999 &
our uncertainties about our own disk space growth, but we have been pretty
stable at 75% to 80% disk space consumption of 12 Gig hard drive for the past
year ... I have been aggressive about killing old unwanted records because I
have plans for some new applications that will need a bit more disk space.
Given that these Tapes will probably wear out long before we need more hard
drive, and we only using beginning part of the media, it makes sense to see
if pricing on 10-12 Gig tapes are more economical.

We had been sticking in the cleaning tape when the orange light goes on, but
CE said that the algorithm for the orange light is based on many factors &
every customer tape use is different so ideally we should be using the
cleaning tape more often, ideally before the orange light goes on.  The
problem is that the servo track gets contaminated & cleaning tape not help
... drive writes start point then tries to read that & can't & retries &
after more & more it fails ... there is no error message until failure but we
can hear it spending extra time getting ready.  For us this has been about
once every 10 working days.  I now plan to add this to my check lists so that
one day a week the tape will be cleaned, tentatively Thursday nite because
Friday is usually when end-month has extra tape usage.

Approx 15 years ago we converted to BPCS from MAPICS.

Now we are in a part of the country where the quality of public utilities is
not much better than in a banana republic.  A weather front comes through
about once a week & every time something goes down in association with that
... a phone line, a remote site's power, something.   PCs also go down a lot
more often than dumb terminals & we now have more people connected by PC than
by dumb terminal. BPCS 405 CD is extremely stable in the scenario of someone
in middle of updating something & their connection goes down ... there are
tools to recover from the damage from that particular operation.

That was not the case with MAPICS ... invariably if ONE person task went down
in middle of update, we had to restore the ENTIRE data base from last backup.
 Then people had to repost whatever transactions they had done since then.
This meant that we ALWAYS had to save our audit trails until the next backup.

We were restoring everything on the average of several times every month.
This meant everyone off the system for 1/2 a day while this was done, then
everyone a day behind in their work while they got caught up reposting
yesterday's stuff.

When a weather front brewing we would chase everyone off the system & if we
had to restore, there were private lynching parties for the individual who
was on the system when the power went out.

15 years ago UPS was much more expensive than today ... we have UPS on CPU &
modem rooms but still cannot justify UPS for every user dumb terminal & PC.

Typically I would ask management if I could delay restore until after the
weather front has passed because if power goes out in middle of restore, that
led to additional nightmare you would not believe ... I remember one time we
had to restart the restore 7 times in a 48 hour time period, had to go back
to grandfather backup, and the end users many many times restart reposting.

One weekend I was installing MAPICS software upgrades - the boss visited me
Sunday afternoon to see how I was doing & he could see I was dragging (I had
worked over Saturday nite non-stop with no sleep) - I said I had about 1/2
hour left to go then one more backup after that then I'd be done.  He talked
me into skipping the backup.  Monday after we had a power outage & I never
did get those software upgrades reinstalled.

At my previous employer, a truck dispatch company, they needed / wanted 24
hours 7 days a week up time, but agreed to taking the system down at midnite
for backup & we in IT were working 1st shift.  The janitor was trained in
running the backup & if ANY problems whatsoever, to CALL US at home ... I was
called in on the average 2-3 nites a week.

> From: brad@elstore.com (Brad Jensen)
>
>  I'm woindering about AS/400 backup solutions. I've got small
>  400s, but I'm wondering what it is like out there with bigger
>  ones. How much DASD are you using? How much backup are you doing
>  every night? (how many GB) Are you using save while active (I
>  assume most people today do.)
>
>  Do you do bigger weekly or periodic backups?
>
>  Have you had to do a restore in the last six months? Did it
>  interrupt or degrade your ability to accept new business?
>
>  I've heard one of the problems with partial restores is that you
>  have to start with tape one and go in sequence, even if it is on
>  tape five. Any way around that?
>
>  Do you backup during the day, or on second or third shift?
>
>  Are you using mutliple tapes? How many? A robot library?
>
>  Is anybody duplicating their backup tapes? Can you write two
>  tapes at once (each a copy of the other)?
>
>  Do you keep a backup set onsite and another offsite?
>
>  --
>  Brad Jensen brad@elstore.com
>  President
>  Electronic Storage Corporation Tulsa OK USA
>  918-664-7276
>
>  LaserVault Report Retrieval & Data Mining
>  www.Laservault.com
>
>  www.eufrates.com - Add distance learning to
>  your site with easy course preparation

MacWheel99@aol.com (Alister Wm Macintyre) (Al Mac)
BPCS 405 CD Manager / Programmer @ Global Wire Technologies Incorporated
http://www.globalwiretechnologies.com = new name same quality wire
engineering company: fax # 812-424-6838