Re: Failed GO SAVE 21 -- MIDRANGE-L

I thought of a stress test. Two weeks now, the GO SAVE 21 has ended
abnormally on the last command, SAV DEV('/QSYS.LIB/TAP35.DEVD') OBJ('/^')
('/QSYS.LIB' *OMIT) ('/QDLS' *OMIT) ('/QNTC' *OMIT) ENDOPT(*UNLOAD)
UPDHST(*YES)

This morning, I grabbed tape 2 from failed backup Wednesday morning.
Verified that it was indeed devoid of any information.

Then proceeded to run the above command, changing ENDOPT to *LEAVE. Idea
was to write to tape something big and be able to repeat enough times to
fill the tape.

I cleaned the drive, per request of system last week. Running the SAV
command four times filled up the tape. I did receive a message requesting
another tape. QSYSOPR also had a message a tad earlier advising the drive
needed to be cleaned. I aborted that SAV, ran the cleaning tape, and
initialized the tape. Then repeated the SAV four more times and received
the next tape request. Cancelled the SAV and initialized the tape.

On one trip back yp the computer room, well before I sawthe message that
drive was dirty, I saw CLN in small letters on the display panel. After
running the cleaning tape, that went away.

One thought I had that prompted this torture test was that either drive was
not detecting end of tape, or the SAV command did not check or could not be
interrupted. So, I have now learned that drive does detect EOT, and SAV
command also gets that status and handles it.

None of this explains why backup has failed the past two times.

Looking around within SST, I never saw anything specific to tape.
Abxolutely no guarantee that I am looking in the right places with SST.

Anyway, if anybody has other ideas, I would appreciate suggestions.

John McKee

On Wed, Aug 26, 2015 at 3:59 PM, <rob@xxxxxxxxx> wrote:

If you have a day or two to prepare for this you may want to INZTAP with
CLEAR(*YES) on two tapes and then use those. That will overwrite
EVERYTHING on those tapes and really put them to the test.

Rob Berendt
--
IBM Certified System Administrator - IBM i 6.1
Group Dekko
Dept 1600
Mail to: 2505 Dekko Drive
Garrett, IN 46738
Ship to: Dock 108
6928N 400E
Kendallville, IN 46755
http://www.dekko.com

From: John McKee <jmmckee3@xxxxxxxxx>
To: Midrange Systems Technical Discussion <midrange-l@xxxxxxxxxxxx>
Date: 08/26/2015 04:55 PM
Subject: Re: Failed GO SAVE 21
Sent by: "MIDRANGE-L" <midrange-l-bounces@xxxxxxxxxxxx>

Hneeding to be changed.ave not tried other saves. Did INZTAP yesterday on
tapes for today. I can find a tape tomorrow and try a save. But, I think
it is somehow related to the tape. Still a worthwhile experiment.

Looking at option 20 now. Message queue delivery is *BREAK. Use system
reply list is N. No changes from two weeks ago to any values.

John McKee

On Wed, Aug 26, 2015 at 3:44 PM, Steinmetz, Paul <PSteinmetz@xxxxxxxxxx>
wrote:

1) Storage management is normal.
2) Did you try any other saves besides a GO SAVE 21?
3) Do you check your option 20 settings?

Paul

-----Original Message-----
From: MIDRANGE-L [mailto:midrange-l-bounces@xxxxxxxxxxxx] On Behalf Of
John McKee
Sent: Wednesday, August 26, 2015 4:39 PM
To: Midrange Systems Technical Discussion
Subject: Re: Failed GO SAVE 21

I was told we have hardware support until end of year.

On 3590 panel, I saw an entry to do a reset. When I did that,

diagnostics

ran. Nothing displayed as if an error had been found.

I called IBM. I was sent to a hardware tech. He said the LIC log in

SST

was software, not hardware. The two options he worked with as far as
hardware logs showed no entries.

He suggested getting current on PTFs, maybe moving to v5r4m5. While we
were talking, two more entries were added to the log, both are described

as

"Storage Management". They were posted less than 30 seonds apart.

Nothing

else has been posted in the last 30 minutes.

If this is PTF related, it definitely goes against the mantra of "if it
ain't broke...". Is there any way to know if it is a PTF issue? System

is

pretty much idling now, but might still be some rumbling if it were to
become unavailable. And, my ancient fears of the dreaded "Link Loader
error" remain. I know it has been said that "Link Loader" error was

fixed

a long time ago - by people on the list who definitely know.

Still, if it is software, since no changes have been made to hardware
since a disk drive failed months ago, I can't help but wonder why a tape
change is now suddenly an issue.

I was told that even without software support, I can order PTFs. That,
from a hardware guy.

Is the answer as simple as "just do it"? And the next GO SAVE 21 will
magically be OK?

John McKee

On Wed, Aug 26, 2015 at 1:21 PM, <rob@xxxxxxxxx> wrote:

It has been a LONG time since I've had a 3590. My evolution on AS/400
(then IBM i) tape was "something" like:
9348
QIC (various sizes)
3570
3580
3590
LTO1
LTO3
LTO4
with an 8mm thrown in there somewhere but we maybe only used it for
vendor software. I can't remember.

I can't remember if the 3590 had a menu system to power it down or
some such thing. My LTO libraries seem to have such a thing. But
even so, I often use the web interface to it instead.

Rob Berendt
--
IBM Certified System Administrator - IBM i 6.1 Group Dekko Dept 1600
Mail to: 2505 Dekko Drive
Garrett, IN 46738
Ship to: Dock 108
6928N 400E
Kendallville, IN 46755
http://www.dekko.com

From: John McKee <jmmckee3@xxxxxxxxx>
To: Midrange Systems Technical Discussion

<midrange-l@xxxxxxxxxxxx>

Date: 08/26/2015 02:12 PM
Subject: Re: Failed GO SAVE 21
Sent by: "MIDRANGE-L" <midrange-l-bounces@xxxxxxxxxxxx>

I can certainly agree that the final save could well be needed.

I am looking at Licensed Internal Code Log. A lot of entries. Not
sure how far back they go. Looks like older than 2014 at least.

My backup this morning resulted in 13 entries:

Source/Sink : 6
Tape support: 5
Damage set: 1
Error analysis: 1

And, two storage management entries from 12:11 and 12:40

I have no idea what the last two entries might be related to. But,
there are 46 entries just from 8/19/15 - and a bunch more further
back. Also DASD support on 08/19/15.

I wish I knew what this was about.

System has not been IPLed for a few months.

To reboot 3590 - is that just cycle the power switch? I hate asking
such naive questions.

John McKee

On Wed, Aug 26, 2015 at 12:14 PM, <rob@xxxxxxxxx> wrote:

You could try rebooting 3590 and the IBM i and see if that helps.

You might want that one final save...

Rob Berendt
--
IBM Certified System Administrator - IBM i 6.1 Group Dekko Dept 1600
Mail to: 2505 Dekko Drive
Garrett, IN 46738
Ship to: Dock 108
6928N 400E
Kendallville, IN 46755
http://www.dekko.com

From: John McKee <jmmckee3@xxxxxxxxx>
To: Midrange Systems Technical Discussion

<midrange-l@xxxxxxxxxxxx

Date: 08/26/2015 12:40 PM
Subject: Re: Failed GO SAVE 21
Sent by: "MIDRANGE-L" <midrange-l-bounces@xxxxxxxxxxxx>

Maintenance was dropped by IBM on 3590. Maybe the point of backups
is past anyway. System was "frozen" earlier in the week. No new
data will be written. Just gets shut down at some point - likely
when something breaks.

I ust thought it odd that a tape change request never came.

Can't help but wonder if this might be another of those stupid "if
it ain't broke" things where a problem might have been identified
and a PTF was prepared. Just not fixed here.

Thanks anyway.

John McKee

On Wed, Aug 26, 2015 at 11:31 AM, DrFranken <midrange@xxxxxxxxxxxx>

wrote:

Smells more like an I/O failure to the drive or an actual error at
the drive. Honestly unless you are a very large shop we don't see
many

3590s

out there any more, they are getting a bit long in the tooth.

Have IBM help you look in PAL and SAL and other logs, I bet
they'll

find

something in there.

- Larry "DrFranken" Bolhuis

www.Frankeni.com
www.iDevCloud.com - Personal Development IBM i timeshare service.
www.iInTheCloud.com - Commercial IBM i Cloud Hosting.

On 8/26/2015 11:33 AM, John McKee wrote:

It happened again. Runs along fine, then aborts.

I did get a job log.

System also created QPSRVDMP.

The 3590 device is in FAILED state.

From looking at QEZDEBUG, I see from the call stack, the last
three

entries

are:

qsrProcessIoReqForGroup_FP7qsrList
qsrSwitchMediaVolume_FP15qsrIO_Request_T13qsrTypeOfFEOV
qsrFeovDevUfcb_FP9qsrFileCb

From looking at QSYSOPR, there is CPI5922 Device description
TAP35 is

not

usable at this time. No other error message related to the drive

before

or
after that message.

From QPSRVDMP

DUMP TAKEN FOR UNMONITORED ESCAPE MESSAGE

MCH5203 Condition of source/sink object TAP35 not valid.

System (v5r4) has 246.1G and %system ASP used is 67.4779

My WAG is that the tape got full. But, unlike last week, I never
saw

a

message requesting a tape change. Last week, I did see that
message

along

with a message that tape drive needed to be cleaned - which I did

last

week.

So, any idea as to why there was no tape change message?

John McKee

--

This is the Midrange Systems Technical Discussion (MIDRANGE-L)
mailing

list

To post a message email: MIDRANGE-L@xxxxxxxxxxxx To subscribe,
unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/midrange-l
or email: MIDRANGE-L-request@xxxxxxxxxxxx Before posting, please
take a moment to review the archives at
http://archive.midrange.com/midrange-l.

--
This is the Midrange Systems Technical Discussion (MIDRANGE-L)
mailing list To post a message email: MIDRANGE-L@xxxxxxxxxxxx To
subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/midrange-l
or email: MIDRANGE-L-request@xxxxxxxxxxxx Before posting, please
take a moment to review the archives at
http://archive.midrange.com/midrange-l.

--
This is the Midrange Systems Technical Discussion (MIDRANGE-L)
mailing

list

To post a message email: MIDRANGE-L@xxxxxxxxxxxx To subscribe,
unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/midrange-l
or email: MIDRANGE-L-request@xxxxxxxxxxxx Before posting, please
take a moment to review the archives at
http://archive.midrange.com/midrange-l.

--
This is the Midrange Systems Technical Discussion (MIDRANGE-L) mailing
list To post a message email: MIDRANGE-L@xxxxxxxxxxxx To subscribe,
unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/midrange-l
or email: MIDRANGE-L-request@xxxxxxxxxxxx Before posting, please take
a moment to review the archives at
http://archive.midrange.com/midrange-l.

--
This is the Midrange Systems Technical Discussion (MIDRANGE-L) mailing
list To post a message email: MIDRANGE-L@xxxxxxxxxxxx To subscribe,
unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/midrange-l
or email: MIDRANGE-L-request@xxxxxxxxxxxx Before posting, please take
a moment to review the archives at
http://archive.midrange.com/midrange-l.

--
This is the Midrange Systems Technical Discussion (MIDRANGE-L) mailing
list To post a message email: MIDRANGE-L@xxxxxxxxxxxx To subscribe,
unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/midrange-l
or email: MIDRANGE-L-request@xxxxxxxxxxxx Before posting, please take a
moment to review the archives at http://archive.midrange.com/midrange-l.

--
This is the Midrange Systems Technical Discussion (MIDRANGE-L) mailing

list

To post a message email: MIDRANGE-L@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/midrange-l
or email: MIDRANGE-L-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/midrange-l.

--
This is the Midrange Systems Technical Discussion (MIDRANGE-L) mailing
list
To post a message email: MIDRANGE-L@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/midrange-l
or email: MIDRANGE-L-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/midrange-l.

--
This is the Midrange Systems Technical Discussion (MIDRANGE-L) mailing list
To post a message email: MIDRANGE-L@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/midrange-l
or email: MIDRANGE-L-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/midrange-l.