On 18-Dec-2014 16:06 -0600, Thomas Garvey wrote:
We have a program that writes records into an externally defined
database file that has received the following error...
<<SNIPped joblog: rewritten for compactness, symptoms,
and adding some explanation>>
MCH3203 DbReuseFind 000FB4 QDBPUT 06B1
12/18/14 08:52:28.640853
Message . . . . : Function error X'1720' in machine instruction.
Internal dump identifier (ID) 0300167E.
msgMCH3203 f/DbReuseFind x/000FB4 T/QDBPUT x06B1
rcx1720 rc1720 vl02001720
The LIC DB code that attempted to locate a location for a [set of]
deleted record(s) failed; a VLIC Log (VLog) was generated [major code
0200 for "function check" and minor code 1720 [for an effective "failed
assumption] that is logged to record the failure. The failed attempt
was a "put", aka a write or insert; intuitively, the dataspace appeared
to support Reuse Deleted Records (REUSEDLT), and thus the LIC DB was
looking for a place to insert the record(s) into a slot held by a
since-deleted row. Something about this search went awry, the error
which likely is an assertion that something should have been, but was
found not to have been [i.e. failed assumption], so rather than
progressing in spite of the bad information [which could have led to
GIGO], the operation was canceled to protect the integrity of the data.
CPF9999 QMHUNMSG *N QDBPUT 06B1
Message . . . . : Function check. MCH3203 unmonitored by QDBPUT
at statement *N, instruction X'06B1'.
msgCPF9999 F/QMHUNMSG t/QDBPUT x/06B1
The LIC DB module that was invoked to implement the /insert-row/
method is, at that fix-level of the OS DB program QDBPUT, occurs at the
failing instruction x/06B1 [for whatever specific type of write
activity, according to the code path].
The generic "function check" condition, aka the *FC, indicates that
the OS Database "Put" program did not monitor for the failure... which
is correct, there is really nothing the DB2 Insert feature can do when
the underlying database support fails, so the message be manifest as an
"unmonitored failure" which also enables some internal diagnostic
messaging support to better record the failure.
CPF3698 QSCPUTR 005E QMHAPD 0500
Message . . . . : Dump output directed to spooled file 1,
job 875639/JOBNAME/E206151053 created on system ITEAM on...
msgCPF3698 F/QSCPUTR x/005E T/QMHAPD x/0500
As part of Software Logging [SC: ¿Service Component?] for software
errors, the prior /unmonitored/ condition is further logged as a
"software problem". Really not germane, just a record that the OS was
told to log, so that was done, and the effect was logged in the joblog.
CPF5257 QDBSIGEX 0539 E20615105R WORKLIB STMT/2123
Message . . . . : Failure for device or member E206151053 file
E206151053 in library WORKLIB.
Cause . . : An error occurred during a read or write operation.
msgCPF5257 F/QDBSIGEX x/0539 t/usrpgm
The user program issuing the failing statement, a WRITE, INSERT,
whatever, is: E20615105R in WORKLIB STMT/2123
The failed I/O must be informed of, to the requesting program. The
I/O error is signaled by the generic "Signal Exception" routine of the
OS Database component, using the Common Data Management [DM] message
range of messages, because the I/O was via the Open Data Path (ODP)
created for the Database Open request; the member, file, and library,
are offered up as part of the information, and that the operation was a
"write" vs a read can be inferred from the OS DB program name.
I examined the file and could not determine if it was damaged. Then
I noticed the 'DbReuseFind' reference (in the original MCH0203
message) which made me check the maximum file size, and number of
deleted records, as I know the file is configured to 'reuse deleted
records'. The file had some 2,500,000 records, and slightly more than
1,000,000 were deleted records, and the file has *NOMAX for Member
Size. So, I reorganized the file, just to see what would happen. It
reorganized just fine, eliminated the deleted records, and I
restarted the job.
The Reorganize Database File Member (RGZPFM) request will purge the
deleted records, and thus purge the reuse-table; i.e. there will be no
table in which deleted rows are tracked, thus the database insert\put
will have no reason to look for the desire or ability to replace a
deleted row with an active row.
Thus, the problem was "circumvented"; the origin of the problem is
not [yet] investigated, and the details of the condition only exist in
the logging, because the object [the dataspace] against which the
failure transpired, no longer exhibits the problem.
Now the job runs fine.
*What do you think happened?*
The VLogs, the VL02001720 and possibly some nearby [esp. just before]
in time, plus possibly some longer ago in the past for the object of the
same address, may contain some information relevant to answering that
question. What is clear however, is that the LIC DB was not happy about
what transpired at that moment with that dataspace with respect to
processing for the REUSEDLT(*YES), and the operation was failed\canceled
with purpose.
BTW, another job, which only deletes records from this file, was
failing with the same Machine Check error, with only a slightly
different Error Code (hex 1716 instead of 1720, as above)
<<again; rewritten for compactness, symptoms, explanation>>
12/18/14 08:52:28.640853
MCH3203 #dbdelim 00343C QDBUDR 0C1D
12/18/14 08:54:09.273373
Message . . . . : Function error X'1716' in machine instruction.
Internal dump identifier (ID) .
msgMCH3203 F/#dbdelim x/00343C T/QDBUDR x/0C1D
rcx1716 rc1716 vl02001716
Not sure why the DMPID did not appear in the message data; that may
itself [and seems to] be a defect.
This time the request was from the OS DB Delete [, Update, Release;
i.e. the "UDR" of the program naming] processor, a request to the DB LIC
method to Delete a Dataspace Entry [DB Delete Image is term for
¿DBdelim?]. Presumably this processing some ~100seconds later
encountered what was effectively /the same/ issue with the ReUse
feature, but from a different perspective; delete instead of insert.
Quite possibly, like alluded for the other failure, the condition would
have persisted.
Notice the only difference is the reference to #dbdelim on the
MCH3203 and the hex error code, now '1716' instead of '1720'.
I am now unsure if I recall the assert\assume minor code for the
Function Check [major code 0200] Vlog; perhaps that is 1716 vs 1720, or
perhaps both may be used in that manner. No matter, this is another
failure for which the database I/O processing was terminated.
Both jobs are running fine now, after the reorg.
*Any thoughts?*
Again, the issue that had [presumably persisted at\from the time of
the first logged incident] transpired was circumvented by that action.
The origin for the issue and any underlying issues that might lead to
the same failures may persist; e.g. code defects, for which the
preventive [and\or corrective] PTFs await being installed, or for which
the problem remains unreported for which such PTF(s) can be generated or
the problem report [APAR] answered [perhaps as unreproducible or as ??].
I seem to recall some number of releases ago, when files could reach
a maximum number of deleted rows. I do not recall if there was a
corrective fix that did not require RGZPFM, CLRPFM, or re-create of the
dataspace, but a design enhancement enabled multiple re-use spaces to
allow for a huge number increase in the mapping\tracking of deleted
records. Possibly the dataspace for the file [the data portion,
associated with] the particular member against which the error
transpired was created on an old release and some corrective action
remained awaited until the member could support the larger number of
deleted rows. That release and the release of the OS being utilized I
do not think were mentioned in the OP. If the Analyze Problem (ANZPRB)
as alluded in the messaging were followed and that led to an issue\PMR
being opened with the Service Provider [and IBM Support looked at the
collected Vlogs], they might be able to determine that the problem was
due to the limit imposed on a down-level object, and that limit has
since been increased, and that what was used as the circumvention might
even have been corrective... or they might determine the issue is a
defect [that has or awaits a PTF].
midrange.com
