Re: RGZPFM while active issue - REORGANIZE RETURNED LESS STORAGE THAN ESTIMATED

On 05-Sep-2014 10:31 -0500, Steinmetz, Paul wrote:

I had a repeat of this issue.
From old PMR, here is the IBM doc summarizing the issue.
Previously, I had the issue at V6R1.
Now on V7R1.
Bottom line, even though the RGZPFM completed with no errors, space
is not returned for some files.
Anyone else seeing this issue?

Was the Display Journal (DSPJRN) output reviewed to determine if any jobs [other than the job requesting the Reorganize Physical File Member (RGZPFM)] had inserted rows [and\or that were then committed or rolled back; those under isolation could be rows that had caused locking conflicts with the reorg] for the file being reorganized?

*NONE Command 09/04/14 17:26:04.085501 QCADRV QSYS 03BA
RGZPFM QGPL 0062
Message: 5500 - RGZPFM FILE(CURBSTONE/CCFP4000) MBR(CCFP4000)
RBDACCPTH(*NO) ALWCANCEL(*YES) LOCK(*SHRUPD)

Does the Key File (KEYFILE) parameter [unspecified in the request] ask to effect the Replace Deleted Records (*RPLDLTRCD), or effect the shipped-default of *NONE? Both to maximize performance and [thus\AFaIK] to assist consolidating rows into their own segment that can be truncated, The *RPLDLTRCD feature should be utilized; note the caveat regarding arrival sequence access path dependencies.

*NONE Diagnostic 09/04/14 17:38:26.730345 QDBFFCPY QSYS 0181
RGZPFM QGPL 0062
Message: REORGANIZE RETURNED LESS STORAGE THAN ESTIMATED.

If the DSPJRN identifies such work, that could easily be for what the TechDoc alludes would explain the condition being diagnosed. Albeit that doc states CPF9898 will be issued, but the above [reformatted] joblog seems to show instead an *impromptu* message being issued [per the *NONE MsgId sent as a Diag], though perhaps that was instead, the alluded "general status message" [for which the term "status" for conspicuous reasons, should have been avoided]?

The TechDoc remains for context, for my

IBM Software Technical Document
Document Number: 631414267
<<SNIP>>
Document Title:Basics on Reorganize While Active
"...

When using *SHRUPD lock, it is important to understand how
production jobs inserting into the file will affect the RGZPFM job
and its ability to return space. After the reorganize job runs
through the file one time, it will look to see if any additional rows
have been inserted at the end of the file since the reorganize
started. The job will note what rows need to be moved and try to move
them to get as much space returned as possible. If a row gets
inserted at the end of the file after the last rows that needed to be
moved were noted, the system will not be able to give back as much
space. More activity on the file (inserts) can prevent space being
returned. Files set up with Reuse Deleted Records *NO requires all
inserts to be placed at the end of the file. This increases the
chance of limited space being returned. Files that are set up to
Reuse Deleted Records *Yes will have a greater chance of production
jobs inserting rows in the middle of the file and not impacting the
RGZPFM job. There is still a chance that rows can get inserted at the
end and prevent space being returned even with Reuse Deleted Records
turned on.

For this reason, you may need to run the RGZPFM several times in
order to see space returned. In some cases, a maintenance window is
needed to run RGZPFM with no inserts happening. If this is the case,
let the reorganize run during the week and move as much as it can and
do much of the work. You should perform this work as close to the
maintenance window as you can and, if it ends earlier, start another
one. Then, during the down time, the reorganize will have less work
to do, and down time should be decreased. There is no way to estimate
how long the RGZPFM will run.

Use the journal to determine activity that is occurring on the file
while the RGZPFM is running (DSPJRN).

At R710 with DB group 24, message (CPF9898) and a general status
message will be issued with the status of this last phase of giving
back space.

Prior to R610, the reorganize job would have to get an exclusive
lock on the file at the end to give any space back. At R610 and
later, this is no longer needed."

Lastly, when Reorganize While Active is used, the system ends at a
16 MB segment boundary. This means that for smaller files or files
with little spaces used for deleted records, you may see the number
of records set to 0; however, no space is returned.

Tips to help improve performance:

1. Use RGZPFM KEYFILE(*RPLDLTRCD). This may reduce the amount of
rows that are needed to be moved. This option should not be used if
arrival sequence is needed for the file.

..."

Best I can infer from what is stated in that document is that the impromptu message intends to suggest that the dataspace segment for which there should exist only deleted rows [that are not there under isolation scoped outside the commit definition for the RGZPFM request] upon the effective completion of the movement of rows, was unable to be achieved; i.e. the feature was unable to gather all of the deleted rows into a single segment [effectively, the end of the file] which could then be pruned from the composite of data segments that comprise the LIC Dataspace object. The document alludes that concurrent work is a likely cause, and that the DSPJRN could assist to diagnose what work was in conflict; and probably any logged conditions of CPF5027 "Record &6 in use by job &9."

In prior releases, any locks held on the dataspace would have prevented the pruning of the segment because the feature [implemented at a user-code level; i.e. nothing proprietary to the OS] used normal locking protocol to obtain an exclusive lock to accomplish the task. Whereas since v6r1, the LIC database may be able to effect the prune while other jobs hold locks [by being the arbiter of locking and thus able to break standard locking protocols], but conspicuously the database is unable to purge that segment when an active record [not deleted or pending action under isolation] had been added to that segment concurrent to the delete\re-insert activity [being performed outside the LIC via standard I\O performed under commitment control].