Re: Backups taking longer after reorg

On 01-Jan-2016 08:57 -0700, Richard Reeve wrote:

<<SNIP>> - I wrote a program that scans through libraries and
reorgs files that have many deleted records. Sounds reasonable
enough.....However, since running this program, my backups are
running much longer. Why would this be? Is there something that
I can do to get the time needed to save these libraries back to a
reasonable timeframe? <<SNIP>>

If "libraries" means "all libraries", then best avoid the QSYS library and careful consideration on other /system/ libraries; esp. important, best to never issue the Reorganize Physical File Member (RGZPFM) request on the files QADB* in QSYS [except to recover from a defect or the specific effect of massive deletion of tracked information]. The quasi-system libraries such as QUSRSYS and QGPL that store user data, and thus are quasi-user libraries, are exceptions to that admonition.

If "many" does not consider also the percentage of delete records in the member, then the choice to reorganize the physical file member data may be at great cost for little gain.

So be sure to temper the requests in a manner that best ensures actual benefits, when weighed against the costs. To ensure that, an algorithm used to decide what to reorganize should be more intelligent than just a constant number of rows as the basis. Given the data is likely to be the fastest part of backups, reducing some megabytes even from each of thousands of files, is likely to effect only negligible reductions in overall time taken to effect saves; reducing the number of objects would be more beneficial.

A reorganize request that effects the rebuild of keyed Access Paths (ACCPTH) [of the Logical Files (LF) over the data] will have those LF indexes rebuilt in background QDBSRV## jobs, asynchronous to the RGZPFM requests. If the reorg activity precedes the backup activity, the CPU intensive access path rebuilds, even running at the lower run priority [RunPty=52 IIRC] than typical work, there will be a noticeable impact on that other work; that other work includes SAVLIB, SAV, et al.

I have considered saving the libs to a save file and making the
system available while backing the save files up to tape (which I'm
assuming would be faster given the I/O rate of disk vs. tape).

Unless the tape drives\media are ancient and slow, as I recall, the Save File (SAVF) on HDD disk [no idea on SSD] will be slower and of course with conspicuous impact on other work on the system for which concurrent access to the disks is required.

But I'm perplexed as to why the reorg would add time to my backup.

Difficult to know without actual investigation of the specific situation, but the async access path rebuilds alluded to both above and by Darryl are a possibility. Before staring the backup, the request to Edit Rebuild Access Paths (EDTRBDAP) would show pending rebuild activity [and WRKACTJOB JOB(*SYS) would show current rebuild activity]; there are also messages, IIRC CPF3145 and CPF3146, that are logged to the QHST History Log and therefore can be reviewed after-the-fact via the Display Log (DSPLOG) to span long after the save activity whereby any seen might indicate [per knowing which were the effect of the prior reorganize activity] that the rebuild work was being performed concurrent to the save activity.