Hi, Rob,
Thanks for sharing that data ...
This speaks volumes about the hazards of using the IFS to store massive numbers of small files, especially for "archival" purposes.
If you are truly just "archiving" the PC "directories of former associates" I think you would be far better served by using a zip utility to zip them all up into a single large zip file for each "former associate." That .zip file can still be stored on the IFS, but that way, it is just one large file, and you are not forcing the IBMi OS to maintain the associations (object ownership, authority, etc.) for thousands upon thousands of small IFS files thare are rarely needed.
When or if the need arises, you can easily open up such a zip file with any number of nice GUI tools to view and browse the contents, and then "extract" only those files that are needed.
This also speaks to the general recommendation (that applies equally well to QSYS.LIB objects as well as the IFS) of ensuring the each object has only its "owner" user profile, and *PUBLIC authority. Any other special "private" authorities should best be handled by using authorization lists, to the extent possible.
Using this approach, archiving many IFS directories that contains thousands, or even hundreds of thousands, of rarely used individual files, can also greatly reduce your overall back-up window, due to the benefits this has, since backing up a few large archive files in the IFS is far more efficient than having to "SAV" many thousands or millions of individual stream files in the IFS.
Hmm... this suggests a possible RFE to enhance the IFS to allow "mounting" a .zip file archive, as if it is part of the file system ... and providing transparent "read-only" access to those files? If read-write access is needed, perhaps such a proposed IFS "plug-in" could even "un-zip" those files to the corresponding directory structure automagically?
(Similar to how you can directly access most .zip archive files directly from e.g. Windows Explorer or similar tools on Linux, etc.)
All the best,
Mark S. Waterbury
On Monday, July 15, 2019, 12:25:40 PM EDT, Rob Berendt <rob@xxxxxxxxx> wrote:
Summary: SAVSECDTA jumped big time. How to resolve (if possible)?
We have an lpar which until recently was at this:
Total disk space on system in 1,000,000
bytes . . . . . . . . . . . . . . . . : 2153137
% of Size in
Description Disk 1,000,000 bytes
User libraries 2.03 43760.10
User directories 89.31 1922884.99
Folders and documents .00 .11
QSYS .10 2144.96
Other IBM libraries .53 11485.87
Licensed Internal Code .39 8299.99
Temporary space .46 9966.79
Unused space 7.08 152334.84
System internal objects .01 312.14
Objects not in a library .00 .02
QTEMP libraries .00 25.50
TOTAL 99.91 2151215.31
At this time a vast majority of the IFS stuff was LARGE Domino archive email files and their DAOS attachments.
Directory . . : /ARCHIVE3
Number of % of Size in
Description Objects Disk Megabytes
Total space used 204533 48.00 1822277.94
SAVSECDTA was under 2 minutes.
23:30:08 Starting SAVSECDTA to devices KDVLVTL.
23:31:49 All security objects saved.
23:31:50 Save security data (SAVSECDTA) complete.
Big project to store a lot more IFS stuff. As if 1.9TB of IFS wasn't enough.
All this new stuff was a large number of files. Basically the PC data directories of former associates. Individual file sizes vary greatly.
Added more disk to prep for this.
Now the system is at:
Total disk space on system in 1,000,000
bytes . . . . . . . . . . . . . . . . : 3980041
% of Size in
Description Disk 1,000,000 bytes
User libraries 5.25 208913.90
User directories 79.75 3174030.40
Folders and documents .00 .11
QSYS .09 3754.26
Other IBM libraries .28 11080.36
Licensed Internal Code .21 8361.01
Temporary space .34 13374.82
Unused space 14.02 558168.76
System internal objects .01 379.38
Objects not in a library .00 .02
QTEMP libraries .00 9.90
TOTAL 99.95 3978072.92
Basically just another TB of IFS and almost 2,000,000 more objects.
Directory . . : /home/FileShare
Number of % of Size in
Description Objects Disk Megabytes
Total space used 1730592 31.41 1192521.33
However the SAVSECDTA jumped from under 2 minutes to about an hour and a half. It was about twice that long until I removed the individual authority on the new files and replaced that with an authorization list.
Before Authorization list:
2:51:11 Starting SAVSECDTA to devices KDVLVTL.
5:42:58 All security objects saved.
5:42:59 Save security data (SAVSECDTA) complete.
After authorization list:
3:45:42 Starting SAVSECDTA to devices KDVLVTL.
5:18:09 All security objects saved.
5:18:11 Save security data (SAVSECDTA) complete.
Rob Berendt
--
IBM Certified System Administrator - IBM i 6.1
Group Dekko
Dept 1600
Mail to: 2505 Dekko Drive
Garrett, IN 46738
Ship to: Dock 108
6928N 400E
Kendallville, IN 46755
http://www.dekko.com
midrange.com
