Storage issue, but where?

["Patrick Trapp" <ptrapp@xxxxxxxxxxxx>]

I'm trying to figure out something without any success.  Here's hoping some
of our more experienced Domino for the iSeries admins can point me in the
right direction.  My apologies if this gets a little long.

I'm seeing a lot of CPF0907 errors in my logs -- "Serious storage condition
may exist. Press HELP" - in fact, there was one every hour all day
yesterday.  I didn't see the error in the logs until this morning when I
was troubleshooting something else.  The strange thing is that I ran a
WRKSYSSTS twice yesterday for other reasons and noted that the system
storage was right at 65% both times.  This morning, when the error was
still happening, it was still at 65%.  Our threshold is set at 90%, so this
error shouldn't be coming up, according to all documentation that I have
found.

One of our developers had some files in a directory in the IFS that were
generated early yesterday.  Since they are the only sizable changes that we
could find, we removed them from the system and the messages stopped
coming.  The thing is, the files were less than 2GB total, and there was
more than 50GB of space available before we hit threshold.

What's going on here?

Possibly related, and the reason I was looking at the logs closer, is that
our Domino server stopped responding early yesterday evening for no reason
that I can find.  I've seen Domino servers stop processing SMTP when the
storage reached 90%, but I've never seen the entire system stop.  Near as I
could tell, everything continued to function, but the processes were all at
0% in WRKACTJOB and there was not a single entry in the logs for over an
hour.  I had to restart the Domino server after an END *IMMED.  I'm still
trying to figure out what happened there, but the only errors I can see are
related to the END *IMMED, nothing jumps out as a reason for the Domino
stoppage.

Any thoughts, suggestions, pointers welcome.  I've been surfing redbooks,
but if you have a recommendation for a reference that might point me in the
right direction, I'm all ears.  Figuratively, of course.

Thanks for reading,
Patrick