Re: High priority I/O with IFS MOV

[DrFranken <midrange@xxxxxxxxxxxx>]

I believe we tried MV from QSHELL and it failed due to the different 
file systems. I will test that again.

        - Larry "DrFranken" Bolhuis

www.Frankeni.com
www.iDevCloud.com - Personal Development IBM i timeshare service.
www.iInTheCloud.com - Commercial IBM i Cloud Hosting.

On 12/1/2016 10:21 AM, Luis Rodriguez wrote:
> I wonder if using QShell's MV command would be faster?
>
> Regards,
>
> Luis
>
> On Thu, Dec 1, 2016 at 11:10 AM DrFranken <midrange@xxxxxxxxxxxx> wrote:
>
> > Customer has a process that accepts many (and I mean MANY) PDFs into the
> > iFS every day. During the month they go into a directory such as
> > '/www/zendsvr6/pdfdocs/abchst'.  They are referenced by the web server.
> > They are PDFs and never, ever changed or deleted.
> >
> > BUT at the end of the month they are moved by month into a dir like:
> > '/www/zendsvr6/pdfdocs/abchsta/2016/11/'
> >
> > They are moved one at a time because there is a control file with all
> > the names AND the process runs while the system is still up. The process
> > uses the MOV command to move the documents.
> >
> > In the past this worked great (for years now) and now on shiny new
> > Power8 it is very very slow. HOWEVER One other thing has changed.
> >
> > The target directory ('/www/zendsvr6/pdfdocs/abchsta/') is now mounted
> > in a user ASP. That ASP is on larger slower disks. This makes perfect
> > sense because those documents are rarely if ever read and never go away.
> > Total document count is approaching triple digit millions in there.
> > Normal busy level on those disks approaches zero, all reads. There are
> > thirty (30) disks in ASP(2).
> >
> > Now if it was only slow I would say: "Well yeah! It's actually COPYING
> > and DELETING the documents now because they are moving from *SYSBAS to
> > ASP 2, onto physically different disks, and they are slower." This would
> > make perfect sense and actually we wouldn't care much. The process
> > changes the control file as it goes so anyone looking them up gets the
> > document whether it has moved or not.
> >
> > The problem is it's killing system performance overall. Even though this
> > process is running at priority 80 if you watch WRKSYSACT it constantly
> > is swapping to priority TEN (10).  Overall cpu is well below 30% (of
> > four cores). There is a large amount of faulting and paging as well
> > which I think would be expected since all the documents are actually
> > being copied. The job currently run in a separate subsystem but is using
> > *BASE memory. *BASE is 1/4TB of memory.
> >
> > Any thoughts on how to reduce the impact to the system here?  Would a
> > CPY followed by a DEL be better than a MOV?
> >
> > Would reducing the jobs timeslice help? It's at default of 5000.
> >
> > --
> >
> >          - Larry "DrFranken" Bolhuis
> >
> > www.Frankeni.com
> > www.iDevCloud.com - Personal Development IBM i timeshare service.
> > www.iInTheCloud.com - Commercial IBM i Cloud Hosting.
> > --
> > This is the Midrange Systems Technical Discussion (MIDRANGE-L) mailing list
> > To post a message email: MIDRANGE-L@xxxxxxxxxxxx
> > To subscribe, unsubscribe, or change list options,
> > visit: http://lists.midrange.com/mailman/listinfo/midrange-l
> > or email: MIDRANGE-L-request@xxxxxxxxxxxx
> > Before posting, please take a moment to review the archives
> > at http://archive.midrange.com/midrange-l.
> >
> > Please contact support@xxxxxxxxxxxx for any subscription related
> > questions.