Re: Large volume file move

I'll give a try to TAR or GNU-TAR (from YUM packages). You can preserve directory structures, security (I know, it's another OS) and some attributes from your files.

With GNU-TAR you will create just one package, then you can gzip this file and avoid on-the-fly compression won't be an issue and you can use FTP/SFTP/SCP or anything you want.

On Windows you can use WINRAR , P7-ZIP or any other package manager to extract data.

Regards

Diego E. KESSELMAN


On 25/09/2018 12:59, Richard Schoen wrote:

Actually ignore save/restore since you're on Windows already.

Here's a tongue-in-cheek use case for moving docs to the IFS:
"I want to move my image documents to my central IBMi system which will cost me more and cause my backups to slow down because I'm not comfortable with Windows."

Personally I think this is a misplaced exercise since you already have a reliable Windows solution. (Unless you are having server hardware issues and don't have a network person.)

Or if you're a Windows-hater, move to Linux and then you can be a Linux-hater too.

Either way you'll have cheaper storage and an OS which is meant to manage small files.

More importantly....Do you currently back up the image files ? I hope so......

Regards,
Richard Schoen
Director of Document Management
e. richard.schoen@xxxxxxxxxxxxxxx
p. 952.486.6802
w. helpsystems.com

------------------------------

message: 3
date: Tue, 25 Sep 2018 17:38:13 +0000
from: Justin Taylor <JUSTIN@xxxxxxxxxxxxx>
subject: RE: Large volume file move

We'll have a chance to test this with the POWER9 hardware before the actual migration. That should shake out any ownership issues.

How could I do a restore/save, I don't know of common format?




-----Original Message-----
From: Richard Schoen [mailto:Richard.Schoen@xxxxxxxxxxxxxxx]
Sent: Tuesday, September 25, 2018 12:19 PM
To: midrange-l@xxxxxxxxxxxx
Subject: RE: Large volume file move

I would expect to have object ownership issues when you move 20 million files owned by the same profile. Unless that's been fixed recently for the IFS.

Probably need to do a test perhaps. Perhaps multiple threads copying as well if you can break it down to a list of files. (perhaps list to a DB first.)

Also how do you know files don't get corrupted during copy. May need some sort of data comparison if those files are important to stay intact.

Save/restore to/from tape perhaps and then file comparison by hashing or reading bytes.

Or initial move to NAS/SAN as Rob mentioned.

Rsync also comes to mind.

Or.....leave them alone and back up your Windows server. IFS is not efficient with lots of small files.

Regards,
Richard Schoen
Director of Document Management
e. richard.schoen@xxxxxxxxxxxxxxx
p. 952.486.6802
w. helpsystems.com