RE: Copying from/to qntc performance

Thanks Larry, I'll also be checking NFS and *maybe* Aspera.

FTP was my first thought but they had already started the project with the QNTC share before they involved me. (-:

Alan



-----Original Message-----
From: MIDRANGE-L [mailto:midrange-l-bounces@xxxxxxxxxxxx] On Behalf Of DrFranken
Sent: Monday, June 29, 2015 1:49 PM
To: Midrange Systems Technical Discussion
Subject: Re: Copying from/to qntc performance

Some quick math. Transferring 1TB of stuff across a GigE line were you able to blow it through at 100% of that 1Gb bandwidth would still take
2.22 hours WITH NO OVERHEAD. In other words just the data bits.

Add IP overhead etc and you're more likely in the 3 hour range just to start. Then as Matt said in his followup the IBM i code for Netserver is rather old but in general windows file sharing simply is not the way to go for large transfers.

I would look to NFS or even FTP both of which utilize bandwidth much much more efficiently than does any version of Samba or CIFS. Also note that QFileSvr.400 while very useful is also bad for large transfers.

As to Aspera they us UDP to be absolutely positive that every bit of available bandwidth is utilized. What I'm not sure of (but I think not) is if they have a client for IBM i. Then you need to consider if it's acceptable for this process to consume 100% of available bandwidth or if others need their share.

- Larry "DrFranken" Bolhuis

www.frankeni.com
www.iDevCloud.com
www.iInTheCloud.com

On 6/29/2015 1:25 PM, Alan Cassidy wrote:

Okay, what finally worked to do this to back it up was a CPYTOSTMF as Scott Klement suggested from the save file, with the DBCCSID = 37 and the remote CCSID = 1252.



One gotcha for me was that even though I was able to create a directory QNTC/11.12.13.14, this connected to nothing because my user-ID was not the one with the remote credentials.



To clarify one of my confused notes earlier, the USERID(remoteuser) parameter worked to get to the remote server.



PERFORMANCE



Now it's an issue of bandwidth. (I think)



This copy is going to a box sitting in a data center, over a dedicated 1G line that thinks it's all local.



It's actually some 250 save files, and they total up to over 1 terabyte, and it's taking way too long to copy them all.



Our manager said IBM has a product that breaks up a big file into parts and transfers them much faster.



I did a search and found IBM's "Aspera High-Speed File Transfer" here:

http://www-03.ibm.com/software/products/en/high-speed-file-transfer



Does anyone have experience with this or is there another suggestion, and does it cost "reasonably" for a small fast-growing shop?



Alan









On 6/5/2015 2:34 PM, Scott Klement wrote:

Tim,

Thanks for letting us know what ended up working for you.

For my own information, some questions if you don't mind:

1) did you try CPYFRMSTMF as I suggested and find that it did not

work? I know that CPYFRMSTMF is often used to populate save files.

2) Why did you specify TOCCSID(*CALC) on your CPY command? I suspect

it's ignored because DTAFMT(*BINARY) is the default on the CPY

command. But to someone who doesn't know that, the TOCCSID(*CALC)

would seem like a curious thing to do. Translating save file data

between CCSIDs makes no sense.

Alan Cassidy
Softare Engineer
Direct Phone - 954-693-0000 Ext. 3433
ACassidy@xxxxxxxxxxxxxx<mailto:ACassidy@xxxxxxxxxxxxxx>
www.hppartners.com<http://www.hppartners.com/>

[HPP Logo FINAL with name-tag-logo-TM]



----------------------------------------------------------------------
---------- Confidentiality Notice: This email may contain confidential
information or information covered under the Privacy Act, 5 USC 552(a), and/or the Health Insurance Portability and Accountability Act (PL 104-191) and its various implementing regulations and must be protected in accordance with those provisions. It contains information that is legally privileged, confidential or otherwise protected from use or disclosure. This e-mail message, including any attachments, is for the sole use of the intended recipient(s). Any unauthorized review, use, disclosure or distribution is prohibited. You, the recipient, are obligated to maintain it in a safe, secure and confidential manner. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. Thank you.
----------------------------------------------------------------------
----------

--
This is the Midrange Systems Technical Discussion (MIDRANGE-L) mailing list To post a message email: MIDRANGE-L@xxxxxxxxxxxx To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/midrange-l
or email: MIDRANGE-L-request@xxxxxxxxxxxx Before posting, please take a moment to review the archives at http://archive.midrange.com/midrange-l.


--------------------------------------------------------------------------------
Confidentiality Notice: This email may contain confidential information or information covered under the Privacy Act, 5 USC 552(a), and/or the Health Insurance Portability and Accountability Act (PL 104-191) and its various implementing regulations and must be protected in accordance with those provisions. It contains information that is legally privileged, confidential or otherwise protected from use or disclosure. This e-mail message, including any attachments, is for the sole use of the intended recipient(s). Any unauthorized review, use, disclosure or distribution is prohibited. You, the recipient, are obligated to maintain it in a safe, secure and confidential manner. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. Thank you.
--------------------------------------------------------------------------------