RE: [EXTERNAL] Re: Duplicating backup to Amazon S3

Brad and Diego,

Thanks for the information and ideas.


Dean Eshleman
Software Development Architect
Everence

1110 North Main Street
PO Box 483
Goshen, IN 46527
(800) 348-7468 ext. 3528
(574) 533-9515 ext. 3528
[Join our Talent Network]<http://www.everence.com/careers>

From: MIDRANGE-L <midrange-l-bounces@xxxxxxxxxxxxxxxxxx> On Behalf Of Diego E. Kesselman
Sent: Thursday, October 6, 2022 1:59 PM
To: Midrange Discussion <midrange-l@xxxxxxxxxxxxxxxxxx>
Subject: [EXTERNAL] Re: Duplicating backup to Amazon S3

This email originated outside of Everence. Do not click links or attachments unless you recognize the sender.

Hi,

We have a solution too, and you could duptap the tapes to an Image Catalog and upload to S3 really fast (I suggest WASABI to avoid EGRESS charges).

And you can upload using a simple CL, but when you need to automate restores that can be complex. That’s the reason to use commercial products like BRMS.

If you want to build a DIY solution you could use a Linux server to upload you Virtual Tape Image Catalog, upload using FTP, compress on the Linux server using ZSTD (really fast) and upload files at the same time with some multipart tuning similar to this:

[default]
region = us-south
output = json
s3 =
max_concurrent_requests = 20
max_queue_size = 10000
multipart_threshold = 256MB
multipart_chunksize = 128MB


Note: You can also use AWS CLI and ZSTD from your IBM, just installing Python3 and ZSTD with YUM or ACS and running on your SSH session (need to adjust the PATH):

Compress:

cd /IMGCLG
zstd *

pip3 install awscli

Once installed I use an alias with BASH, in this case to connect with WASABI (same protocol used on IBM Cloud Object Storage or AWS S3):

alias ws3='aws s3 --endpoint-url=https://s3.us-central-1.wasabisys.com<https://s3.us-central-1.wasabisys.com> --profile wasabi'

And now I can connect to the cloud with easy PASE commands:

ws3 ls s3://mybucket/ <s3://mybucket/>
ws3 cp /IMGCLG/ s3://ibmi01/20221006/ <s3://ibmi01/20221006/> —recursive

Good luck!

Diego E. Kesselman
diegokesselman@xxxxxxxxx<mailto:diegokesselman@xxxxxxxxx>

El 06/10/2022, a las 12:21, Brad Stone <bvstone@xxxxxxxxx<mailto:bvstone@xxxxxxxxx>> escribió:

<vendor>
We have products that allow you to send files to Google Drive or Microsoft
OneDrive/Office 365. I personally use it for my backups, but of course I
don't have terabytes of data, so the bottleneck would be the connection and
the size of data you're backing up.

www.bvstools.com/g4g.html<http://www.bvstools.com/g4g.html>
www.bvstools.com/g4ms.html<http://www.bvstools.com/g4ms.html>

</vendor>

Of course, if you do have terabytes of data, physical storage may be good
even if you find a cloud solution.

On Thu, Oct 6, 2022 at 12:14 PM Dean Eshleman via MIDRANGE-L <
midrange-l@xxxxxxxxxxxxxxxxxx<mailto:midrange-l@xxxxxxxxxxxxxxxxxx>> wrote:

Hi,

We currently do backups to a Data Domain VTL using BRMS. Our weekly
backup we duplicate to physical tape using the DUPMEDBRM command. Our
management would like to get rid of the physical tape and wondered if we
could duplicate the backup to Amazon S3 or another cloud storage option. I
assume this is possible. I ran across the "Cloud Storage Solutions for I"
product. I did notice the limitation of 2TB per day. I guess it would
also depend on our internet connection. Is that the product we would need
to do this? Are there other options? Currently, I'm not sure how big our
backup is. I'll check into that. We are on 7.3 and up to date on PTF's as
of July this year. TIA

Dean Eshleman
Software Development Architect
Everence

1110 North Main Street
PO Box 483
Goshen, IN 46527
(800) 348-7468 ext. 3528
(574) 533-9515 ext. 3528
[Join our Talent Network]<http://www.everence.com/careers>

Confidentiality Notice: This information is intended only for the
individual or entity named. If you are not the intended recipient, do not
use or disclose this information. If you received this e-mail in error,
please delete or otherwise destroy it and contact us at (800) 348-7468 so
we can take steps to avoid such transmission errors in the future. Thank
you.
--
This is the Midrange Systems Technical Discussion (MIDRANGE-L) mailing list
To post a message email: MIDRANGE-L@xxxxxxxxxxxxxxxxxx<mailto:MIDRANGE-L@xxxxxxxxxxxxxxxxxx>
To subscribe, unsubscribe, or change list options,
visit: https://lists.midrange.com/mailman/listinfo/midrange-l<https://lists.midrange.com/mailman/listinfo/midrange-l>
or email: MIDRANGE-L-request@xxxxxxxxxxxxxxxxxx<mailto:MIDRANGE-L-request@xxxxxxxxxxxxxxxxxx>
Before posting, please take a moment to review the archives
at https://archive.midrange.com/midrange-l<https://archive.midrange.com/midrange-l>.

Please contact support@xxxxxxxxxxxxxxxxxxxx<mailto:support@xxxxxxxxxxxxxxxxxxxx> for any subscription related
questions.

Help support midrange.com by shopping at amazon.com with our affiliate
link: https://amazon.midrange.com<https://amazon.midrange.com>

--
This is the Midrange Systems Technical Discussion (MIDRANGE-L) mailing list
To post a message email: MIDRANGE-L@xxxxxxxxxxxxxxxxxx<mailto:MIDRANGE-L@xxxxxxxxxxxxxxxxxx>
To subscribe, unsubscribe, or change list options,
visit: https://lists.midrange.com/mailman/listinfo/midrange-l<https://lists.midrange.com/mailman/listinfo/midrange-l>
or email: MIDRANGE-L-request@xxxxxxxxxxxxxxxxxx<mailto:MIDRANGE-L-request@xxxxxxxxxxxxxxxxxx>
Before posting, please take a moment to review the archives
at https://archive.midrange.com/midrange-l<https://archive.midrange.com/midrange-l>.

Please contact support@xxxxxxxxxxxxxxxxxxxx<mailto:support@xxxxxxxxxxxxxxxxxxxx> for any subscription related questions.

Help support midrange.com by shopping at amazon.com with our affiliate link: https://amazon.midrange.com<https://amazon.midrange.com>

--
This is the Midrange Systems Technical Discussion (MIDRANGE-L) mailing list
To post a message email: MIDRANGE-L@xxxxxxxxxxxxxxxxxx<mailto:MIDRANGE-L@xxxxxxxxxxxxxxxxxx>
To subscribe, unsubscribe, or change list options,
visit: https://lists.midrange.com/mailman/listinfo/midrange-l<https://lists.midrange.com/mailman/listinfo/midrange-l>
or email: MIDRANGE-L-request@xxxxxxxxxxxxxxxxxx<mailto:MIDRANGE-L-request@xxxxxxxxxxxxxxxxxx>
Before posting, please take a moment to review the archives
at https://archive.midrange.com/midrange-l<https://archive.midrange.com/midrange-l>.

Please contact support@xxxxxxxxxxxxxxxxxxxx<mailto:support@xxxxxxxxxxxxxxxxxxxx> for any subscription related questions.

Help support midrange.com by shopping at amazon.com with our affiliate link: https://amazon.midrange.com<https://amazon.midrange.com>

Confidentiality Notice: This information is intended only for the individual or entity named. If you are not the intended recipient, do not use or disclose this information. If you received this e-mail in error, please delete or otherwise destroy it and contact us at (800) 348-7468 so we can take steps to avoid such transmission errors in the future. Thank you.