Re: Thoughts on this design for Power server installation across two data centers

Thanks. Thanks very helpful. I'm now reminded when we implemented a new
server we had 4 NICs connected to the network but we had then bound to a
single virtual IP which is used by all systems and staff accessing the box.
I didn't know that on IPL IBMi does gratuitous ARP on the virtual IP.

If you have more than one LPAR on the box and say you keep the production
DR LPAR off but have the test LPAR on presumably each LPAR can have its own
virtual IP but bound to the same 4 NICS?

Then we could keep test and dev going and only IPL the DR LPAR when needed.

On Wed, Dec 18, 2019, 2:31 AM DrFranken <midrange@xxxxxxxxxxxx> wrote:

So this networking piece is the most often component to 'get wrong' or
in many cases, 'simply ignored.' OOps. This is a key component where the
network team and the IBM i team absolutely need to be working together.
This because it is exceptionally rare for an IBM i System Admin to be
very good at HA and simultaneously good at networking at this level!

You mention a 'stretched VLAN' normally referred to as a bridged VLAN.
This is certainly an option and if it is available to you it is quite
easy to utilize from the IBM i side. Bring down the 'Access interfaces'
(Those used to access the system) on the production side and then bring
them up on the HA side. A couple notes about that.
a) Don't use IP Addresses when you STRTCPIFC, instead configure
alias
names. This way you end and start them by name not address. In the
future you might change them or you might change methods and those names
don't change. It also means the SAME Startup and shutdown programs can
be used on both ends. `
b) Make those interfaces virtual IP addresses. Why? Because when
you
start a virtual IP interface it does a 'gratuitous ARP'. What is that?
It means that the interface announces on the network: "HEY! Anyone
talking to IP address 192.168.0.1, it's now at mac address <new vIP mac
here>!' This means that all switches and routers along the path will get
that update instantly. Without this ARP the table entries will have to
time out or be manually cleared by admins before the HA server is
accessable. Same occurs of course on a switch back.

Another option is the use of routing protocols. In this scenario an IP
subnet that is NOT currently in use in your network is used for these
interfaces. So if you use 192.168.x.x in your network then perhaps
172.31.255.0 / 28 is used. Interfaces in this subnet are again placed on
both servers as with the bridged VLAN configuration. However in this
case routers are used to direct traffic to the correct server. Advantage
over the bridge is it can work almost anywhere while a bridge is not
always available. Routers don't have to pass broadcast traffic as a
bridge does. Disadvantage of course is configuring the routers which is
more effort than the bridged VLAN.

Load balancer equipment is always an option. It's generally quite good
but quite expensive! With these units though both internal and external
traffic can be handled and can be moved to other external connections
easily. The units I am most familir with are from BIG-IP but others are
out there and also work well. These units typically monitor both
endpoints and when they detect production down and HA up, they redirect
traffic to the available node.

The DNS swap has been mentioned and that can also be valid but must be
planned for. TTLs for the DNS must be kept low so that changes to DNS
propegate quickly. Default DNS TTLs are 1 or 2 days. That's clearly too
long!! This also can handle both internal users and external users as
well. Internal DNS is often active directory so staff generally can make
this change easily.

And of course some of you use the 'twoicons' approach. Here there are
are two icons on each desktop. One that connects to the production
system and one to the HA system. Users must select the correct icon.
This is my lease favorite approach!!

- Larry "DrFranken" Bolhuis

www.Frankeni.com
www.iDevCloud.com - Personal Development IBM i timeshare service.
www.iInTheCloud.com - Commercial IBM i Cloud Hosting.

On 12/17/2019 3:34 AM, Laurence Chiu wrote:

I had a quick chat to my network person. He thought the best way (if one

of

the hosts was going to be down most of the time but still use the same IP
as production) was to use a stretched VLAN across the two data centres.

So

if the production host went down, then when the secondary came up
advertising the same IP, it would be found. When I talked about external
connections I should have said external to the data centre but still

behind

the corporate firewall. No public access to these servers. So no waiting
for DNS changes to be flushed externally.



On Tue, Dec 17, 2019 at 1:56 AM Rob Berendt <rob@xxxxxxxxx> wrote:

I would test those router changes. It's not as simple as it sounds. And
test on a quarterly basis. How do you plan on having "zero" downtime

while

all your routers are being updated? And you're waiting the x number of
hours for the internet to flush stuff through for your external sites?

Here all the stuff that external people get to, like our websites and

file

sites, goes through an IBM Edge server which checks availability of the
endpoints and routes accordingly. So when we bring down our production

web

server the edge server routes automatically to the backup one.

Internally, our production and backup systems have different IP
addresses. We actually have a line in the switch program which
communicates to our Windows DNS server and changes the IP address for

our

production server to the backup server. Tested quarterly and just did

it

this weekend. Granted any 5250 session dies and comes back to life on

the

new system but we always do our switch at the same time on Saturday

morning

and the switch back at the same time Sunday evening.

Oh, and yes, IBM's license key's are keyed to system values QSRLNBR and
QMODEL. See the command WRKSYSVAL. However, as someone else pointed

out

you can load multiple values into WRKLICINF. Some applications,

however,

do not use WRKLICINF and store the keys in the application data file
libraries themselves. Infor LX does. I'm sure other software does

also.

And these are most definitely tied to QSRLNBR, QMODEL and often stuff

only

available via certain API's like number of processors allocated to that
lpar or all lpars of IBM i on that system.

Rob Berendt
--
IBM Certified System Administrator - IBM i 6.1
Group Dekko
Dept 1600
Mail to: 2505 Dekko Drive
Garrett, IN 46738
Ship to: Dock 108
6928N 400E
Kendallville, IN 46755
http://www.dekko.com


-----Original Message-----
From: MIDRANGE-L <midrange-l-bounces@xxxxxxxxxxxxxxxxxx> On Behalf Of
Laurence Chiu
Sent: Saturday, December 14, 2019 12:10 AM
To: Midrange Systems Technical Discussion <

midrange-l@xxxxxxxxxxxxxxxxxx>

Subject: Re: Thoughts on this design for Power server installation

across

two data centers

CAUTION: This email originated from outside of the organization. Do not
click links or open attachments unless you recognize the sender and know
the content is safe.


Hi Evan

I think what you're say about IP addresses etc. is okay. For the

mainframe

we replicate the entire SYSRES volume and data so when we bring up the

DR

LPAR, it looks exactly like production which is what we want. For

testing

we create routing rules to stop production traffic reaching the DR box
since it has the same IP addresses as production. In DR that would be

fine.

For the Power servers, given it has a number of external connections (to
mainframe and other systems as well as inbound connections), having the
same IP address etc. would be ideal. If production is down, then I want

all

network traffic to reach the DR box. We might need some router changes

to

enable this but users would be none the wise.

I would imagine IBMi would IPL okay with the same keys as the production
instance. It would not check the licence against the CPU ID would it? As
for ISV software, then we might have to load keys again but that should

not

be an issue.

I don't know much about the log shipping products but at the moment the

RPO

is not zero for the DR instance and I want to make it zero. SAN

replication

should do that (sites about 50 km's apart).

On Sat, Dec 14, 2019 at 9:11 AM Evan Harris <auctionitis@xxxxxxxxx>

wrote:

Hi Laurence

if you do SAN replication only then what you have in your other data

centre

is an exact copy of production, including such things as IP addresses,

host

names, Product License keys (possibly serial number based) etc. If you

use

PowerHA or one of the log shipping products (Mimix, QuickEDD et ) they
allow you to maintain separation between these things by excluding

certain

files/libraries (log-shipping) or keeping them in separate disk pools
(PowerHA uses IASPs to control the replicated data) I am glossing over

a

lot of details here.

To reference your comment about why you can't bring the system up using

SAN

replication, remember that the IBM i uses single level storage so that

you

don't actually have an isolated boot disk; the whole system is one
contiguous storage space. When the system is started on a SAN copy

after

an

abnormal failure it will go through an extended database recovery to

ensure

the system is consistent,so you need to be prepared for that in terms

of

your RTO. If you plan the failover and shut the source down nicely then

the

database (aka pretty much the whole system) will be in a consistent

state

at the other end and your system and will start up fine - but as I said
earlier it will have the same serial number, TCP IP address and other
system attributes as the original system. This may or may not be a

problem

for you especially if you have taken it into account.

If this will work for you and given the amount of data your are talking
about IBM VM restart technology product (formerly GDR) might be enough

to

meet your needs.

On Fri, Dec 13, 2019 at 10:42 PM Laurence Chiu <lchiu7@xxxxxxxxx>

wrote:

I did answer in another thread. The point of LTO tapes was to take

offsite

backups in a another city. So while I realise that replicate between

two

VTL's does mean there is an offsite copy, for our archiving purposes

it's

not enough.

Interesting point about Mimix. If I plan to do SAN to SAN mirroring

(using

say an IBM V5020) then why do I need software to mirror also? We did

get

a

quote for Mimix and it exceed the cost the SAN's! I have never heard

of

Quick-EDD but will check it out.

For encryption I was thinking of using SKLM which we use to encrypt

the

data on our DS8886's. Those servers are not accessible externally so

are

well protected from ransomware (I hope!)

On Fri, Dec 13, 2019 at 1:18 AM Rob Berendt <rob@xxxxxxxxx> wrote:

Why physical LTO7 tapes? We use two VTL's which replicate from one

DC

to

the other. I easily take a backup performed on a system at one DC

and

restore it on the system at the other DC.
We use BRMS and it does a dandy job.
We used to use BRMS with physical tapes and that did a fine job of
tracking where the tapes were, preparing the case to send to Iron

Mountain,

receiving tapes back in, etc. But, really, only using VTL saved a

LOT

of

manpower. There was one time, (pre VTL) we needed a rush tape back

from

Iron Mountain. They buggered up the job and delivered it to the

wrong

building in our complex. The building was owned by a totally

different

company. Took awhile to straighten out that mess. Strong argument

for

tape encryption though.

About the only argument for retaining physical media I've heard is

the

case of ransomeware hitting your VTL's. Then again, if you're

encrypting

your tapes you'd better make sure your keystore is ransomware proof.

We have no physical tapes, and no physical tape drives left. None.
I've done bare metal restores to new systems using the VTL's.

For system to system replication we were using Mimix but switched to
Quick-EDD.

Rob Berendt
--
IBM Certified System Administrator - IBM i 6.1
Group Dekko
Dept 1600
Mail to: 2505 Dekko Drive
Garrett, IN 46738
Ship to: Dock 108
6928N 400E
Kendallville, IN 46755
http://www.dekko.com

--
This is the Midrange Systems Technical Discussion (MIDRANGE-L) mailing

list

To post a message email: MIDRANGE-L@xxxxxxxxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: https://lists.midrange.com/mailman/listinfo/midrange-l
or email: MIDRANGE-L-request@xxxxxxxxxxxxxxxxxx
Before posting, please take a moment to review the archives
at https://archive.midrange.com/midrange-l.

Please contact support@xxxxxxxxxxxx for any subscription related
questions.

Help support midrange.com by shopping at amazon.com with our

affiliate

link: https://amazon.midrange.com

--

Regards
Evan Harris
--
This is the Midrange Systems Technical Discussion (MIDRANGE-L) mailing

list

To post a message email: MIDRANGE-L@xxxxxxxxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: https://lists.midrange.com/mailman/listinfo/midrange-l
or email: MIDRANGE-L-request@xxxxxxxxxxxxxxxxxx
Before posting, please take a moment to review the archives
at https://archive.midrange.com/midrange-l.

Please contact support@xxxxxxxxxxxx for any subscription related
questions.

Help support midrange.com by shopping at amazon.com with our affiliate
link: https://amazon.midrange.com

--
This is the Midrange Systems Technical Discussion (MIDRANGE-L) mailing

list

To post a message email: MIDRANGE-L@xxxxxxxxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: https://lists.midrange.com/mailman/listinfo/midrange-l
or email: MIDRANGE-L-request@xxxxxxxxxxxxxxxxxx
Before posting, please take a moment to review the archives
at https://archive.midrange.com/midrange-l.

Please contact support@xxxxxxxxxxxx for any subscription related
questions.

Help support midrange.com by shopping at amazon.com with our affiliate
link: https://amazon.midrange.com
--
This is the Midrange Systems Technical Discussion (MIDRANGE-L) mailing

list

To post a message email: MIDRANGE-L@xxxxxxxxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: https://lists.midrange.com/mailman/listinfo/midrange-l
or email: MIDRANGE-L-request@xxxxxxxxxxxxxxxxxx
Before posting, please take a moment to review the archives
at https://archive.midrange.com/midrange-l.

Please contact support@xxxxxxxxxxxx for any subscription related
questions.

Help support midrange.com by shopping at amazon.com with our affiliate
link: https://amazon.midrange.com

--
This is the Midrange Systems Technical Discussion (MIDRANGE-L) mailing list
To post a message email: MIDRANGE-L@xxxxxxxxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: https://lists.midrange.com/mailman/listinfo/midrange-l
or email: MIDRANGE-L-request@xxxxxxxxxxxxxxxxxx
Before posting, please take a moment to review the archives
at https://archive.midrange.com/midrange-l.

Please contact support@xxxxxxxxxxxx for any subscription related
questions.

Help support midrange.com by shopping at amazon.com with our affiliate
link: https://amazon.midrange.com