Re: Thoughts on this design for Power server installation across two data centers -- MIDRANGE-L

So this networking piece is the most often component to 'get wrong' or in many cases, 'simply ignored.' OOps. This is a key component where the network team and the IBM i team absolutely need to be working together. This because it is exceptionally rare for an IBM i System Admin to be very good at HA and simultaneously good at networking at this level!

You mention a 'stretched VLAN' normally referred to as a bridged VLAN. This is certainly an option and if it is available to you it is quite easy to utilize from the IBM i side. Bring down the 'Access interfaces' (Those used to access the system) on the production side and then bring them up on the HA side. A couple notes about that.
a) Don't use IP Addresses when you STRTCPIFC, instead configure alias names. This way you end and start them by name not address. In the future you might change them or you might change methods and those names don't change. It also means the SAME Startup and shutdown programs can be used on both ends. `
b) Make those interfaces virtual IP addresses. Why? Because when you start a virtual IP interface it does a 'gratuitous ARP'. What is that? It means that the interface announces on the network: "HEY! Anyone talking to IP address 192.168.0.1, it's now at mac address <new vIP mac here>!' This means that all switches and routers along the path will get that update instantly. Without this ARP the table entries will have to time out or be manually cleared by admins before the HA server is accessable. Same occurs of course on a switch back.

Another option is the use of routing protocols. In this scenario an IP subnet that is NOT currently in use in your network is used for these interfaces. So if you use 192.168.x.x in your network then perhaps 172.31.255.0 / 28 is used. Interfaces in this subnet are again placed on both servers as with the bridged VLAN configuration. However in this case routers are used to direct traffic to the correct server. Advantage over the bridge is it can work almost anywhere while a bridge is not always available. Routers don't have to pass broadcast traffic as a bridge does. Disadvantage of course is configuring the routers which is more effort than the bridged VLAN.

Load balancer equipment is always an option. It's generally quite good but quite expensive! With these units though both internal and external traffic can be handled and can be moved to other external connections easily. The units I am most familir with are from BIG-IP but others are out there and also work well. These units typically monitor both endpoints and when they detect production down and HA up, they redirect traffic to the available node.

The DNS swap has been mentioned and that can also be valid but must be planned for. TTLs for the DNS must be kept low so that changes to DNS propegate quickly. Default DNS TTLs are 1 or 2 days. That's clearly too long!! This also can handle both internal users and external users as well. Internal DNS is often active directory so staff generally can make this change easily.

And of course some of you use the 'twoicons' approach. Here there are are two icons on each desktop. One that connects to the production system and one to the HA system. Users must select the correct icon. This is my lease favorite approach!!

- Larry "DrFranken" Bolhuis

www.Frankeni.com
www.iDevCloud.com - Personal Development IBM i timeshare service.
www.iInTheCloud.com - Commercial IBM i Cloud Hosting.

On 12/17/2019 3:34 AM, Laurence Chiu wrote:

I had a quick chat to my network person. He thought the best way (if one of
the hosts was going to be down most of the time but still use the same IP
as production) was to use a stretched VLAN across the two data centres. So
if the production host went down, then when the secondary came up
advertising the same IP, it would be found. When I talked about external
connections I should have said external to the data centre but still behind
the corporate firewall. No public access to these servers. So no waiting
for DNS changes to be flushed externally.

On Tue, Dec 17, 2019 at 1:56 AM Rob Berendt <rob@xxxxxxxxx> wrote:

I would test those router changes. It's not as simple as it sounds. And
test on a quarterly basis. How do you plan on having "zero" downtime while
all your routers are being updated? And you're waiting the x number of
hours for the internet to flush stuff through for your external sites?

Here all the stuff that external people get to, like our websites and file
sites, goes through an IBM Edge server which checks availability of the
endpoints and routes accordingly. So when we bring down our production web
server the edge server routes automatically to the backup one.

Internally, our production and backup systems have different IP
addresses. We actually have a line in the switch program which
communicates to our Windows DNS server and changes the IP address for our
production server to the backup server. Tested quarterly and just did it
this weekend. Granted any 5250 session dies and comes back to life on the
new system but we always do our switch at the same time on Saturday morning
and the switch back at the same time Sunday evening.

Oh, and yes, IBM's license key's are keyed to system values QSRLNBR and
QMODEL. See the command WRKSYSVAL. However, as someone else pointed out
you can load multiple values into WRKLICINF. Some applications, however,
do not use WRKLICINF and store the keys in the application data file
libraries themselves. Infor LX does. I'm sure other software does also.
And these are most definitely tied to QSRLNBR, QMODEL and often stuff only
available via certain API's like number of processors allocated to that
lpar or all lpars of IBM i on that system.

Rob Berendt
--
IBM Certified System Administrator - IBM i 6.1
Group Dekko
Dept 1600
Mail to: 2505 Dekko Drive
Garrett, IN 46738
Ship to: Dock 108
6928N 400E
Kendallville, IN 46755
http://www.dekko.com

-----Original Message-----
From: MIDRANGE-L <midrange-l-bounces@xxxxxxxxxxxxxxxxxx> On Behalf Of
Laurence Chiu
Sent: Saturday, December 14, 2019 12:10 AM
To: Midrange Systems Technical Discussion <midrange-l@xxxxxxxxxxxxxxxxxx>
Subject: Re: Thoughts on this design for Power server installation across
two data centers

CAUTION: This email originated from outside of the organization. Do not
click links or open attachments unless you recognize the sender and know
the content is safe.

Hi Evan

I think what you're say about IP addresses etc. is okay. For the mainframe
we replicate the entire SYSRES volume and data so when we bring up the DR
LPAR, it looks exactly like production which is what we want. For testing
we create routing rules to stop production traffic reaching the DR box
since it has the same IP addresses as production. In DR that would be fine.

For the Power servers, given it has a number of external connections (to
mainframe and other systems as well as inbound connections), having the
same IP address etc. would be ideal. If production is down, then I want all
network traffic to reach the DR box. We might need some router changes to
enable this but users would be none the wise.

I would imagine IBMi would IPL okay with the same keys as the production
instance. It would not check the licence against the CPU ID would it? As
for ISV software, then we might have to load keys again but that should not
be an issue.

I don't know much about the log shipping products but at the moment the RPO
is not zero for the DR instance and I want to make it zero. SAN replication
should do that (sites about 50 km's apart).

On Sat, Dec 14, 2019 at 9:11 AM Evan Harris <auctionitis@xxxxxxxxx> wrote:

Hi Laurence

if you do SAN replication only then what you have in your other data

centre

is an exact copy of production, including such things as IP addresses,

host

names, Product License keys (possibly serial number based) etc. If you

use

PowerHA or one of the log shipping products (Mimix, QuickEDD et ) they
allow you to maintain separation between these things by excluding

certain

files/libraries (log-shipping) or keeping them in separate disk pools
(PowerHA uses IASPs to control the replicated data) I am glossing over a
lot of details here.

To reference your comment about why you can't bring the system up using

SAN

replication, remember that the IBM i uses single level storage so that

you

don't actually have an isolated boot disk; the whole system is one
contiguous storage space. When the system is started on a SAN copy after

an

abnormal failure it will go through an extended database recovery to

ensure

the system is consistent,so you need to be prepared for that in terms of
your RTO. If you plan the failover and shut the source down nicely then

the

database (aka pretty much the whole system) will be in a consistent state
at the other end and your system and will start up fine - but as I said
earlier it will have the same serial number, TCP IP address and other
system attributes as the original system. This may or may not be a

problem

for you especially if you have taken it into account.

If this will work for you and given the amount of data your are talking
about IBM VM restart technology product (formerly GDR) might be enough to
meet your needs.

On Fri, Dec 13, 2019 at 10:42 PM Laurence Chiu <lchiu7@xxxxxxxxx> wrote:

I did answer in another thread. The point of LTO tapes was to take

offsite

backups in a another city. So while I realise that replicate between

two

VTL's does mean there is an offsite copy, for our archiving purposes

it's

not enough.

Interesting point about Mimix. If I plan to do SAN to SAN mirroring

(using

say an IBM V5020) then why do I need software to mirror also? We did

get

a

quote for Mimix and it exceed the cost the SAN's! I have never heard of
Quick-EDD but will check it out.

For encryption I was thinking of using SKLM which we use to encrypt the
data on our DS8886's. Those servers are not accessible externally so

are

well protected from ransomware (I hope!)

On Fri, Dec 13, 2019 at 1:18 AM Rob Berendt <rob@xxxxxxxxx> wrote:

Why physical LTO7 tapes? We use two VTL's which replicate from one

DC

to

the other. I easily take a backup performed on a system at one DC

and

restore it on the system at the other DC.
We use BRMS and it does a dandy job.
We used to use BRMS with physical tapes and that did a fine job of
tracking where the tapes were, preparing the case to send to Iron

Mountain,

receiving tapes back in, etc. But, really, only using VTL saved a

LOT

of

manpower. There was one time, (pre VTL) we needed a rush tape back

from

Iron Mountain. They buggered up the job and delivered it to the

wrong

building in our complex. The building was owned by a totally

different

company. Took awhile to straighten out that mess. Strong argument

for

tape encryption though.

About the only argument for retaining physical media I've heard is

the

case of ransomeware hitting your VTL's. Then again, if you're

encrypting

your tapes you'd better make sure your keystore is ransomware proof.

We have no physical tapes, and no physical tape drives left. None.
I've done bare metal restores to new systems using the VTL's.

For system to system replication we were using Mimix but switched to
Quick-EDD.

Rob Berendt
--
IBM Certified System Administrator - IBM i 6.1
Group Dekko
Dept 1600
Mail to: 2505 Dekko Drive
Garrett, IN 46738
Ship to: Dock 108
6928N 400E
Kendallville, IN 46755
http://www.dekko.com

--
This is the Midrange Systems Technical Discussion (MIDRANGE-L) mailing

list

To post a message email: MIDRANGE-L@xxxxxxxxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: https://lists.midrange.com/mailman/listinfo/midrange-l
or email: MIDRANGE-L-request@xxxxxxxxxxxxxxxxxx
Before posting, please take a moment to review the archives
at https://archive.midrange.com/midrange-l.

Please contact support@xxxxxxxxxxxx for any subscription related
questions.

Help support midrange.com by shopping at amazon.com with our affiliate
link: https://amazon.midrange.com

--

Regards
Evan Harris
--
This is the Midrange Systems Technical Discussion (MIDRANGE-L) mailing

list

To post a message email: MIDRANGE-L@xxxxxxxxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: https://lists.midrange.com/mailman/listinfo/midrange-l
or email: MIDRANGE-L-request@xxxxxxxxxxxxxxxxxx
Before posting, please take a moment to review the archives
at https://archive.midrange.com/midrange-l.

Please contact support@xxxxxxxxxxxx for any subscription related
questions.

Help support midrange.com by shopping at amazon.com with our affiliate
link: https://amazon.midrange.com

--
This is the Midrange Systems Technical Discussion (MIDRANGE-L) mailing list
To post a message email: MIDRANGE-L@xxxxxxxxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: https://lists.midrange.com/mailman/listinfo/midrange-l
or email: MIDRANGE-L-request@xxxxxxxxxxxxxxxxxx
Before posting, please take a moment to review the archives
at https://archive.midrange.com/midrange-l.

Please contact support@xxxxxxxxxxxx for any subscription related
questions.

Help support midrange.com by shopping at amazon.com with our affiliate
link: https://amazon.midrange.com
--
This is the Midrange Systems Technical Discussion (MIDRANGE-L) mailing list
To post a message email: MIDRANGE-L@xxxxxxxxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: https://lists.midrange.com/mailman/listinfo/midrange-l
or email: MIDRANGE-L-request@xxxxxxxxxxxxxxxxxx
Before posting, please take a moment to review the archives
at https://archive.midrange.com/midrange-l.

Please contact support@xxxxxxxxxxxx for any subscription related
questions.

Help support midrange.com by shopping at amazon.com with our affiliate
link: https://amazon.midrange.com