Re: Cannot connect to IBM i through anything but local switch - SOLVED

Just posting this back so that it doesn't remain a mystery question:

The solution was: Get the firewall out of the way! IOW - The firewall
was the problem. I spent considerable time with Watchguard(It's not
us!) and IBM i support(It's not us!) and IBM AIX/VIOS support(It's not
us!). Since it was 2 Power guys against the firewall guys, who do you
think I believed? As an experiment (and I should have tried this
earlier), I put another router in front of the network the Blade Center
and JS12 was on. I left everything else the same. Same network and
subnet. Same gateway. Waddaya know? The IBM i could suddenly find it's
way out to the Internet and the inbound traffic made it's way to the blade!

The AIX guy I talked to said that some firewalls don't 'like' some
virtual mac addresses. The addresses are fine but there is something
about the way that arp responds that some firewalls aren't happy with.
In my case I could see the firewall sending an ARP request for
10.0.10.210 and 210 would reply with it's mac address (which was the
correct one) but the arp table on the firewall was never updated with
the mac (just 00:00:00:00:00...) No one, including the firewall
provider, can say why. So I know that the firewall was the culprit.
The Watchguard firewall guys say to update the firmware, which I will do
at some point, but once again, the i's have it.

FYI

Pete Helgren
www.petesworkshop.com
GIAC Secure Software Programmer-Java

On 1/15/2013 3:12 PM, Pete Helgren wrote:

This *sounds* like a routing issue but some of this doesn't add up.
At one point my IBM i (6.1) was:

1. Accessible from a different local logical and physical network
using 5250 port 992 and HTTP and HTTPS.
2. Accessible from the external (Internet) with HTTP and HTTPS (as it
should be, filtered by the firewall).

This is one of those classic "It worked before and now doesn't work
but nothing changed" scenarios (of course, *something* changed).

BladeCenter S with a few blades in it. All but my JS-12 are 'sleeping'
at the moment.
There is one switch at the BCS end. All devices on the switch are on
the same physical and logical network (10.0.10.0)
That switch(remote) is connected to local switch over a fiber link
(both local and remote switches are NetGear 24 port GB switches).
There are two interfaces plugged into the local switch: my laptop and
a connection to the firewall (10.0.10.2 - it's the gateway) so
essentially the remote and local switch are on the same physical and
logical network, it just has a fiber link between the two.

IBM i is at 10.0.10.205 and the BCS Chassis (AMM) is at 10.0.10.230 on
the remote switch. My laptop on the local switch is at 10.0.10.50.
All connected interfaces are subnetted to the 10.0.10.0 network
(255.255.255.0). Neither switch is VLAN'ed.

From my laptop on the *local* switch I can ping the gateway
(10.0.10.2) and the outside world. I can ping the interface on the i
(10.0.10.205) and interface on the AMM (10.0.10.230). I can bring up a
web site on the IBM i at address 10.0.10.210. I can bring up the AMM
web interface at 10.0.10.230 on the BCS. Basically, everything is
accessible to my laptop when it is on the local switch. So far so good.

On a second switch on a difference interface on the firewall
(192.168.1.0 network) I have no such luck now. It *was* working just
the same as being on the local switch: I pretty much had access to all
interfaces on the IBM i blade and the BCS chassis. Also, there is a
firewall route that sends all traffic on 97.77.83.54 to the internal
address of 10.0.10.210. None of that works now. From the 192.168.2.0
network I can only get to one interface: The BCS AMM interface at
10.0.10.230 (it too is subnetted to /24 255.255.255.0).

I fully understand that I should get to all the interfaces on the
10.0.10.0 network when I am on the switch. What I am having trouble
understanding is how I can get to ONLY one interface on that network
when I am on the 192.168.1.0 network. If it was purely a routing
issue, then it would be all or nothing (or follow a subnetting
pattern). Why can't I get to 10.0.10.205 or 10.0.10. 210 when I can
get to 10.0.10.230? The common denominator for all the inaccessible
interfaces (inaccessible from the other network) is that they are all
on the JS-12 Blade.

This may be TMI to solve the issue but if you have any ideas or
insights I'd love to hear them. Basically the issue is: On the local
switch I can get to all ports and all addresses. From another NIC on
the firewall (different physical and logical network) I can get to
ONLY 1 interface, all others are inaccessible. I changed the Ethernet
config on the IBM i side (since it has all the interfaces that cannot
be reached). The config is:

Line speed . . . . . . . . . . . . : 1G
Current line speed . . . . . . . . : 1G
Duplex . . . . . . . . . . . . . . : *AUTO (This was set to *FULL
but I reset it to *AUTO)
Current duplex . . . . . . . . . . : *FULL
Serviceability options . . . . . . : *NONE
Maximum frame size . . . . . . . . : 1496

I have thunk and thunk and thunk about this for about two weeks and I
haven't had any epiphanies as to why I have the issue. It's a
firewall issue or some weird config problem.