× The internal search function is temporarily non-functional. The current search engine is no longer viable and we are researching alternatives.
As a stop gap measure, we are using Google's custom search engine service.
If you know of an easy to use, open source, search engine ... please contact support@midrange.com.


  1.  Home
  2. MIDRANGE-L
  3. March 2018

RE: What would cause hundreds of interactive users all start dropping, from multiple LPARs

We had this happen some years ago and it was a spanning tree issue. No failing hardware. Spanning had been misconfigured. At the time we have subnets that spanned multiple buildings and since then they changed to only having one subnet in a building. Problem has never returned.

Mike Cunningham

-----Original Message-----
From: MIDRANGE-L <midrange-l-bounces@xxxxxxxxxxxx> On Behalf Of Arco Simonse
Sent: Wednesday, March 7, 2018 3:08 PM
To: Midrange Systems Technical Discussion <midrange-l@xxxxxxxxxxxx>
Subject: Re: What would cause hundreds of interactive users all start dropping, from multiple LPARs

Hi Paul,

With regard to "network noise" that you mention:
If your network switches use Spanning Tree Protocol (STP) and it is improperly configured, and when one of the switches is failing or becomes disconnected then it can happen that the network becomes a long timeout (up to 60 seconds) because the other switches are listening and rebuilding the STP network topology. When the failing device is reconnected then the timeout will happen again and so on.
Could that be (part of) the issue?

Best regards,
-Arco

Op woensdag 7 maart 2018 heeft Steinmetz, Paul <PSteinmetz@xxxxxxxxxx> het volgende geschreven:

We had a repeat of this issue from 02/07/17.

Multiple iSeries LPARs - Pencor05, Pencor06, Pencor07, Penco08 My
connections to 4 different LPARS, 3 different NICS, all at the same time.
Not all users at once.
About 20 every minute.
Each user only once (so far)
No entries in PAL, LIC.
Only this pattern in each joblog and QHST.

CPF5140 Diagnostic 70 03/06/18 09:27:21.942782
QWSERROR QSYS 0573 QWSGET QSYS 0683
Message . . . . : Session stopped
by a request from device PALPR5505A.
CPF5503 Diagnostic 30 03/06/18 09:27:21.942879
QWSERROR QSYS 0595 QWSGET QSYS 0683
Message . . . . : Input or Output
request failed. See message CPF5140.
CPC1197 Completion 50 03/06/18 09:27:21.943629
QWTPITP2 QSYS 0645 *EXT *N
Message . . . . : Job ended because
device error occurred.
CPF4410 Diagnostic 00 03/06/18 09:27:21.944839
QWSCLOSE QSYS 098E QUIRCLAP QSYS 002D
Message . . . . : File QDUI132 in
library QSYS closed, device PALPR5505A may
be unusable.

This time we have some additional info.
We lost some fiber connections to some remote offices due to this past
weekend's Northeaster.
To recover, some dark, spare, fibers were being lit up.
There is suspicion that during this process, some "noise" may have
been created on the network, causing devices to lose their IP address.
Each time a session dropped, we saw a new lease was obtained on the
PC, we also saw this back in 2017.

Anybody from the group ever have similar experiences or are there any
tools/methods to confirm this?

It will happen again.

Paul





-----Original Message-----
From: Steinmetz, Paul
Sent: Friday, February 17, 2017 3:16 PM
To: 'Midrange Systems Technical Discussion'
Subject: RE: What would cause hundreds of interactive users all start
dropping, from multiple LPARs

We found what happened but we did not find the source.
The NIC card on the PCs were sent a command to shutoff.

From the PC Windows System log.
Event ID 7036 - The TCP/IP NetBIOS Helper service was successfully
sent a stop control.

Paul

-----Original Message-----
From: MIDRANGE-L [mailto:midrange-l-bounces@xxxxxxxxxxxx] On Behalf Of
Steinmetz, Paul
Sent: Wednesday, February 08, 2017 4:18 PM
To: 'Midrange Systems Technical Discussion'
Subject: RE: What would cause hundreds of interactive users all start
dropping, from multiple LPARs

Interesting find

PC obtained a new lease shortly before the CPF5140 was posted.
I'm not a network person, but could there have been an issue with
leases expiring and being renewed?

CPF5140 Diagnostic 70 02/07/17 13:30:55.498675

Paul



-----Original Message-----
From: Steinmetz, Paul
Sent: Tuesday, February 07, 2017 4:43 PM
To: 'Midrange Systems Technical Discussion'
Subject: RE: What would cause hundreds of interactive users all start
dropping, from multiple LPARs

Started at 12:45.
Multiple iSeries LPARs - Pencor05, Pencor06, Pencor07, Penco08 My
connections to 4 different LPARS, 3 different NICS, all at the same time.
Not all users at once.
About 20 every minute.
Each user only once (so far)
No entries in any logs. PAL, LIC, QHST

Problem appears to have stopped around 14:25.
iSeries performance graph showed high number of TCP connection resets
during the 12:45 thru 14:25 problem window.

from 3 LPARS, 3 different NICs
Appears that only 5250 Client Access and GUI was impacted (PC traffic).
from 3 LPARS, 3 different NICs All other applications (hundreds)
connecting to both internal and external remotes were fine.

Possibly an issue with QTVDEVICE jobs ???

CPF5140 Diagnostic 70 02/07/17 12:45:56.790988
QWSERROR QSYS 0630 QWSGET QSYS 067C
Message . . . . : Session stopped
by a request from device QPADEV004S.
CPF5503 Diagnostic 30 02/07/17 12:45:56.791059
QWSERROR QSYS 0652 QWSGET QSYS 067C
Message . . . . : Input or Output
request failed. See message CPF5140.

Paul

-----Original Message-----
From: Steinmetz, Paul
Sent: Tuesday, February 07, 2017 2:06 PM
To: 'Midrange Systems Technical Discussion'
Subject: RE: What would cause hundreds of interactive users all start
dropping, from multiple LPARs

Problem still occurring.
Started at 12:53.
Not all users at once.
About 20 every minute.



-----Original Message-----
From: MIDRANGE-L [mailto:midrange-l-bounces@xxxxxxxxxxxx] On Behalf Of
Ken Meade
Sent: Tuesday, February 07, 2017 1:45 PM
To: Midrange Systems Technical Discussion
Subject: RE: What would cause hundreds of interactive users all start
dropping, from multiple LPARs

We saw something similar a few years ago. Turned out it was a glitch
in an addressing table in one of our switches.

-----Original Message-----
From: James H. H. Lampert [mailto:jamesl@xxxxxxxxxxxxxxxxx]
Sent: Tuesday, February 07, 2017 1:40 PM
To: Midrange Systems Technical Discussion
Subject: Re: What would cause hundreds of interactive users all start
dropping, from multiple LPARs

On 2/7/17, 10:34 AM, Steinmetz, Paul wrote:
What would cause hundreds of interactive users all start dropping,
from
multiple LPARS

Massive communications glitch?

--
JHHL


--
This is the Midrange Systems Technical Discussion (MIDRANGE-L) mailing
list To post a message email: MIDRANGE-L@xxxxxxxxxxxx To subscribe,
unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/midrange-l
or email: MIDRANGE-L-request@xxxxxxxxxxxx Before posting, please take
a moment to review the archives at http://archive.midrange.com/midrange-l.

Please contact support@xxxxxxxxxxxx for any subscription related
questions.

Help support midrange.com by shopping at amazon.com with our affiliate
link: http://amzn.to/2dEadiD
--
This is the Midrange Systems Technical Discussion (MIDRANGE-L) mailing
list To post a message email: MIDRANGE-L@xxxxxxxxxxxx To subscribe,
unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/midrange-l
or email: MIDRANGE-L-request@xxxxxxxxxxxx Before posting, please take
a moment to review the archives at http://archive.midrange.com/midrange-l.

Please contact support@xxxxxxxxxxxx for any subscription related
questions.

Help support midrange.com by shopping at amazon.com with our affiliate
link: http://amzn.to/2dEadiD
--
This is the Midrange Systems Technical Discussion (MIDRANGE-L) mailing
list To post a message email: MIDRANGE-L@xxxxxxxxxxxx To subscribe,
unsubscribe, or change list options,
visit: https://lists.midrange.com/mailman/listinfo/midrange-l
or email: MIDRANGE-L-request@xxxxxxxxxxxx Before posting, please take
a moment to review the archives at
https://archive.midrange.com/midrange-l.

Please contact support@xxxxxxxxxxxx for any subscription related
questions.

Help support midrange.com by shopping at amazon.com with our affiliate
link: http://amzn.to/2dEadiD

--
This is the Midrange Systems Technical Discussion (MIDRANGE-L) mailing list To post a message email: MIDRANGE-L@xxxxxxxxxxxx To subscribe, unsubscribe, or change list options,
visit: https://lists.midrange.com/mailman/listinfo/midrange-l
or email: MIDRANGE-L-request@xxxxxxxxxxxx Before posting, please take a moment to review the archives at https://archive.midrange.com/midrange-l.

Please contact support@xxxxxxxxxxxx for any subscription related questions.

Help support midrange.com by shopping at amazon.com with our affiliate link: http://amzn.to/2dEadiD

________________________________
This email may contain confidential information about a Pennsylvania College of Technology student. It is intended solely for the use of the recipient. This email may contain information that is considered an “educational record” subject to the protections of the Family Educational Rights and Privacy Act Regulations. The regulations may be found at 34 C.F.R. Part 99 for your reference. The recipient may only use or disclose the information in accordance with the requirements of the Federal Educational Rights and Privacy Act Regulations. If you have received this transmission in error, please notify the sender immediately and permanently delete the email.

As an Amazon Associate we earn from qualifying purchases.

This thread ...
Follow-Ups:
Replies:
Follow On AppleNews
Return to Archive home page | Return to MIDRANGE.COM home page

This mailing list archive is Copyright 1997-2024 by midrange.com and David Gibbs as a compilation work. Use of the archive is restricted to research of a business or technical nature. Any other uses are prohibited. Full details are available on our policy page. If you have questions about this, please contact [javascript protected email address].

Operating expenses for this site are earned using the Amazon Associate program and Google Adsense.