× The internal search function is temporarily non-functional. The current search engine is no longer viable and we are researching alternatives.
As a stop gap measure, we are using Google's custom search engine service.
If you know of an easy to use, open source, search engine ... please contact support@midrange.com.


  1.  Home
  2. MIDRANGE-L
  3. February 2020

Re: Apache web jobs going crazy on V7R4

Well, I pointed the domain at my backup, cleared the logs and this morning
pointed back to my server. No more weird issues. I have a feeling if it
was a DoS attack maybe it was thwarted by Google, since that's where all
the requests would have went when pointed a the backup site (static Google
sites page).

I was hoping to get exactly what you were talking about... but no issues so
far today so no way to track. I had cleared all the logs in hopes to get
new ones with mainly the DoS attacker's IPs.

There were at least 600 HTTP server jobs running and I've NEVER seen user
profile QSECOFR in the user list when viewing the instance's active jobs.
So something was going on that was out of the ordinary... maybe a bit
overwhelmed writing history logs or something. Nothing I could find let me
know what was going on with the user spaces or QSECOFR jobs.

As far as bringing the site down, it only dropped the CGI stuff (site is
SPA and uses a lot of SSI as well for little pieces here and there)...
static seemed to serve up fine. And since it was at my house the
bottleneck would be the internet connection that I have out here in the
sticks... kids couldn't play their online games... hehe... But that was
probably a saving grace not allowing more requests in.

Bradley V. Stone
www.bvstools.com
MAILTOOL Benefit #16 <https://www.bvstools.com/mailtool.html>: No external
"helper" PC system required. 100% IBM i native!

On Tue, Feb 11, 2020 at 11:32 AM Nathan Andelin <nandelin@xxxxxxxxx> wrote:

If you haven't configured HTTP logging yet, I'd suggest doing that in order
to determine (if possible) the source of a DoS attack if it were to happen
again. In the case of one of our clients, one of my colleagues who has
since retired had coded some JavaScript that backfired and flooded the
server with something like 40 million repeating requests over a 24 hour
period. That didn't take the site down, thankfully.

Since the job log you posted contains only information and completion
messages (severity 00), that doesn't appear to be the cause of the crash.
Do you have any other ideas what caused the site to fail?


On Tue, Feb 11, 2020 at 6:22 AM B Stone <bvstone@xxxxxxxxx> wrote:

I have a feeling it was a DoS attack. I was for sure I remember from the
old days (V5R4) that Apache had built in DoS protection, but this attack
since I saw most of the user id's of the jobs were QTHMHTTP1 they were
probably hitting a script directly.

Then the QSECOFR jobs seemed to be some sort of issue as well. If it
happens again, I'll try to capture a few job logs, but so far things seem
ok. But it was a little unnerving seeing QSECOFR as the user id on a web
job. VERY odd.

I did download and install the latest CUM since my backup site was up as
well. But I don't think that's the issue since it worked fine for months
and just exploded last night. And it was only one of the many sites I
run.

Bradley V. Stone
www.bvstools.com
MAILTOOL Benefit #2 <https://www.bvstools.com/mailtool.html>: The
ability
to specify a "From" and/or "Reply To" email address!

On Mon, Feb 10, 2020 at 9:42 PM B Stone <bvstone@xxxxxxxxx> wrote:

Tonight I got a message my website was down, so I went to look and
there
were hundreds of jobs running under QTMHHTTP, QTMHHTP1 and, QSECOFR.

The job log for the ones for QSECOFR look like this over and over:

CPCA984 Completion 00 02/10/20 21:26:10.199949
QP0ZCHGU QSYS *STMT QC2SYS QSYS *STMT
From user . . . . . . . . . :
QSECOFR
From module . . . . . . . . :
QP0ZCHGU
From procedure . . . . . . :
main

Statement . . . . . . . . . :
371

To module . . . . . . . . . :
QC2SYS
To procedure . . . . . . . :
_C_NEU_system
Statement . . . . . . . . . : 35

Message . . . . : User Trace
option
changed for job
685003/QTMHHTTP/BVSTOOLSV5.

CPI2201 Information 00 02/10/20 21:26:11.321905
QSYGRAUT QSYS 165D QSYGRAUT QSYS 165D
From user . . . . . . . . . :
QSECOFR
Message . . . . : Authority
given
to user QTMHHTTP for object QP0Z684492 in
QUSRSYS object type *USRSPC.

CPC2201 Completion 00 02/10/20 21:26:11.350459
QSYGRAUT QSYS 165D QC2SYS QSYS *STMT
From user . . . . . . . . . :
QSECOFR
To module . . . . . . . . . :
QC2SYS
To procedure . . . . . . . :
system
Statement . . . . . . . . . : 13

Message . . . . : Object
authority
granted.
CPI2201 Information 00 02/10/20 21:26:11.383449
QSYGRAUT QSYS 165D QSYGRAUT QSYS 165D
From user . . . . . . . . . :
QSECOFR
Message . . . . : Authority
given
to user QTMHHTTP for object QP0Z684492 in
QUSRSYS object type *USRSPC.

CPC2201 Completion 00 02/10/20 21:26:11.383556
QSYGRAUT QSYS 165D QC2SYS QSYS *STMT
From user . . . . . . . . . :
QSECOFR
To module . . . . . . . . . :
QC2SYS
To procedure . . . . . . . :
system
Statement . . . . . . . . . : 13

Message . . . . : Object
authority
granted.
CPI2201 Information 00 02/10/20 21:26:11.430401
QSYGRAUT QSYS 165D QSYGRAUT QSYS 165D
From user . . . . . . . . . :
QSECOFR
Message . . . . : Authority
given
to user QTMHHTTP for object QP0Z684492 in
5770SS1 V7R4M0 190621 Display Job Log
S216709W 02/10/20 21:31:02 CST Page 2

It looks like a system job doing something, but I can't figure out
what.
I thought maybe a DOS attack, but it doesn't look like that.

I ended the server and started it back up and it just started doing it
again. I may just IPL to see if that helps as I plan to shut the one
server down anyhow.

Thanks.


--
This is the Midrange Systems Technical Discussion (MIDRANGE-L) mailing
list
To post a message email: MIDRANGE-L@xxxxxxxxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: https://lists.midrange.com/mailman/listinfo/midrange-l
or email: MIDRANGE-L-request@xxxxxxxxxxxxxxxxxx
Before posting, please take a moment to review the archives
at https://archive.midrange.com/midrange-l.

Please contact support@xxxxxxxxxxxx for any subscription related
questions.

Help support midrange.com by shopping at amazon.com with our affiliate
link: https://amazon.midrange.com

--
This is the Midrange Systems Technical Discussion (MIDRANGE-L) mailing list
To post a message email: MIDRANGE-L@xxxxxxxxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: https://lists.midrange.com/mailman/listinfo/midrange-l
or email: MIDRANGE-L-request@xxxxxxxxxxxxxxxxxx
Before posting, please take a moment to review the archives
at https://archive.midrange.com/midrange-l.

Please contact support@xxxxxxxxxxxx for any subscription related
questions.

Help support midrange.com by shopping at amazon.com with our affiliate
link: https://amazon.midrange.com


As an Amazon Associate we earn from qualifying purchases.

This thread ...
Replies:
Follow On AppleNews
Return to Archive home page | Return to MIDRANGE.COM home page

This mailing list archive is Copyright 1997-2024 by midrange.com and David Gibbs as a compilation work. Use of the archive is restricted to research of a business or technical nature. Any other uses are prohibited. Full details are available on our policy page. If you have questions about this, please contact [javascript protected email address].

Operating expenses for this site are earned using the Amazon Associate program and Google Adsense.