Re: Overwhelming a Web Application Server

Nathan Andelin skrev:

From: Thorbjoern Ravn Andersen
If there is one thing experience has taught me, it is that you must measure instead of guessing when analyzing performance problems.

Maybe we spent too much time talking about session limits and expiration. Hmmm, who came up with that idea in the first place ;-)

I did. It is the lowest hanging fruit :-)

It doesn't fix anything but it may buy you time to figure out what the next fruit is.

Your point about measurement is well taken. Walden pointed it out too. Unfortunately, my information in this case is from a user's perspective, and a developer's perspective - not that of a site administrator. I don't have profiling data.

First of all, what is the actual application doing stuff written in? Apache itself is so fast to serve static pages that it can easily saturate a 100 Mbps network connection.

Then, if profiling data is not easily available then you can always add logging yourself to the application. Just having the start time and end time of each invocation (this probably requires a job number or so?) allows you to derive the processing time of each request, the number of concurrent requests at any time plus more I cannot remember right now.

It might be something as simple as a lock forcing some processing step to be strictly sequential, causing all jobs to stand in line waiting.

But if you're willing to indulge in a somewhat hypothetical discussion, what specific measurements would you look at? Under IBM i, I think most operators would check CPU utilization & page faults, first - just because that's easy. Suppose the Tomcat server is consuming 95%. Would that mean much? What would you check after that?

Generally if a modern PC (I'm not familiar enough with the AS/400 to say if it applies here too) has so much computing power so if Tomcat uses 95% there is something doing unneccesary work (busy looping e.g). Page faults are interesting, as the Classic JVM is basically unbounded, so the heap may grow very large which doesn't go well with small pools.

For the Classic JVM, I would look at ANZJVM and DMPJVM over some time to get an idea what the system is doing. Also enabling verbose garbage collection can get you an idea if the system does excessive garbage collections implying the process needs more memory.

Then I would ask again :-)