Re: Thinking out loud about a new RPG web framework

Re: DIM'ing data structures for 10,000 sessions for state web serving -

Aaaron wrote:
<i>It seems this would immediately put a limitation on the framework being used for public purposes because it would be limited by the RPG language vs. being limited by processing power (you can buy more processing power, but you can't buy more RPG DIM element entries). One could argue that 10,000 users is a lot, but in my mind that number could be eaten up pretty fast based on current RPG customers using this framework to take online orders and such.</i>


I agree, this is a fundamental question, and subsequent decision that needs to be made, Aaron. Do we take full advantage of i5/OS and ILE to go beyond the same ol' same ol', or instead limit ourselves to the same ol' that everyone else can do?

If we use any aspect of the iseries capability, then we should go for it and use it all to go beyond. There might be a philosophy of just use as little as possible, such as calling RPG programs from Java or PHP, with the idea that the RPG could be rewritten into another language if moved to another system, but we're not going to produce something superior by staying lowest common denominator as much as possible.

My goal is to exceed the norm with iseries i5/OS and ILE capability. Yes, that's iseries only, but everyone else has their "alleged" generic solutions, and if that's the best then the iseries will run it, if it's even close to actually being generic.

But a ton of solutions are being developed in .NET/.ASP for IIS under Windows, they utilize every aspect of their Windows environment to produce the optimal solution, and you don't see them sweating over whether it works under i5/OS or AIX or Linux. Because it doesn't, it's a proprietary solution.

But more importantly, to the point of session users and the data structre dimensioning scheme, the following:

1. Dimensiong data structures is the easy way to do this and explain it, but in a large and complex environment I would provide in the code an alloc of memory for each of the data structures for a session and access them with basing pointers. I wouldn't even deallocate the memory, just build up the pool as needed for concurrent sessions. This would be equivalent to dynamic dimensioning.

2. Data structure indexes can go to 32K. I set 10,000 as an example of an ambitious goal of running a major web app system from one iseries, which given optimal use of RPG and dataq servers as I describe with state maintained I think is possible. I would give something like a 20% overhead factor for web page processing versus 5250 processing and say that the concurrent users capable of being served would be 20% less than interactive session capability for a given box. That's just off the top of my head, no benchmarks. It could be more of an overhead than that.

3. In a robust environment where we're talking about of 10,000 or more logged in users of a business system, e-commerce, and the like, you start getting into a server farm of iseries. But the if the session maintains state, then session web pages have to be directed to the iseries where that user is logged in. Now here's where it gets interesting.

The standard wisdom is that stateless web serving allows any web page to go to any farm web server in any mixture from different sessions. But then the state has to be built up and restored, and that's the red herring. A database server has to have the state stored in it, and you have to get to the right database server, unless you can keep multiple database servers synched in real time. Which we need to do anyway for high availability, but there's a differnce between high availability synching for failover and keeping multiple production systems in synch, such as two or n way synching between the databases to start with instead of one way to the high availability backup server.

Less than acceptable ways of doing this store state in a cookie and/or hidden fields in the web page, and with encrypted data will get a lot of people by, but I think if anyone imagined our 5250 sessions and programs through the years storing all the program variables, authorities, and everything else we have in an interactive session in the 5250 binary stream which carries it around so that it could "go to any AS/400" in a server farm, we would not be in any way, shape, or form the reliable, secure, and nearly bulletproof system that we are, I think most anyone in the iseries community would agree with that. And that's what using cookies and hidden fields is doing.

So rather than store state in a central database that all the farm servers access to restore state for a given web page, the session communications in a very large complex environment in my scheme would be directed to the farm iseries the session is logged into, so the routing software has to have a level more intelligence than round robining to next farm server. But then state is there and appropriate database servers are just accessed to retrieve business data as we do in normal programming.

In reality, a complex environment would have more than one business system, and part of that smart routing would be routing to appropriate iseries running the system rather than all iseries running every business system and users logging in on random ones, etc.

That's just from one of your comments, Aaron. But these are good questions for us to come up with an optimal solution, as you suggest.

rd