Re: 'green screen' not sellable --> WE(?) are the problem

Scott/Joep



It may be a bug that QZSRCGI with specific user profiles closes down when
the threshold is reached. But since nobody has noticed there is no reason
to continue that discussion.



Security is an interesting point. To me there are basically three types of
applications. Internal applications that is used in protected networks
where external clients either connects to the system via VPN or at least
HTTPS and external applications that is public applications where users may
be business partners that log on to the web system and only is allowed to
process partial data (their own) in the database. A third is of course what
I would call a CMS based system that doesn’t require log in but may publish
general isolated information from the business system.



Most systems I have done are the type 1 & 2 and to make this more complex I
also make systems that have to seeming less connects into other cloud
applications running in other domains somewhere on the net.



A little off-topic a demo of type 3 apps can be found here



http://www.powerext.com/2015/06/14/demo-of-a-powerext-widget-in-wordpress/



I have made my own log in validation where there are two types of users,
internal that is validated against OS user profiles and external that is
validated against an internal validation list. Internal users has to be
defined in the internal user profile table to gain access since the user
profiles has information that points on group profiles so I’m able to have
different designs down to field level controlling the systems behavior.



Now I run stateless in a SOA design. This means that the programs the
launces the UI is completely separated from the programs that
delivers/process data in the system. This is done by generic REST(CRUD)
services.



When any user logs in the system creates a user specific menu containing
the UI functions the user is allowed to process. In this process each
function is created in a request table the holds the 32 byte session and a
32 byte random request id and the function (program) name and server side
parameters that are hidden from the user. The system can of course be
broken but a 64 byte (10^64) randomized string where the seed for the
randomization is constantly changed (and passed between random stateless
users) combined with a 10 char function name isn’t likely to be within the
timeframe it exists.



Any CGI program in the system starts by validating this information and
rejects the access if session/request/function name doesn’t match.



Since the UI CGI programs that launches the FDA (Front-end Driven
Application) and then goes to sleep relies on separate REST(CRUD) services
the UI CGI program use the same mechanism and creates handles in the
request table for each REST(CRUD) service it requires and in the process
may pass server side parameters to the service.



The result is that no CGI program can run without a specific handle in the
request table and the handles are of course deleted when the session is
destroyed. The result is also that you are able to pass internal server
side parameters that restricts what the service can process/do thus
avoiding DOM/javascript injections.



The security system may of course be combined with other more general
security measures.



My best bid on the security issues using CGI is to keep your CGI library(s)
clean and your CGI programs protected by a server side mechanism like the
one I have described and that goes for both persistent and stateless
environments that I agree doesn’t differ so much in a security perspective.



1) RPGOA. If customers just want a browser based GUI why not just screen
scrape their 5250 environment? There are very advanced screen scrapers
today that require no rewrite of 5250 code. To me we are a little in a
twilight zone here. There are many ways to reuse existing code. Screen
Scraping is one, trying to maintain the record-based access paradigm with
RPGOA is another, isolating business logic from existing code to be used in
both existing and modern code and thereby build new possibly combined with
screen scraping in the transfer window is yet another.



2) RPGOA. Since the UI CGI programs and the REST(CRUD) services are
totally separated the REST(CRUD) service doesn’t know what information
(fields) that are passed to it. In the same time the UI CGI programs are
“design” driven based on the user settings. This means that the same UI CGI
program may generate a form with 5 fields for one user while another may
have a form with 100 fields both using the same base REST(CRUD) services
for data handling.



Using RPGOA/EXFMT will not tell the REST(CRUD) service which fields that
are passed to the handler. Using CGIDEV2 mydata = zhbGetVar(‘var1’); will
neither. But powerEXT’s mydata = getExtVar(‘var1’:mydata); will search the
input buffer for the variable and if not found return the default value in
parameter 2 to the input field.



Now try to show me this done from a RPGOA/EXFMT where you move fields ‘én
block’ and need to get the handlers fields into internal fields.
getExtVar() will btw also work in a traditional CGIDEV2 program if my
service program is included in the CGIDEV2 program.



The RPGOA/WRITE and traditional CGIDEV2 updHtmlVar/wrtSection has basically
the same problem. It is difficult to let the generation of output be
dynamical controlled by BL. Therefore I constructed my xmlNode() and
jsonNode() support that not only construct the output without fixed formats
and templates but also encode special characters properly according to the
format.



This doesn’t mean that powerEXT doesn’t support templates it just do it in
one procedure instead of having to load the template first
echoCgiDev2(‘/mytemplate.xml’:’mysection’);.



When you write that RPGOA is a simple as EXFMT TheFormat; it is a modified
truth because you need to external define the format or an internal data
structure and you probably also need to synchronize/convert internal fields
with the handler format fields and how do you bind let’s say EANNBR to the
XML element



<com:ID schemeID="EAN" 86112112 /com:ID>



that I code internal as xmlNode(‘com:ID’:’schemeID=”EAN”’:%char(EANNBR));



This is admitted a question of programming style but my LEAN time studies
shows that my internal method is much faster than the traditional CGIDEV2
template based approach.



Sorry I quoted you, but I remembered our early discussions about RPGOA and
I agree that RPGOA was much hyped. “Now you can just change a single line
in your 5250 program and then it connects to a mobile phone” etc. IBM
marketing just forgot that RPGOA also needed a handler and a UI and those
was not so easy to write.



I fully aware that Profound has other means than the record-based access
paradigm else you would probably not be in business.

On Sun, Jun 14, 2015 at 5:33 AM, Scott Klement <midrange-l@xxxxxxxxxxxxxxxx>
wrote:

Henrik,

On 6/13/2015 8:56 AM, Henrik Rützou wrote:

If you use the "Passwd %%SYSTEM%%, Userid %%CLIENT%%, require valid-user"
setup the Apache server will start and close down QZSRCGI jobs as I
described. I admit it is some years since I was playing around with it but
I can’t imagine it has changed.

This may be a bug that was fixed a long time ago? We have hundreds of
customers running shops based on this technology, many of them running
thousands of simultaneous users. They run their businesses on it, all day
every day, and we've been doing this for 5 years. Haven't had a single
complaint of this happening in the time I've worked for Profound Logic.

Sorry, but I have to believe this was either a bug that has been fixed
since then, or that you may have made a mistake somewhere.

If you keep jobs persistence by using Accept-HTSession handles you will,
with a large user pool, eventually run into thresholds since you need a
long timeout to avoid casual users to be thrown out of the system.

What do you consider a "long timeout"? We have a user-configurable
timeout, so each customer chooses a different one. We also have a
"keep-alive" functionality in our software where the JavaScript component
will send "no op" requests in the background to prevent timeouts from
occurring. This way, if a customer doesn't wish to time out, they can set,
for example, a 30 minute timeout with a 15 minute keep-alive. The session
will stay active forever as long as the display is active, but if it loses
network, browser is closed, etc, they'll time out in 30 minutes.

We do have customers using shorter timeouts. We've even tested them as
small as one minute (you wouldn't do that in production, most likely) and
haven't had problems.

We also have JavaScript code that closes the persistent connection when
you close your browser window, so in "normal" circumstances, you don't have
to wait for the timeout. (Only when there's an unexpected close, like
network or power is lost.)

In the persistent and stateful environment you have no way to restart your
web server without every user is thrown out. In a stateless environment
you
can restart the webserver and the only thing the users experience is a
short delay before they can continue. That doesn’t mean that you can throw
a user or users out or time them out in the stateless environment.

Yes, that's true, restarting the server is a bigger problem using
persistent connections.

It hasn't been a problem for our customers to keep the server running,
however. Just as you don't restart your telnet server during production use
of 5250, you don't restart your HTTP server during production use of web.

If you use the "Passwd %%SYSTEM%%, Userid %%CLIENT%%, require valid-user"
setup you use system user id’s and passwords to gain access to the CGI
library. Are you telling me that these same system user ids/passwords
can’t
be used to run FTP if the FTP server is started?

Oh, so you are saying that the same userid/password works in FTP as well
as HTTP. You aren't saying that they get logged into FTP automatically,
then? That's what I thought you meant.

Yes, if you use Passwd %%SYSTEM%%, that's true. But it's also true of
non-persistent CGI using system passwords, as well as other technologies
like 5250. There's nothing here that's exclusive to persistent CGI.

And, you don't _have_ to use Passwd %%SYSTEM%%. About half of our
customers do, and the other half do not (we have our own password screen,
et al, that they can use instead.) You could also use things like
validation lists, kerberos/sso, LDAP, etc. Again, this is exactly the same
as non-persistent CGI.

QTMHHTP1 is to me a “shxtty” profile since it has no password and no
authority. You can of course validate users against OS profile and get a
handle to transfer underlying new jobs to another user profile but then
you
lose all the benefits of a stateless active environment since you isn’t
able (at my best knowledge) to change the current user profile on an
active
program.

I don't understand what you're saying here. If you don't like the QTMHHTP1
profile, use a different one... what's the problem? But, again, this is
exactly the same with persistent vs. non-persistent. There is no difference
in the way user profile work between them.

Apache supports LDAP, Kerberos, Validation Lists, and system passwords.
Or you can write your own sign-on logic in your application. Again, this
is exactly the same with both persistent and non-persistent CGI.

You can switch in the middle of an active session if you want using either
the profile handle or profile token APIs, just like any other program
running on the system.

The real problem here is that the stateless environment needs to have
access to a variety of other systems to be able to function such as the
ERP
system, archive systems or other tools etc. so you really need a user
profile that has all the needed access.

Again, that's the same whether you are using stateless or persistent
CGI... Many customers use standard OS profiles for this reason, and are
very happy with them.

In other circumstances, you could use Enterprise Identity Mapping (EIM) so
that people sign on with one userid/password, but it maps to another
underlying profile of your choice... if that helps? We mostly see
customers do this so they can have a "single signon" using a kerberos
server.

I don't understand what this has to do with persistent CGI. As I've said
many times now, authentication is no different whether you're using
persistent or non-persistent CGI.

Your statement “That's a good thing -- you don't want the profile people
can get into without any credentials to have a lot of authority!” is not
correct. No outsider can know what user profile you use in the Apache
servers QZSRCGI jobs since no one can access the .conf file if Apache is
configured proper and even if they knew the user profile they would not
know the password.

So you think the best solution would be to always run Apache as QSECOFR so
that they always have access to everything? Fine, you can do that if you
want. As you pointed out, there's an Apache directive you can use to
control the user profile that this runs under. And, again, this has
nothing to do with persistent CGI, as it's the same for both persistent and
non-persistent CGI.

I personally think for the non-signed-in user (such as a public facing web
site) it's best to give them as little authority as possible, because if
they can find some sort of security flaw in your code or a bug in Apache,
they will be limited in what they can do. But, it's up to you.

Basic or Digest Authentication will not save the day. It is simply not
enough and very equal to run security level 20 on IBM I and it is what I
call “false security” since it grants full access to any program in the
CGI
library and doesn’t secure you from DOM and javascript injections. What
you
really need is only to be able to access specific program/services in the
CGI library and in the same time often only allow the program/service to
process specific sections in a DB table without having to maintain a
complex security environment.

Again, this has nothing to do with whether it's persistent or not.

And it doesn't always grant authority to all programs in a library. This
is up to you to configure how you want it. If you use UserID %%CLIENT%%,
then you can use object-level security to determine which objects you want
to grant access to. Same with using EIM to map profiles...

If you don't use basic/digest, you can code your own authentication into
your program. In that case, QTMHHTP1 (or whatever you configured) would
need access to your authentication program -- but your software can choose
what to allow access to beyond that.

This is all configurable stuff, and it has nothing to do with persistent
vs. non-persistent CGI.

I could elaborate how to build a secured and SOX audited and approved
general system, but it is a little off-topic in this tread.

We have many customers who are SOX audited, and they are passing their
audits with our software.

In general the RPGOA handler is in that respect a RPG “oddball” since the
technique is nowhere to be found in any other programming language and is
not even supported in the other ILE based languages like COBOL, C or CL
that all communicates with each other using parameters to pass information
among them.

Yes, I agree with you. But, a lot of people have found that "oddball"
technique to be useful.

My point is that using it doesn't make you "monolithic" and doesn't lock
you into an architecture where programs must stay active. There are
choices here... you can choose (mainly for backward compatibility) to keep
programs running between page requests, or you can choose to rewrite
things. you can choose to break up your programs into modular pieces, or
you can choose to keep the old code. RPGOA doesn't lock you into either
situation.

To many customers, being able to start with what they already have and
gradually modernize them is a very helpful and important thing.

So “Open Access gives us very little we didn't have before.” - I think you
know who wrote that some 5 years ago or should I provide you with a link?
It is interesting reading ;-)

The problem I was having with RPGOA 5 years ago was that people were
over-hyping it. People were saying that IBM was "finally investing in RPG"
and "saving the platform" and other hyperbole. It didn't (and doesn't)
give us anything we didn't have before -- it only gave us a different way
of doing the same thing.

Indeed, at Profound Logic, we wrote pre-compilers for RPG, so you don't
actually need Open Access to use our software. However, as IBM keeps
adding features to RPG (for example, the free-format dcl-f declarations) it
becomes harder to maintain those precompilers, whereas Open Access
continues to work perfectly.

I must admit, it was hard for me 5 years ago to see why people just didn't
use subprocedure calls instead of OA. We've had that capability forever!

But here's what I've found now that I work with OA daily:

1) RPGOA provides the ability to keep backward compatibility. This is
HUGE because it eliminates the need to rewrite. For many customers, this
makes the difference between modernizing vs. either staying with 5250, or
switching to a package. (And switching to a package often means leaving
the platform.)

2) RPGOA can also provide a cleaner interface for certain tasks because it
automatically gathers all of the fields in your program related to a given
file operation and passes them to the handler.

Consider CGIDEV2's approach on a screen with 50 fields:

MyData = zhbGetVar('var1');
OtherData = zhbGetVar('var2');
MoreData = zhbGetVar('var3');
. . .
YetAnotherVariable = zhbGetVar('var50');

// calcs done here

updHtmlVar('var1': MyData);
updHtmlVar('var2': OtherData);
updHtmlVar('var3': MoreData);
...
updHtmlVar('var50': YetAnotherVariable);
wrtsection('TheSection');

More than 100 lines of code (not including the calcs on the data) to
read/write a screen with 50 variables. By contrast, with OA it looks like
this:

READ TheFormat;
// calcs done here
WRITE TheFormat;

Two lines of code. (Possibly with result-field data stuctures added).
That assumes that you are using a stateless handler (like our Universal
handler at Profound) that works in a manner similar to CGIDEV2.

Of course, if you're using a stateful handler (like our Rich Display
handler) where the RPG program remains active between page requests, the
code is simply this:

// calcs
EXFMT TheFormat;

This is not the main reason people use RPGOA, but it is worth some
consideration that the compiler automatically gathers the fields.

Now, I’m not a big fan of RPGOA mainly because it is RPG language
proprietary and keeps RPG programmers in the dark age of RLA like DDS
based
instruction set (READ/WRITE/EXFMT) and stateful 5250 program designs
instead of moving/pushing them forward using SQL for DB access

I agree that RLA is considered an old technique today. And the fact that
it uses a record format can sometimes be a drawback (such as being limited
to what fits in a 32k record).

But, keep in mind that this RLA is only on the display file, you can (and
should!) still use SQL for database.

And, Profound does offer alternatives. You can still do web-based
programming using tools like CGIDEV2, eRPG SDK, PowerExt or vendor tools
such as Profound's RPGsp or similar.

In the situation where you have something that doesn't fit will into the
record-based access paradigm (such as a large text field for "notes" or
other word-processor like field) you can still use AJAX calls for those
types of things, and have the program get the data the web-like way. So
you can use RPGOA where it works well, and use direct AJAX (reading
stdin/query string, etc) when that works better.


--
This is the Midrange Systems Technical Discussion (MIDRANGE-L) mailing list
To post a message email: MIDRANGE-L@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/midrange-l
or email: MIDRANGE-L-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/midrange-l.