Re: Screen Scraping Application

On Tue, 18 Feb 2003, Irving Hutagalung wrote:
>
> Basically, I want to create an application to get information from
> AS/400 through TN5250 protocol. When I browse through the internet, I
> found out about this tn5250. After browsing through the code and the
> docs, my development team is lost =)
>

I think you're going to find that this is more complicated than you think
it is.   lib5250 is a library of functions for writing a 5250 terminal
emulator.

In the package are pre-written emulators for Windows, Curses (One method
of text-only programming in Unix-like environments), SLang (another way to
do Unix-like) and you can get (separately) a GTK+ version (GTK+ = an X11
toolkit)

What you seem to be looking for is a library for interacting with one of
these preexisting terminals, rather than a library for writing your own
terminal.   Something like HLLAPI, perhaps...   Unfortunately, support
like this has not been writing for our TN5250 client.

You could, of course, write your own HLLAPI style support for TN5250...
contribute it back to the project, that would be great.

You could also write your own "terminal" front-end to TN5250.   This would
be easier than adding HLLAPI support, but probably still more work than
what you're expecting.

A better option than either of these, quite frankly, is to not use 5250
support for this.   Simply write a program that runs on the AS/400 side
that listens on a socket, and write a program on the Solaris box that
connects to it.   This would be simpler, more robust, and it would perform
better.   If the application on your AS/400 is "very simple" as you say,
then I really don't see why you aren't doing it this way...

If your AS/400 developer is not familiar with writing sockets
applications, I wrote a tutorial on sockets for RPG programmers here:

       http://www.scottklement.com/rpg/socktut/

> So, need your help to give some hints on how to use tn5250 from another
> application. Basically, the application is a C application in Solaris.
> If possible, I want to use tn5250 as a library to connect that
> application to AS/400. Is this possible? Where should my team start
> looking at the source? Is there any changes required in the tn5250
> source? I prefer not to change anything in the tn5250 code.
>

If you do decide that you must use a "screen scraper" approach for this,
and want to take up my suggestion of writing your own "terminal" front-end
to tn5250...

Start in cursesterm.c, look at the function called
tn5250_curses_terminal_new() You'll see that what this basically does is
create a structure.   In that structure are all of the parts of the
"terminal object" that are called by lib5250.

Take a look at this code:

   r->init = curses_terminal_init;
   r->term = curses_terminal_term;
   r->destroy = curses_terminal_destroy;
   r->width = curses_terminal_width;
   r->height = curses_terminal_height;
   r->flags = curses_terminal_flags;
   r->update = curses_terminal_update;
   r->update_indicators = curses_terminal_update_indicators;
   r->waitevent = curses_terminal_waitevent;
   r->getkey = curses_terminal_getkey;
   r->beep = curses_terminal_beep;

These are all functions in that same source member.  They are assigned to
the structure.   lib5250 will use them as "callbacks" it calls them when
the appropriate event occurs.

So, you'd need to write your own routines to supply here...
Your version of the "terminal_update" function would probably simply query
the data that's in the display buffer to determine which screen you're on,
etc.   Save the appropriate keystrokes into a buffer.

Then, when curses_terminal_getkey is called by lib5250, supply the next
keystroke that you want to send to the terminal.

Aside from cursesterm, the other source file that you'll be interested in
is tn5250.c itself.   All this does is the basic setup... it assigns the
correct "terminal object" to lib5250, as well as other things.   For your
purposes, you'll probably just want to make your own copy of it, and
change it to use your "terminal object" instead of using
cursesterm/slangterm.

Of course, in both of these cases, you'll use your own code -- in seperate
source files -- and not modify the existing cursesterm.c or tn5250.c.
Just by making your own version of these two source files, you should be
able to do what you want to do...  just link them against lib5250

Also, your routines could call the appropriate curses/slang/gtk/windows
routines if you want the resulting screens to be displayed to the user
as they happen.   in this case, your code would be like a "layer inserted
in between the terminal and the 5250 library"

Also, you can skip the sign-on screen altogether by supplying

env.USER=<userid>
env.IBMSUBSPW=<password>
env.IBMCURLIB=<initial library>
env.IBMIMENU=<initial menu>
env.IBMPROGRAM=<program to call>

in your TN5250 configuration.   This will sign you on as the given userid
& password, skipping the sign-on screen completely.    Just keep in mind
that if you save this in a config file, anyone with access to read that
file can see the user/password.   And it's also sent over the network in
plain-text (unless you're using SSL)....   but that's true of the
keystroke method as well.

> To give more background, the application that resides in the AS/400 is a
> very simple application. The AS/400 developer already creates an
> application for my team, which requires very little screen interaction.
> In fact, the "screen" is used only to receive data and to send data. So,
> don't think of this screen as a "normal screen". The data itself will be
> in the form of fixed-length format.
>
> If done using a "true" TN5250 terminal emulation (I've tried this using
> tn5250 running on Windows XP), this is what I do (human interaction):
> 1. connect to AS/400, using the workstation ID, user ID, and password
> 2. a blank screen will be shown
> 3. I type the data (example: "GET12345BAL", which means "Get (GET)
> balance (BAL) for account number 12345") and press enter.
> 4. the screen will be changed to the reply message (example:
> "GET12345BAL  135", which means the account balance is USD 135).
>
> So, my application need to:
> 1. connect to AS/400
> 2. send the keystrokes with the appropriate format.
> 3. get the screen content to get the reply.
>

Yep.  Again, you can do it the way I've outlined above.   But, I'd really
recommend writing a direct communication between a solaris program & an
OS/400 program using sockets.


> I've run through the archive of this mailing list (at least for the
> recent message up until August 2002), and only found the following
> related messages:
> http://archive.midrange.com/linux5250/200210/msg00007.html
> http://archive.midrange.com/linux5250/200209/msg00020.html
>

Yes, there has been some interest in this, but nobody yet has implemented
it to my knowledge.