Screen Scraping, Re-facing, WebEnabling; Terminology 101

Mornin' all !

Let me take a stab at some common definitions & terminology, please feel free 
to constructively add/correct information, as I hope to add this to the 
Midrange Dot Com FAQ site. Before you email me, please read *all* the text, 
including the footnotes. And finally, apologies to any vendor *not* mentioned 
as an example - is was not on purpose ! If you'd like to email me and try to 
convince me in which category your offering belongs, I'll update the FAQ 
version of this posting accordingly.

I Hope this helps those who have been confused over the last few days !

--phil


===============
Screen Scraping
===============

What Is It ?
------------

In its simplest terms, screen scraping is intercepting the original screen[1] 
output as-is from the original program but manipulating that screen information 
before presenting it to the user on the display device[2] in such a way as to 
present the user with a different look & feel of the same screen information.

How Is It Done ?
----------------

There are three main methods of achieving 5250 screen scraping, listed below in 
their most used order;

(i) Post-5250 Stream Creation Client Conversion - Software running on the PC 
intercepts the original 5250 data stream and converts it in some fashion to use 
the PC's underlying Windowing system & controls. For example taking 5250 input 
fields and converting them to Windows input controls. A high proportion of 
implementations use a 'thick client'[3] on the PC to render the 
converted/modified screen data. A smaller number of implementations use an 
'intelligent client'[4]. Post-5250 Stream Creation Client Conversion is, 
despite what sales brochures state, *not* implementable using a 'thin 
client'[5].

(ii) Post-5250 Stream Creation Server Conversion - Software running on the 
server where the 5250 data stream originates performs the same conversion as 
would have been done on the PC in (i). In the majority of implementations, this 
server side software uses the 'virtual terminal' API's to get the screen 
information. The advantage over (i) is that with additional server software, it 
is possible to implement the rendering in all three client types; thin, 
intelligent and thick as in (iii) below.

(iii) Pre-5250 Stream Creation Server Conversion - This is slightly more 
involved, as it requires intercepting the 5250 stream a lot earlier in the 
process and on the server. Almost all of the implementations on the iSeries of 
this kind of screen scraping achieve this by hooking into the System Entry 
Point Table (SEPT) and replace the program pointers to the original 5250 
programs (QWSGET, QWSPUT, etc) with program pointers to software that will 
render the 5250 in a form that the PC is expecting. Pre-5250 Stream Creation 
Server Conversion can send the new screen information to any of the three 
client types; thin, intelligent or thick.

The key point to note is that for screen scraping, the original program is 
*totally unaware* that it isn't talking to a real 5250 display - the original 
program is untouched code wise.

Another important point to remember about screen scraping is that because the 
screen scraping software still relies upon the 5250 data stream, the process is 
still interactive, and thus still incur the interactive 'tax'.

Examples Of Who Does Screen Scraping ?
--------------------------------------

(a) Post-5250 Stream Creation Client Conversion -
      Jacada, http://www.jacada.com/apps/5250_screen_scraping/welcome.htm
      WRQ, www.wrq.com/products/hostintegration
      LookSoftware, http://www.looksoftware.com/prod_centric.htm

(b) Post-5250 Stream Creation Server Conversion -
      CABEL, http://www.astoweb.com/en/prodotto.shtml
      LookSoftware, http://www.looksoftware.com/prod_newlook_beyond_refacing.htm

(c) Pre-5250 Stream Creation Server Conversion -
      Zephyr, 
http://www.3270-5250-mainframe-legacy-extension.com/legacy-extension-articles.htm
      IBM, http://www-3.ibm.com/software/webservers/hostintegration/
      IBM, 
http://www-1.ibm.com/servers/eserver/iseries/ondemand/solutions/webfacing.html


=========
Re-facing
=========

What Is It ?
------------

In its simplest form, re-facing is changing the underlying application source 
code to produce either an output that is something other than a 5250 data 
stream, or both a 5250 data stream *and* something other than a 5250 data 
stream (in most cases the something other is a HTML or XML form of the stream).

How Is It Done ?
----------------

There are a number of ways, each getting deeper and deeper (or closer and 
closer depending upon your point of view) to a re-write of the application 
source code. In every case however, it boils down to the lines of code in the 
source that write to the *DSPF (e.g. EXFMT) being changed to create the 
'something other than a 5250 data stream'.

A key point with re-facing is that it is now possible to run the application in 
batch mode, which of course can help reduce your interactive 'tax' hit.

Examples Of Who Does It This Way ?
----------------------------------

  SSA, http://www.ssagt.com/products/bpcs_v8.htm
  PSC/400, http://www.plutabrothers.com/p1.html
  Crys@xxx, http://www.crysalid-server.com/crysalid_vo/contain-synthesis.html


=====================
Webfacing/WebEnabling
=====================

Unfortunately, IBM has a screen scraping (see section 1) product called 
WebFacing, which can confuse matters, I personally prefer the term 
'WebEnabling'.

WebEnabling, is simply using either Screen Scraping or Re-facing to provide end 
users with a browser based front-end to an existing 5250 application. Simple !


---- Footnotes ----

[1] 99% of the time, it is screens that are scraped, however it's not limited 
to screens.

[2] Again, 99% of the time the original 5250 data stream is modified to be 
rendered on a display device, but it's not limited to rendering for a display 
device.

[3] Thick Client - this is when a complete, standalone *application* is 
installed on the PC. For example SSA's BPCS C/S V6.04 (CEA applications)

[4] Intelligent Client - this is when custom software is installed on the PC, 
but this software is executed within a non-related software product. For 
example ActiveX controls or Java Applets for Internet Explorer.

[5] Thin Client - in it's purest form this is just a browser, any browser IE, 
Netscape, Opera, take your pick. In it's common form, it a browser plus a 
number of public plugins, such as Shockwave, RealPlayer, WMP, PDF reader, etc. 
How do 'public' plugins differ from custom software plugins in [4] ? Well 
public plugins are more prevalent, that is they are shared in use across many, 
many, different websites, where as you'll never find a need for a custom plugin 
on sites like Amazon ! Also, public plugins also most often use 'standardised' 
protocol URI's such as rtp://, mms:// etc.