Re: How to present large datasets on the Web?

At 09:03 PM 12/2/02, you wrote:

Hiya guys

In the context of providing/exchanging  data via HTML - say with SOAP, Web
services or some kind of process that hoovers up HTML into a database-  it
seems to me that the idea of sending 5000 records over the web is not as
unlikely a scenario as it might seem.

Certainly there are some data transfer needs out there.  That wasn't the
impression I got from the Buck's request, though.  Human operators are good
with a 'screenful' at a time.  If he is really trying to move all the data,
as in some kind of batch request, then I would consider constructing what
an earlier poster to this thread proposed - a custom socket server.  They
aren't terribly difficult to write, even in RPG, and they are supposedly
easy to read from Java.

Given Buck's original request, I wonder if there is a way to send the first
page of the HTML request (equivalent to the first page of the subfile) and
somehow continue extracting the remainder of the data and cache it so that
the query is executed once, but subsequent extracts do not have to repeat
the database access.

Many things can be cached.  Some web setups cache pages.  Some have caches
of persistent database connections.  I'm sure that a database recordset
could somehow be cached, but I've never tried it.   One reason why I
haven't messed with building caches is that -- on an iseries anyway -- they
seem perhaps as likely to wind up back on disk as they are to stay resident
in memory.  So, why bother?  Also, data in caches does go stale -- and I
usually want my users to see the most up to date data.