RE: How to present large datasets on the Web?

(this is a composite of several messages)

>is it an option for you to show pieces of the phone
>bill and not the entire 5000 right up front?

That's how it's set up; a summary that the end-user will drill down into in
order to get details.  The interesting thing is that a commercial entity can
have tens of thousands of calls in the smallest breakdown (i.e. AT&T long
distance, One Rate Plan).

>What are your customers using the website
>for, printing out a copy of the bill?

No, they still get a paper copy of the bill (legal requirement).  This is
for electronic presentation.  The trick is that most end-users' bills have
only a few dozen long distance calls, but if we do not accommodate the
commercial accounts we'll never sell it.

>Does the Web person need to go through an RPG pgm
>for security reasons?

The idea was to re-use our existing business logic as much as possible.

>Who would want to view an HTML page that big?

A commercial customer looking to do a bunch of scans on phone numbers.  And
before you ask, yes, it was presented as a requirement.  I countered with a
search engine.  We'll see how that turns out.

>So, the best way to do that is to convert the
>invoice into pdf format.

This was floated as an option, but the customer wanted plain HTML.  I will
suggest this option again; maybe the customer will listen to the voice of
someone who has done this already...

>Also provide a link to save the whole set in a .csv
>file from the ifs. They can use it in Excel.

That is an option being floated as well.

>But I can still provide a link to all the data, yet
>display one page at a time to the browser.

There's the crux of the problem.  How do you do this?  JDBC?  Like I said,
the original plan was to reuse as much of the existing RPG logic as
possible.  In a Java ProgramCall with connexion pooling environment, I don't
think I can rely on the same job running the "continued" request.  That is,
client A runs the RPG program which does a SETLL and READE loop for 100
records.  Client B runs the same RPG program which re-positions the file
pointer and returns it's own set of 100 records.  Client A comes back asking
for the next page...  In this environment I would have to supply the next
key to the Java client I guess.  And what about 'page back?'  Sigh.

>Implemented with Net.Data and SQLRPGLE programs. Net.Data
>is also used to produce the html and csv pages in batch.
>(Not called from the HTTP server, a job of its own on
>a separate jobq in a separate sbs)

Thanks Anton!  Do you have several jobs running in batch or just one?

>If you have a few big customers that have to much data for a
>web page, it may be better to transfer that data to their
>computer, and let them process it localy.

That's another good suggestion.  Everybody has been focussed on displaying
the results in the web page, but this may be a better solution for the large
customer.  And even the small customer can choose to download...

Thanks for the suggestions...  Good discussion!
  --buck