RE: design question, how to handle this project

> From: Joe Sam Shirah
> 
> in a homogeneous environment ( which can here
> be expanded to any system with a DBMS and JDBC driver ) that just
wants to
> grab the data for processing, these layers introduce complexity and
impact
> performance.  Remember that SQL provides a standard for formats too
and
> you can just talk directly to the DBMS engine over the network.

If it's a truly homogenous environment, DDM is another option.  I
haven't compared DDM to JDBC access over a network, but it's certainly
something to consider.

Another issue is whether this is a pure query or a data consolidation
environment.  Depending on the complexity of the data to be returned, a
message based approach with remote data queues may be an option as well.
It does require code on the client, but it may not be a bad idea.

There are also security issues.  A typical JDBC approach would require a
database of user IDs and passwords on any machine.  That's one of the
knocks I always have with JDBC.  Until a better network-wide
authentication mechanism is in place, this is a security hole that could
cause fits in an audit.

You also have the issue of whether or not the data from the multiple
machines is itself meant to be collated.  That is, do we want data from
machine one interleaved with data from machine two?  That's beyond the
capabilities of any of the mechanisms discussed so far, and in fact
might best be handled by a tiered client/server interface.  There are
two answers for this business problem: you can either read all the data,
write it to a temporary file, and then reread it in the correct
sequence, or you can read it one record at a time from each machine
using a controller to perform what would in effect be matching record
logic.  The latter is far more effective and much better from a
performance standpoint (especially when optimizing to get the first page
out as fast as possible), although at the cost of extra overhead.

As usual, the question is not as simple as "how do I do this?"  It is
instead "what are the possible solutions and which best fits my business
requirements?"  And without a better idea of the business requirements,
any advice would be incomplete.

Joe