Re: Something to read overlunch: Eclipse IDE in thebrowser

Aaron Bartell wrote:

Use data queues and send "end of data set" messages so the queue knows when
it has received all entries. A similar approach would be to make data queue
records as long as possible and "chunk" multiple logical entries onto one
data queue record. Each data queue record would have to have a header area
where you would denote whether there were additional records needing to be
read.

Thoughts?

Data queues have certain properties that make them appealing, specifically the ability to have a single server support multiple clients. I've done a lot of that over the years; it has pluses and minuses. It's best suited for situations where your server load needs to be controlled; you can more easily control the resources assigned to a given activity by controlling its server.

However, for pure OLTP (and the even faster response time requirements of Web 2.0), I've moved almost entirely to a program call approach. As server CPW grows in leaps and bounds, its more about call-to-call performance rather than load control, and for that nothing beats direct program calls. And its quite easy to do "chunking" with program calls - you simply call the program multiple times. One of the simplest approaches is to call a program with two data structures: a control DS and a data DS. The control DS is used to indicate the operation to perform and to return the result code. The data DS is simply for passing data. To emulate a "result set", you perform an "init" opcode, and then start doing "get next" operations. The control DS returns a flag indicating EOF.

One of the many benefits of this approach is that paging is entirely under control of the client. It can start or stop reading records at any time and reset the control pointers. You can even use embedded SQL in your called program with a cursor and in effect page your SQL calls.

It's a great architecture.

Joe