Re: VARLEN vs "BLOCK" Storage

[Pete Hall <pbhall@xxxxxxxxxx>]

At 07:13 06/14/2001, Michael Smith wrote:
>My manager is considering developing a storage algorithm to store a 
>variable length text field in a 32K record.
>
>He plans to using a unique key field with a "pointer" into the 32K record; 
>in essance "packing" multiple fields into one 32K block.
>
>I have suggest that a better method would be to define the text field as 
>VARLEN will an allocated length.
>
>0001.00      A          R TEST00
>0002.00      A            TEXT       32000          VARLEN(100)
>         ****************** End of data *************************
>
>My manager is concerned by the amount of storage that my suggestion would 
>use... to much DASD" (as defined by the DSPFD command).

I developed systems using something similar to the "packed" method back in 
the very early '80s on a Z-80 system running under a pre DOS version of the 
MS DOS operating system. In addition I linked one segment to the next so 
that I could retrieve multiple data segments in order. It was a pretty good 
approximation of a singly linked list. At that time, I thought it was 
really cool. That was, however, in an earlier century.

Try loading your file with empty records and see how the object size 
relates to the record count. DSPFD and DSPFFD always show records as if 
they were expanded to their maximum length. These commands also show 
timestamps as if they were 26 bytes in length, when they really only take 8 
on disk. A variable length field requires a length word (2 bytes), and 
beyond that, afaik, it's pretty much all data (there is some system 
overhead, but IBM manages it). Any part of the field that overflows the 
minimum length is actually stored in a separate physical storage location, 
and does not impact the length of a physical record. In addition, you don't 
have to deal with blown pointers.

If your manager is really bent on rolling his own variable length support, 
you might suggest looking at stream files. Personally, I think the variable 
length field, which finally made its way into RPG only recently, and is 
really easy to use and very reliable, is the only way to go.

hth
Pete
Pete Hall
pbhall@execpc.com
http://www.execpc.com/~pbhall/

+---
| This is the Midrange System Mailing List!
| To submit a new message, send your mail to MIDRANGE-L@midrange.com.
| To subscribe to this list send email to MIDRANGE-L-SUB@midrange.com.
| To unsubscribe from this list send email to MIDRANGE-L-UNSUB@midrange.com.
| Questions should be directed to the list owner/operator: david@midrange.com
+---