RE: Variable length records and network files

[Buck Calabro <mcalabro@xxxxxxxxxxxx>]

On Friday, May 15, 1998 6:47 AM, Tony.Minehan@hr-m.b-m.defence.gov.au 
[SMTP:Tony.Minehan@hr-m.b-m.defence.gov.au] wrote:
>      Every 2 weeks we are sent a file from another government agency. 
The
>      file is text only and the record length ranges from 8 bytes (most
>      records) to 251 bytes (very few, but very important records). An
>      average number of records is around 6 million. The file is sent 
from
>      a
>      S370 via SNA (FTP is not an option at present) and arrives here as 
a
>      network file. Currently we receive the file into a single field 251
>      byte text file and then do our processing. Over the next couple of
>      fortnights, the file is likely to grow to around 65 million 
records.
-snip-
>      message. We are on V3R2. Any ideas?

Tony,
  Neil wrote about using the IFS to store the file; I think this might 
something for you to look into.  I copied a source file member to the IFS 
with the following command:
CPYTOSTMF FROMMBR('/qsys.lib/buck.lib/qrpglesrc.file/ifsapi.mbr') 
          TOSTMF('/qopensys/ifsapi.txt')
          CVTDTA(*TBL)
          TBL('/qsys.lib/qascii.tbl')

You can also FTP the file from your DB2 file system to the IFS:
ftp loopback
namefmt 1
put /qsys.lib/buck.lib/qrpglesrc.file/ifsapi.mbr /qopensys/ifsapi.txt
quit

I can use WRKLNK (from the /400) to see it, or I can use Windows Explorer, 
Network Neighbourhood to see it.  It takes up 24k on the DB2 file system, 
and 8k on the IFS.  I'd bet that your savings would be greater.  Using the 
conversion table allows me to read it from the PC if I want to (FTP 
converts automatically.)  I have some code cobbled together from posts 
kindly placed here on Midrange-L that will read an IFS file in an RPG IV 
program.  Feel free to email me if you want a copy: it'll need work, but 
it's a start...

A second option involves making multiple PF's out of your single 
downloaded file.  Assuming that there's a record ID that allows you to 
distinguish between the 8 byte records and the 251 byte records, you can 
always spin through your input file and put the 8 byte records into an 8 
byte PF, the 251 byte records into a 251 byte file, etc.  This may prove 
unwieldy if you need to preserve the sequence of records.

Then there's the option of a quickie RPG IV program to EVAL 
VarData=%trim(FixedData)...

Again, thanks to all the midrangers who've *given* me the info to be able 
to do this type of stuff!

Buck Calabro
Commsoft, Albany, NY
mailto:mcalabro@commsoft
+---
| This is the Midrange System Mailing List!
| To submit a new message, send your mail to MIDRANGE-L@midrange.com.
| To unsubscribe from this list send email to MIDRANGE-L-UNSUB@midrange.com.
| Questions should be directed to the list owner/operator: david@midrange.com
+---