Re: How to avoid file corruption transferring members from one file to another

On 10/21/2014 2:40 PM, Gary Thompson wrote:

I have program FTP_ASN_1 sending EDI 856 ASN documents to a remote server.
If an FTP connection can't be made to the remote, the ASN document is
copied to a member file WAIT_Q, and the ASN program continues with the
next ASN as our warehouse completes the next shipment.

We may not be able to connect to the remote FTP server for a few minutes
or some hours, possibly a day.

I have program FTP_ASN_2 pulling ASN documents that have not yet been sent
from members in a file I call SEND_Q. A separate file is used to avoid contention
with FTP_ASN_1.

When FTP_ASN_2 finds no member in SEND_Q, it stops its FTP process and looks
to copy any members from WAIT_Q to SEND_Q and then remove all _copied_
members from SEND_Q.

My question is how to avoid 'collisions' between FTP_ASN_1 and FTP_ASN_2 during
the time members are being 'moved' from WAIT_Q to SEND_Q ?

My ideas have so far included:

Use a single data area and require both FTP_ASN_1 and FTP_ASN_2 to lock the data
area before adding new or copying/deleting WAIT_Q members.

Have both FTP_ASN_1 and FTP_ASN_2 get exclusive control of WAIT_Q with ALCOBJ.

Looking for any advice or comments about design pitfalls . . .

The multi-member file was a good solution for transitioning from card to
disk. In the days of very smart database optimisers, putting a key on
the record and keeping all the records together in one table / member is
more useful. Rather than copy / move records from place to place,
normalise the data into a header (with status) and detail. Instead of
moving from WAIT_Q to SEND_Q, just change the status of the header from
WAIT to SEND.

I fully realise that this advice is easy to give and not so easy to
follow, as I have had to do this very thing several times, and it's
always been a struggle. Start out by creating the header file and use
that to maintain the status of each FTP session. When the send process
runs out of orders in SEND state, it can call the procedure to move the
WAIT orders to SEND; for now, that procedure will do what it always did
(copy from WAIT_Q to SEND_Q) but it'll also do one more thing: after the
copy is complete, it will change the status to SEND, at which time the
send process knows it can safely handle the order.