RE: How to avoid file corruption transferring members from one file to another

Buck,
Thank you, your suggestion for a normalized DB is spot on, and where I want to get to, but time is scarce.
Today, I'm creating one data area both FTP_ASN_1 and FTP_ASN_2 must lock/read/update/release before changing WAIT_Q member(s).
I'm depending on that data area to prevent member corruption.

-----Original Message-----
From: MIDRANGE-L [mailto:midrange-l-bounces@xxxxxxxxxxxx] On Behalf Of Buck Calabro
Sent: Wednesday, October 22, 2014 8:34 AM
To: midrange-l@xxxxxxxxxxxx
Subject: Re: How to avoid file corruption transferring members from one file to another

On 10/21/2014 2:40 PM, Gary Thompson wrote:

I have program FTP_ASN_1 sending EDI 856 ASN documents to a remote server.
If an FTP connection can't be made to the remote, the ASN document is
copied to a member file WAIT_Q, and the ASN program continues with the
next ASN as our warehouse completes the next shipment.

We may not be able to connect to the remote FTP server for a few
minutes or some hours, possibly a day.

I have program FTP_ASN_2 pulling ASN documents that have not yet been sent
from members in a file I call SEND_Q. A separate file is used to avoid contention
with FTP_ASN_1.

When FTP_ASN_2 finds no member in SEND_Q, it stops its FTP process and
looks to copy any members from WAIT_Q to SEND_Q and then remove all
_copied_ members from SEND_Q.

My question is how to avoid 'collisions' between FTP_ASN_1 and
FTP_ASN_2 during the time members are being 'moved' from WAIT_Q to SEND_Q ?

My ideas have so far included:

Use a single data area and require both FTP_ASN_1 and FTP_ASN_2 to
lock the data area before adding new or copying/deleting WAIT_Q members.

Have both FTP_ASN_1 and FTP_ASN_2 get exclusive control of WAIT_Q with ALCOBJ.

Looking for any advice or comments about design pitfalls . . .

The multi-member file was a good solution for transitioning from card to disk. In the days of very smart database optimisers, putting a key on the record and keeping all the records together in one table / member is more useful. Rather than copy / move records from place to place, normalise the data into a header (with status) and detail. Instead of moving from WAIT_Q to SEND_Q, just change the status of the header from WAIT to SEND.

I fully realise that this advice is easy to give and not so easy to follow, as I have had to do this very thing several times, and it's always been a struggle. Start out by creating the header file and use that to maintain the status of each FTP session. When the send process runs out of orders in SEND state, it can call the procedure to move the WAIT orders to SEND; for now, that procedure will do what it always did (copy from WAIT_Q to SEND_Q) but it'll also do one more thing: after the copy is complete, it will change the status to SEND, at which time the send process knows it can safely handle the order.

--
--buck

'I had nothing to offer anybody except my own confusion' - Jack Kerouac
--
This is the Midrange Systems Technical Discussion (MIDRANGE-L) mailing list To post a message email: MIDRANGE-L@xxxxxxxxxxxx To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/midrange-l
or email: MIDRANGE-L-request@xxxxxxxxxxxx Before posting, please take a moment to review the archives at http://archive.midrange.com/midrange-l.