On 11/17/2015 6:34 AM, rob@xxxxxxxxx wrote:

I would find your technique onerous to the system.

I agree. I was just suggesting how I would do it from a pure green screen

Then there's the temporary space to consider. If you have a 10GB stream
file being posted into your 200GB stream file and you copy them into DB2,
do the merge, and then put them back outside of the qsys.lib system it
really starts to add up.

In that scenario, my 1st question is why do you even need a 210GB CSV that
your periodically appending data to. It seems to me that a bunch of
compressed CSV files would do better. Naturally, replace CSV with EDI, or
other efficient format.

Now, if you were dealing with small amounts of data, I'd actually suggest
using xml with a well defined XSD schema. A lot of CPU and IO overhead in
exchange for validation that gives well defined errors that can be
duplicated on both sides of the transmission.


As an Amazon Associate we earn from qualifying purchases.

This thread ...


Follow On AppleNews
Return to Archive home page | Return to MIDRANGE.COM home page

This mailing list archive is Copyright 1997-2021 by midrange.com and David Gibbs as a compilation work. Use of the archive is restricted to research of a business or technical nature. Any other uses are prohibited. Full details are available on our policy page. If you have questions about this, please contact [javascript protected email address].

Operating expenses for this site are earned using the Amazon Associate program and Google Adsense.