Re: Calculate hash for a *very* large file

On Thu, Jun 18, 2009 at 5:36 PM, Scott Klement <rpg400-l@xxxxxxxxxxxxxxxx>wrote:

You don't have to load the entire file into memory ahead of time in
order to perform an MD5 hash.

I wrote an article that demonstrates creating an MD5 hash over a stream
file in the IFS. Not quite what you're looking for, but it's written in
RPG, you should be able to modify it for your purposes. The point is...
it performs an MD5 hash over a (potentially) very large file without
loading the whole thing into memory first.

Thanks Scott! I downloaded the source code and compiled it. Got it to work
on a 150,000 record file, but it choked on a file with 4,750,000 records.
Both files had a record length = 2000. Error message given was: Object is
too large to process. (CPE3525) The problem is that message is issued in
the ReportError procedure, and I am unable to determine where the error
actually occurred.

I'll take another look at this in the morning.

- Dan