Personally, I think there is nothing better suited to this than
Python. (I'm open to the idea that there might be things that are
*equally* well suited. ;)
If all the data is going to be processed on the i, then use iSeries
Python. It comes with the standard Python CSV module for reading and
writing CSVs (as IFS stream files) which can handle the quirks of
Excel-generated CSVs (if that's indeed where they come from) or
less-quirky CSVs as the case may be. It can access IBM physical files
via SQL or RLA about equally easily, and do so dynamically (i.e. you
don't need to know file names or record formats ahead of time).
For any kind of logic involving arrays or mapping, or strings, or
regex, or anything at all dynamic, it is far and away better than RPG.
John