Re: CSVR4 and UTF-8

Hi Scott,

I created a iRPGUnit test service program for CSVR4. It covers the Unicode part of CSVR4 and it could be enhanced step-by-step whenever needed.

Test files are created and removed automatically.

Send me a PM if you want to include it into the CSVR4 package.

Regards,

Thomas.

Am 06.05.2020 um 05:54 schrieb Scott Klement:

Hi Greg,

I've updated the CSVR4 utility on my web site:

http://www.scottklement.com/csv/

(Be careful that your browser hasn't cached the old copy when you re-download it.)

The CSV_open() function now has an additional parameter that lets you tell CSVR4 to do its internal processing using Unicode fields.  You would call it like this:

file = CSV_open('/path/to/ifs.csv': *omit:*omit:*omit: *on);

Passing *ON in the final parameter enables the internal Unicode support, with that it will convert your file (assuming it is properly labelled as CCSID 1208) into CCSID 1200 (UTF-16 Unicode) for internal processing within CSVR4.

There is also a new function called CSV_getFldUni() which returns a Unicode field (vs. CSV_getFld which returns EBCDIC).   You would call it like this:

 D name            s             50c   varying ccsid(1200)
   .
   .
      dow CSV_loadRec(file);

         CSV_getFldUni(file: name: %size(name));
         .
         . ...other fields here, etc...
         .
      enddo;

This should be easy to understand if you're already familair with CSVR4.  The only difference between CSV_getFld and CSV_getFldUni is that the latter returns teh data into an RPG CCSID(1200) type 'C' (VARUCS2) field.

Since the field is now in Unicode, your code should run without errors. Then you could use %XLATE to convert the curly quotes to straight quotes -- which do exist in EBCDIC -- to solve your problem.

There's a full example named CSVDEMO4 included in the download that shows reading the file in Unicode, replacing the quotes, and converting the fixed-up field into EBCDIC so that it can be printed.

Good luck!

-SK