Re: CSVR4 and UTF-8

Hi Scott,

For 1) -- That is what I expressed earlier this day (March, 6th):

"In this example there is an unexpected x'00' before x'00A0', so that we would get an extra x'00' byte if we scanned for x'00A0'. Looks like a bug in fgets()."

In addition to that it might be worth to know that there is a PTF for the Unix-type read() function, which fixes a problem for UTF8 files with a BOM. We spotted that problem earlier this year:

https://www.ibm.com/support/pages/ptf/MF67069

For 2) -- You are correct. I did not look at your code carefully enough. When it comes to an "If" I always think about the "else". Hence I automatically cleared "peFldData".

Regards,

Thomas.

Am 06.05.2020 um 23:33 schrieb Scott Klement:

Hello Thomas,

For (1) -- I agree that it is crashing, but I do not see any mistakes? It looks to me like the fgets() routine from IBM does not work for Unicode data where the line ends in LF.  (I tried different settings for CRLN= on the open, that did not help.)

I am working on rewriting CSVR4 to use the Unix-type open(), read(), lseek() and close() APIs instead of the POSIX C fopen(), fseek(), fgets() and fclose() APIs to solve this problem.   Since the Unix-type APIs do not do any sort of buffering or have the ability to read according to text-file lines, I will need to write that functionality myself.  It may take several hours, but I'll let you know when it is available.


For (2) -- you are correct.  Thank you for finding that! However, it is not necessary to set peFldData = '' (it is already cleared a few lines earlier).  Your fix would be simpler as follows:

      if max > 0;
         %len(peFldData) = max;
         %subst(peFldData:1:max) = myEbcData;
      endif;




On 5/6/2020 1:47 AM, Thomas Raddatz wrote:

Hi Scott,

Very fast reaction and good job. This is truly what I think and no ironie dspite the two errors I found.

1) CSVR4 crashes when called for a CSV file with line delimiter LF.

2) CSVR4 crashes when fields are empty.

I fixed the second problem with the following code snippet, applied 4 times in CSVR4:

                if max = 0;
                   peFldData = '';
                else;
                   %len(peFldData) = max;
                   %subst(peFldData:1:max) = myEbcData;
                endif;

The first problem waits for being addressed.

CSVDEMO4 is great and I am sure it will help Greg.

Regards,

Thomas.