Re: CSV files with Byte Over Marks

On 6/19/2018 4:15 PM, Soucy, Michael wrote:

For your job:
DSPJOB option 2. Page down 3(?) times, it's the Coded character set identifier.

The value I see is: 65535.

For your system value:
DSPSYSVAL QCCSID

The system value is the same as the job.

Yuck.
The bad news is that your system is configured as though it were the
year 1990. The good news is that you can change that. I changed mine
at a time when I applied PTFs and needed to do an IPL anyway, but there
are definitely people who changed it in the middle of the day.
CHGSYSVAL QCCSID(37)

The best news is that you can change just /your job/ immediately and
test again!
CHGJOB CCSID(37) - Use 37 if you are in the US and have an English
language installed.
Test the program again.

If that works (and I suspect it will), you can make this process run
correctly without changing the system CCSID by doing a SBMJOB ...
CCSID(37). Or using a user profile whose CCSID is 37.

Note for the archives: 37 should only be used if you want US English EBCDIC.

For your files:
DSPFD lib/file Page down 2(?) times, it's the CCSID

The file is on the IFS. It currently says " 1208", because that is what I changed it to using the CHGATR command as suggested earlier. What it was before was 1252. This format has the BOM which is what I'm trying to get rid of.

I'd say that the BOM isn't the issue: the CCSID(65535) probably is.
I'd also say that you probably want to keep the BOM, but it seems quick
to test.

That's why I wanted you to look at the IFS file in hex - to see exactly what bit patterns are physically stored in there.

When I look at the file in hex mode this is what I see.

Hex: EFBBBF22 30303031
Char: "0001

Does this mean the file is in ASCII format?

Normally, files destined for Windows UTF-8 has the BOM, and there's
definitely a BOM in this data. So I'd call it UTF-8. That said, UTF-8
is a variable length encoding, where one 'character' can be encoded with
one, two, three, or four 8-bit bytes. By rule, the first 128
'characters' are mapped to the same 8-bit encoding that ASCII 819 is
mapped to. This is for compatibility with older software.
https://en.wikipedia.org/wiki/UTF-8