Re: CGIDEV2 doesn't like CCSID 1208? -- MIDRANGE-L

Hi Adam,

I'm not sure what difference it's going to make... When CGIDEV2 reads your template file into memory, it converts it to EBCDIC. When Apache sends it back to the browser, it converts from EBCDIC to (by default) ISO-8859-1 (CCSID 819). So it doesn't really matter what your stream file in the IFS is coded as -- it's going to get translated to EBCDIC and then back to 819 before being sent to the browser.

However, if you want to change CGIDEV2 to work with UTF-8 (or any other IFS CCSID that's either double or mixed-byte) I did post the needed code changes in the following message from Easy400Group's mailing list archives:
http://tech.groups.yahoo.com/group/Easy400Group/message/10335

CGIDEV2 was originally written for pre-V5R1 -- and the IFS didn't support CCSIDs back then, it was codepage based. You can't use multi-byte or mixed-byte encodings with a codepage. (UTF-8 is mixed byte.)

If you changed your O_CODEPAGE to O_CCSID on the first open() statement, that would fix that statement -- but the next statement would fail. When you specify O_TEXTDATA without O_CCSID, it tries to do codepage translation rather than CCSID translation, which again wouldn't work with a mixed byte encoding, so you'd have to addd O_CCSID and a CCSID value of 0 (for "current job's default") onto the 2nd open() statement.

Adam Glauser wrote:

I have a program which uses CGIDEV2 to create an XML file. It reads a template from the root filesystem, fills in the template, and writes the completed XML file to the same directory (with a different filename, of course). I've worked around the problem, but it's nagging me and I can't figure out why this is happening.

I realized as I was updating the program for the next tax year that I was generating the file as CCSID 819 (ISO-8859-1), but my <?xml ...?> header stated UTF-8. I must have just used a text editor to convert it before sending it last year.

I tried changing the output file to be CCSID 1208 (UTF-8) by changing the CCSID parameter on wrtHTMLtoSTMF(), but I get errno=3490 (ECONVERT) when CGIDEV2 attempts to re-open the output file to write the output.

<goryDetails>
wrtHTMLtoSTMF does something like this:
create a temporary filename (desired filename + 'TEMP' + job number)
unlink tempfile if it exists
open tempfile to create it (empty with proper CCSID) with:
C eval FileHandle = open(
C StmfTemp:O_CREAT + O_WRONLY +
C O_TRUNC + O_TEXTDATA + O_CODEPAGE +
C O_LARGEFILE:
C S_IRWXU + S_IROTH + S_IXOTH:CodePage)
where CodePage is 1208, which I've passed as a parameter.

re-open tempfile for writing:
C eval FileHandle = open(
C StmfTemp:O_CREAT + O_WRONLY +
C O_TRUNC + O_TEXTDATA:
C S_IRWXU + S_IROTH + S_IXOTH)

At this point, I get the ECONVERT error. I found a forum post* from Scott Klement stating that CGIDEV2 doesn't support multi-byte CCSIDs for writing to STMF, but I don't think UTF-8 counts. I'm also not sure why the above would fail for multi-byte CCSIDs, the documentation for O_TEXTDATA doesn't mention such a limitation.
</goryDetails>

If I simply change the CodePage parameter to 819 and my template to use <?xml version="1.0" encoding="ISO-8859-1"?>, everything works fine.

FWIW, I one thing I tried was making the template file UTF-8 encoded. I learned about the BOM bytes** that some software writes at the beginning of multi- or variable-byte encoded files. CGIDEV2 doesn't like the BOM bytes, but seems to work fine with a template file that is CCSID 1208.

I don't know where to look next - any suggestions?
Adam

* http://forums.systeminetwork.com/isnetforums/archive/index.php?t-47517.html (or http://tinyurl.com/6d4wk8 )

** http://en.wikipedia.org/wiki/Byte-order_mark#Representations_of_byte_order_marks_by_encoding (or http://tinyurl.com/5rojua)