# Re: CPYFRMIMPF and Unicode - UTF-16 in particular

     Subject: Re: CPYFRMIMPF and Unicode - UTF-16 in particular From: CRPence Date: Sun, 03 Mar 2013 11:54:31 -0800 List-archive: List-help: List-id: Midrange Systems Technical Discussion List-post: List-subscribe: , List-unsubscribe: ,

On 02 Mar 2013 15:56, Vernon Hamberg wrote:
<<SNIP>>
I am surprised that the FROMCCSID does not affect things when
working with IFS files - that kind of irks me. It seems it is
documented, I just find it odd.

Since its introduction in v4r3 until v5r2, the FROMCCSID parameter had the following note on the syntax diagram: "This parameter is valid only if FROMFILE is specified." However since v5r3, CL command syntax diagrams are no longer available in the online documentation, and that [at least for this command,] the /notes/ in the syntax diagram were lost in the conversion to the effective /table of parameters/ that are now being shown instead; a side-effect of the 5250 help and online documentation being sourced the same. I would have expected that /note/ to have been moved into the FROMCCSID parameter of the newer online documentation; just as I would the other /notes/ to have made their way into the docs. I see none of them in the IBM i 7.1 docs:
http://publib.boulder.ibm.com/infocenter/iseries/v7r1m0/topic/cl/cpyfrmimpf.htm

Anyone who cares enough that the current documentation should be [more] accurate, could submit an electronic reader-comment via the _Send feedback_ link available at the top of every page of "iseries" InfoCenter. I did not submit this one. Here is a link to the syntax diagram with /notes/ in some old online documentation, which could be used as reference for the /notes/ that are missing documentation:
http://publib.boulder.ibm.com/iseries/v5r2/ic2924/index.htm?info/cl/cpyfrmimpfsyn.htm

Anyhow...

The database file has the concept of a flat file. One representation of the flat file can not have a CCSID assigned. That type of flat file is the /program described/ file. Such a file always has CCSID(*HEX) as the CCSID tagging, to denote that no translation should transpire for its data.

Thus support was required for [assuming or] assigning a CCSID to the data of the database file.mbr named on the FROMFILE parameter [i.e. a database file as input *instead of* a STFM as input], because the input as a program-described file would need to be, well, *described by* the program. The import feature being the /program/ using that file, must know how to describe the file. But only the user, by specification on a parameter, can impart that specific detail to the program. Thus was born the FROMCCSID parameter. This parameter was [and is still?] even conditioned in its prompted appearance, appearing only upon the prerequisite specification of a FROMFILE [i.e. the FROMCCSID appears when prompting, only when the specified input-file is a database file; except when using F9=All-parameters].

By contrast, the STMF was considered to always have a CCSID. The stream file defined as a binary stream has the CCSID(65535) which means "do not translate the data". That also implies the data is not "text" data. The stream file defined as a text stream has a CCSID other than 65535, from which to translate the data. But because the import\export is from\to *text* data, there is not really much relevance for a binary stream.

If the STMF was incorrectly tagged as binary even though the contents were text, the solution is to change the CCSID of that STMF [e.g. CHGATR theSTMF *CCSID value(the_ccsid)]. Yet for a program-described file, there is no ability to change the CCSID, so only an override specification from the user on the command will suffice; i.e. CHGPF thePgmDesc CCSID(the_ccsid) fails with CPD322D rc4 "The file is a program described file.", so without FROMCCSID there would be no resolution.

However I must admit that [¿design?] conclusion arguably could be considered somewhat of an oversight, because the /effective/ STMF as a logical representation as a /file/ of data versus an actual STMF, might also be unable to have a CCSID assigned. The *USRSPC as source for the data would be an example of a logical STMF for which apparently no CCSID can be set\established. CHGATR for example, can not set the CCSID for a *USRSPC object, because it is a /QSYS.LIB object rather than a STMF.

Yet from the perspective of the stream-file features, the 8=Display Attributes via WRKLNK shows the Job [default] CCSID for such logical representations, and the STMF /copy/ features try very hard either to ensure EBCDIC encoding is what gets there or to insist what is there must be EBCDIC encoding. Even EDTF harps about unexpected ASCII data with CPDB609, because the User Space being forcibly considered to have an EBCDIC data due to the /job CCSID/ only being able to be an EBCDIC CCSID.

Maybe you could argue for the FROMCCSID to enable overriding the CCSID for that scenario. Probably better than arguing for that for using an actual STMF. I say that because IMO, the attempt by the stream-file features to insist that *USRSPC data must be an EBCDIC encoding seems quite off-base; while the stream-file features try very hard to insist that is the case, there is no way to make it so. Since text data of any encoding could have been placed there, how else but as with the FROMCCSID parameter for program-described files, can the user tell the import feature that the input data is something other than what the stream-file feature [falsely] suggests?

Though I had never tried import from a User Space in the past, I had no luck on a quick test on v5r3. It failed with a nasty exception in data conversion when I had UTF8 or UTF16 data in the user space. But then I found the import worked great when the CCSID of the data in the User Space was in the job CCSID; i.e. when the data was in the CCSID that the stream-file feature told me was the CCSID of the file, per 8=Display-Attributes.

Given that capability could be successfully argued for, then perhaps best not forget the export. On v5r3 the CPYTOIMPF with a User Space as target would not allow a specification of the codepage for the TOSTMF; as would be expected both because the implementation is using the stream-file copy methods, and the stream-file feature considers the *USRSPC to have the job CCSID with no ability to change. The newer STMFCCSID parameter may try to set the CCSID on the creation, and I am not sure if that might fail similarly; although interestingly, qsh cmd('touch -C 1200 /qsys.lib/mylib.lib/test.usrspc') created a *USRSPC just fine, but of course shows the object with my Job [default] CCSID, so the CCSID specification was apparently just ignored.

<<SNIP effective bitching and FUD>>

Ugh. I chose to not respond publicly because I have little positive to say about the users or the pre-v5r4 feature.

Return to Archive home page | Return to MIDRANGE.COM home page

This mailing list archive is Copyright 1997-2013 by MIDRANGE dot COM and David Gibbs as a compilation work. Use of the archive is restricted to research of a business or technical nature. Any other uses are prohibited. Full details are available here. If you have questions about this, please contact