Re: CPYFRMIMPF and Unicode - UTF-16 in particular

Hi Chuck

I'm considering using CPY - not CPYFRMIMPF - to copy the STMF into a user space - haven't succeeded so far, maybe due to the things you describe here.

Failing that, I'll use the IFS APIs and be done with it. Once in a variable in RPG (possibly based on a pointer to the user space), I can figure out (guess) which encoding it is in. Then run that transform API, which looks very easy to use.

I'm considering some kind of requirement that would let us use the FROMCCSID to affect the FROMSTMF.

I see that CPY has a FROMOBJ parameter, and FROMCCSID is used on that for *TEXT. I think I really want *BINARY.

Will see if that works soon enough, I think.

Vern

On 3/3/2013 1:54 PM, CRPence wrote:

On 02 Mar 2013 15:56, Vernon Hamberg wrote:

<<SNIP>>
I am surprised that the FROMCCSID does not affect things when
working with IFS files - that kind of irks me. It seems it is
documented, I just find it odd.

Since its introduction in v4r3 until v5r2, the FROMCCSID parameter
had the following note on the syntax diagram: "This parameter is valid
only if FROMFILE is specified." However since v5r3, CL command syntax
diagrams are no longer available in the online documentation, and that
[at least for this command,] the /notes/ in the syntax diagram were lost
in the conversion to the effective /table of parameters/ that are now
being shown instead; a side-effect of the 5250 help and online
documentation being sourced the same. I would have expected that /note/
to have been moved into the FROMCCSID parameter of the newer online
documentation; just as I would the other /notes/ to have made their way
into the docs. I see none of them in the IBM i 7.1 docs:
http://publib.boulder.ibm.com/infocenter/iseries/v7r1m0/topic/cl/cpyfrmimpf.htm

Anyone who cares enough that the current documentation should be
[more] accurate, could submit an electronic reader-comment via the _Send
feedback_ link available at the top of every page of "iseries"
InfoCenter. I did not submit this one. Here is a link to the syntax
diagram with /notes/ in some old online documentation, which could be
used as reference for the /notes/ that are missing documentation:
http://publib.boulder.ibm.com/iseries/v5r2/ic2924/index.htm?info/cl/cpyfrmimpfsyn.htm

Anyhow...

The database file has the concept of a flat file. One representation
of the flat file can not have a CCSID assigned. That type of flat file
is the /program described/ file. Such a file always has CCSID(*HEX) as
the CCSID tagging, to denote that no translation should transpire for
its data.

Thus support was required for [assuming or] assigning a CCSID to the
data of the database file.mbr named on the FROMFILE parameter [i.e. a
database file as input *instead of* a STFM as input], because the input
as a program-described file would need to be, well, *described by* the
program. The import feature being the /program/ using that file, must
know how to describe the file. But only the user, by specification on a
parameter, can impart that specific detail to the program. Thus was
born the FROMCCSID parameter. This parameter was [and is still?] even
conditioned in its prompted appearance, appearing only upon the
prerequisite specification of a FROMFILE [i.e. the FROMCCSID appears
when prompting, only when the specified input-file is a database file;
except when using F9=All-parameters].

By contrast, the STMF was considered to always have a CCSID. The
stream file defined as a binary stream has the CCSID(65535) which means
"do not translate the data". That also implies the data is not "text"
data. The stream file defined as a text stream has a CCSID other than
65535, from which to translate the data. But because the import\export
is from\to *text* data, there is not really much relevance for a binary
stream.

If the STMF was incorrectly tagged as binary even though the contents
were text, the solution is to change the CCSID of that STMF [e.g. CHGATR
theSTMF *CCSID value(the_ccsid)]. Yet for a program-described file,
there is no ability to change the CCSID, so only an override
specification from the user on the command will suffice; i.e. CHGPF
thePgmDesc CCSID(the_ccsid) fails with CPD322D rc4 "The file is a
program described file.", so without FROMCCSID there would be no resolution.

However I must admit that [¿design?] conclusion arguably could be
considered somewhat of an oversight, because the /effective/ STMF as a
logical representation as a /file/ of data versus an actual STMF, might
also be unable to have a CCSID assigned. The *USRSPC as source for the
data would be an example of a logical STMF for which apparently no CCSID
can be set\established. CHGATR for example, can not set the CCSID for a
*USRSPC object, because it is a /QSYS.LIB object rather than a STMF.

Yet from the perspective of the stream-file features, the 8=Display
Attributes via WRKLNK shows the Job [default] CCSID for such logical
representations, and the STMF /copy/ features try very hard either to
ensure EBCDIC encoding is what gets there or to insist what is there
must be EBCDIC encoding. Even EDTF harps about unexpected ASCII data
with CPDB609, because the User Space being forcibly considered to have
an EBCDIC data due to the /job CCSID/ only being able to be an EBCDIC CCSID.

Maybe you could argue for the FROMCCSID to enable overriding the
CCSID for that scenario. Probably better than arguing for that for
using an actual STMF. I say that because IMO, the attempt by the
stream-file features to insist that *USRSPC data must be an EBCDIC
encoding seems quite off-base; while the stream-file features try very
hard to insist that is the case, there is no way to make it so. Since
text data of any encoding could have been placed there, how else but as
with the FROMCCSID parameter for program-described files, can the user
tell the import feature that the input data is something other than what
the stream-file feature [falsely] suggests?

Though I had never tried import from a User Space in the past, I had
no luck on a quick test on v5r3. It failed with a nasty exception in
data conversion when I had UTF8 or UTF16 data in the user space. But
then I found the import worked great when the CCSID of the data in the
User Space was in the job CCSID; i.e. when the data was in the CCSID
that the stream-file feature told me was the CCSID of the file, per
8=Display-Attributes.

Given that capability could be successfully argued for, then perhaps
best not forget the export. On v5r3 the CPYTOIMPF with a User Space as
target would not allow a specification of the codepage for the TOSTMF;
as would be expected both because the implementation is using the
stream-file copy methods, and the stream-file feature considers the
*USRSPC to have the job CCSID with no ability to change. The newer
STMFCCSID parameter may try to set the CCSID on the creation, and I am
not sure if that might fail similarly; although interestingly, qsh
cmd('touch -C 1200 /qsys.lib/mylib.lib/test.usrspc') created a *USRSPC
just fine, but of course shows the object with my Job [default] CCSID,
so the CCSID specification was apparently just ignored.