Re: CPYFRMIMPF and Unicode - UTF-16 in particular

Bruce -

Perhaps I was a bit vague in my response...

I am aware of QlgTransformUCSData, but what I meant by 'directly handle' did
not include the necessity of writing programs which utilize an API to
convert data from/to Unicode (like UTF-16 LE or UTF-16 LE with BOM). It
seems to me that IBM could have built this support into the CPYFRMIMPF and
CPYTOIMPF commands to make it easier for me to send/receive UTF-16 data to
my customer (a BI group), who prefers to get Unicode UTF-16 LE).

- sjl

"Bruce Vining" wrote in message
news:mailman.17117.1362008341.10847.midrange-l@xxxxxxxxxxxx...

The Transform UCS Data API, documented at
http://publib.boulder.ibm.com/infocenter/iseries/v5r4/topic/apis/qlgtrans.htm,
supports UTF-16 LE. The API has been available since V4R5.

There is not a specific CCSID for it, but you can certainly
convert/transform the data to a format that does have a CCSID.

Bruce Vining

On Wed, Feb 27, 2013 at 4:51 PM, TheBorg <sjl_abc@xxxxxxxxxxx> wrote:

Vern -
>From my Unicode research, IBM I cannot directly handle UTF-16 LE...
-sjl


"Vernon Hamberg" wrote in message
news:mailman.17096.1362001857.10847.midrange-l@xxxxxxxxxxxx...

Hi

We are on 7.1, I don't know the PTF level. I'm looking at importing
tab-delimited data from text files that are in little-endian UTF-16
(called Unicode in Notepad Save As). That is what Windows uses by
default. Yes, there is an option in Notepad to save as big-endian UTF-16
(called Unicode big-endian), and I may be recommending that. Or just use
ANSI. But mistakes do happen, and it'd be nice to have the program
handle things.

I also tried saving the text file with TextPad, which also has the 2
UTF-16 "endian" options. There is a difference in how Notepad and
TextPad do it - the former puts in the BOM at the start - the Byte Order
Marker, which is x'FEFF' for big-endian (normal order, the one IBM i
uses) and x'FFFE' for little-endian (bytes are in reverse order).

In any case, we want to be able to process Unicode text files. So I have
4 Unicode text files, tab-delimited.

*** It seems that CPYFRMIMPF converts individual bytes, not characters,
as I'll show below. ***

CPY will do the conversion correctly for the big-endian ones, but for
little-endian, the result is an empty stream file.

So double-clutch using CPY, then CPYFRMIMPF from the copy works.

2 questions:
1. Can CPYFRMIMPF actually do this correctly? you see the parameters I
tried.
2. Can IBM i handle UTF-16LE (little-endian)? Is there a CCSID for that,
like 1200 is for UTF-16BE (big-endian).

Thanks
Vern

uc-big-endian-notepad.txt (x'FEFF' at start of file)
uc-little-endian-notepad.txt (x'FFFE' at start of file)
uc-big-endian-textpad.txt (nothing at start of file)
uc-little-endian-textpad.txt (nothing at start of file)

The CPYFRMIMPF command I'm testing looks like this - sample result
follows the first one, others are similar - a blank in clear text is
actually x'00' - and note that x'FEFF' is converted to x'8EDF'.

CPYFRMIMPF FROMSTMF('uc-big-endian-notepad.txt') TOFILE(VERN/FLAT3000)
MBROPT(*REPLACE) FROMCCSID(1200) TOCCSID(37) RCDDLM(*ALL) FLDDLM('~')
þÿ b r a n d s t o r e _
8D0809080908000A0A09090806
EF02090105040502030609050D

CPYFRMIMPF FROMSTMF('uc-little-endian-notepad.txt')
TOFILE(VERN/FLAT3000) MBROPT(*REPLACE) FROMCCSID(1200) TOCCSID(37)
RCDDLM(*ALL) FLDDLM('~')

CPYFRMIMPF FROMSTMF('uc-big-endian-textpad.txt') TOFILE(VERN/FLAT3000)
MBROPT(*REPLACE) FROMCCSID(1200) TOCCSID(37) RCDDLM(*ALL) FLDDLM('~')

CPYFRMIMPF FROMSTMF('uc-little-endian-textpad.txt')
TOFILE(VERN/FLAT3000) MBROPT(*REPLACE) FROMCCSID(1200) TOCCSID(37)
RCDDLM(*ALL) FLDDLM('~')

--
This is the Midrange Systems Technical Discussion (MIDRANGE-L) mailing
list
To post a message email: MIDRANGE-L@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/midrange-l
or email: MIDRANGE-L-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/midrange-l.