Re: Suggestions for working with mixed CCSIDs (37 and 13488

Thanks Bruce.

I had already thought about copying the DDS for some of the legacy files and changing the character fields to graphic and CCSID 13488, but had not thought of the CHGPF options. For now, we are running the company on the legacy system (CCSID 37) and testing on the new system (CCSID13488) with different software on another machine. Our main problem at this time is getting sample data from the legacy system to populate portions of the data base on the new system. So far, it looks like our easiest method might be to temporarily duplicate some data from the old system to the new system and change the DDS to CCSID 13488.

After we go live on the new system, converting the legacy database sounds like a good option, if we do not need to use the legacy software. I will run these options by the people I am working with.

At 04:41 AM 1/28/2008, you wrote:

Hi, Dave

A few observations:

1. I don't know that I agree (or that you really mean to say) "Working with the UCS-2 fields is kind of a pain". I believe the pain you are feeling is related to the migration of existing data from from data type to another. Migration can be (OK, is) painful but UCS-2 is the right way for any application to handle character based data if we're living in the global world of the 21st century :-)

2. CPYF with *MAP to UCS-2 should certainly work. For that matter, changing the DDS of the migrated files, changing the character fields to Graphic and specifying CCSID(13488), and then using CHGPF specifying the DDS source file, will also work (without having to duplicate the migrated data). Along the same lines ALTER TABLE should also work.

3. I suspect that some of the problems you are encountering when working with a mixed encoding scheme (EBCDIC and Unicode) environment will disappear when everything is in Unicode -- though I don't use the tools you mention so can't say for sure. If these tools just don't play well with Unicode then I would contact your tool providers. Unicode encoded fields were introduced to DB2 for i5/OS back in V3 days, so any actively marketed tool should support them by now. Within the IBM tools there are some that do not play well with Unicode -- OPM RPG, DFU, etc. If the tools you use feel "comfortable" providing an OPM/DFU level of support then I would start looking for some new tools :-)

4. You should find the conversion from CCSID 37 data to CCSID 13488 data to be straightforward (see #2 above). Translation doesn't come into this. Translation would be the false expectation (that some have) that Unicode allows you to convert in CCSID 37 encoded English text ('Yes') to Unicode and then have CCSID 880 encoded Cyrillic text ('Äà') or CCSID 273 encoded German text ('Ja') come out. Conversion is having CCSID 880 encoded English text ('Yes') and CCSID 273 encoded English text ('Yes') come out correctly -- the data has been converted (to a different EBCDIC, ASCII, or Unicode encoding), but not translated.

Hope this helps,
Bruce Vining
http://www.brucevining.com/

Dave Murvin <davem@xxxxxxxx> wrote:
Hello,

I have a situation at a client shop where they are converting to a
new system that uses double byte graphic fields (CCSID 13488 - UCS-2)
for all the character fields. They would like to be able to join
queries to the old data base which uses regular character fields
(CCSID 37 - English). For now, the CCSID 13488 data base runs on
V5R4 and the CCSID 37 database runs on V5R2. After conversion, the
V5R2 database will be copied to the V5R4 machine for historical purposes.

Working with the UCS-2 fields is kind of a pain. The tools we are
using, such as wrkdbf, Linoma, and Sequel, do not work with the UCS-2
fields. For example, you can not run an sequel query over the CCSID
37 data base and insert the results into the CCSID13488 data
base. The fields will not map. Likewise, you can not create a join
logical over two files where the join fields are different CCSIDs.

We have built a logical over one of the CCSID 13488 based data files
and redefined all the non-numeric fields as character CCSID
37. Using this method, we can inquire and change the fields in the
CCSID 13488 file through our new logical using Linoma. wrkdbf does
not seem to be able to handle this method either (it can view the
data this way, but it does not let us change it because it still sees
the base UCS-2 field type and we would need to use CCSID 13488 Hex to
change the field).

>From documentation, it looks like you should be able to use CPYF
*Map to transfer data, and I have done in internally in RPGLE
programs by defining fields like the CCSID 13488 fields then moving
the character data to them. It seems like it will be a royal pain if
these are the only tools we have. We do not want to create bunches
of logicals just so the users can sequel to get their information if
we don't have to. It is also a pain for oneze, twoze record changes
for testing.

I would have thought the CCSIDs would convert automatically. Is the
problem conversion versus translation that Bruce mentioned in a post
in the archives?

Any suggestions out there for better methods or tools that play well
with this situation?

Dave Murvin
DRM Enterprises, Inc.

Re: Suggestions for working with mixed CCSIDs (37 and 13488 - UCS-2)