Re: The complete CCSID Q&A -- MIDRANGE-L

Paul,

What is the CCSID of your source file containing the DDS constants? I
suspect (but can't say absolutely for sure) that you're falling into the
following point:

------------------------
When a display file object is created, it is tagged with the coded
character set identifier (CCSID) of the source file.

At compile time:

* All character data is read from the primary source file without any
character conversion being performed.

* User message text (identified by the MSGCON keyword in DDS) remains the
same because it is assumed to be in the same CCSID as the primary source
file.

At run time, the constant data is converted based on the CHRID parameter
value used to create the display file object. This conversion is optional
and can occur only when the CHRID is set to *JOBCCSID or indirectly with
CHRIDCTL. This conversion is from the display file CCSID to the character
identifier (CHRID) of the device. The field-level keyword NOCCSID (no coded
character set identifier) allows the user to specify fields within the DDS
that are never to be converted.

Note
To use data management support of CCSIDs, you must change source physical
files tagged with CCSID 65535 to a CCSID value that is associated with the
data.

------------------------

So having *JOBCCSID doesn't cause the constant to be converted to the job
CCSID, but does tell workstation data management to convert the constant
(from the SRCF CCSID) to the CHRID/CCSID of the terminal.

If starting from scratch and working in a multilingual/multi-CCSID
environment, you would want to correctly tag all resources (database,
source files, jobs, etc) *and* use *JOBCCSID. Strictly speaking this, using
*JOBCCSID, would not be a concern if working in a single CCSID environment
but as one never knows what paths the company might take in the future in
terms of growth/change it would be prudent. In any case you would want
correct tagging of database just so ASCII and Unicode conversions to other
environments works as expected.

Bruce

On Tue, Jan 22, 2019 at 9:01 AM Paul Nicolay <paul.nicolay@xxxxxxxxxx>
wrote:

Bruce,

My test PF which is tagged as 1148 has the square brackets as x4A and x5A
which is correct.

In my initial tests I only changed the QCCSID value from 65535 to 1148 and
found out that as long as the emulator codepage and job CCSID were set
identical that the data showed up fine. This means that the system
translates the data correctly between 1148 of the database and the terminal
(whether it be 1148 or 37).

So far so good... until I placed following test line in my DDS;

A 4 4'[]'

... to have the brackets as constants on my screen. Great was my suprise
that it only showed up fine with emulator codepage set at 1148 and job set
at 1148. The combination 37/37 which shows my database fields correctly
doesn't show the constants correctly !?

This resulted me digging deeper into other settings like the CHRIDCTL and
*JOBCCSID. Once I used these everything (both database fields and the
constant) showed up correctly, no matter which combination of 1148/37 I
used on the emulator codepage and job CCSID !?

So my question is actually... should I change the CHRIDCTL and
*JOBCCSID... would this be the recommended setting if I could start from
scratch (or do I introduce other issues) ? It amazes me that IBM writes a
621 page manual but doesn't tell me this, it even hardly touches these
settings !?

Kind regards,
Paul
________________________________________
From: MIDRANGE-L <midrange-l-bounces@xxxxxxxxxxxxxxxxxx> on behalf of
Bruce Vining <bruce.vining@xxxxxxxxx>
Sent: Monday, January 21, 2019 19:15
To: Midrange Systems Technical Discussion
Subject: Re: The complete CCSID Q&A

Hoping I got this right as it's been so long and there are several
"gotchas" in workstation support...

In CCSID 1148 code point x'BB' is the vertical bar (|). In CCSID 37 code
point x'BB'is the closing (also known as the right) bracket (]).

If the database is 1148 then the intent of x'BB' is the vertical bar. If
the job CCSID is 1148 then the x'BB' will not be changed during I/O
operations involving database data management. If the job CCSID is 37 then
the x'BB' will be changed to x'4F' (the vertical bar in CCSID 37) by
database data management.

If the DSPF CHRID is *DEVD (and the DDS CHRID keyword is not specified in
the DDS) then no change to the code point values will occur. So if the job
CCSID and the device configuration agree (both 1148 or both 37) then the
display should be correct. If the job CCSID and the device configuration do
not agree (for instance 37 and 1148) then *some* code point values will
display incorrectly based on the CCSIDs involved. The code points of
interest for your case are:

37: 1148:
X'4A' ¢ (cent sign) [ (left bracket)
X'4F' | (vertical bar) ! (exclamation point)
X'5A' ! (exclamation point) ] (right bracket)
X'5F' ¬(logical not) ˆ (circumflex accent)
X'B0' ˆ(circumflex accent) ¢ (cent sign)
X'BA' [ (left bracket) ¬(logical not)
X'BB' ] (right bracket) | (vertical bar)

So, for instance, if the job CCSID is 37, the device is 1148, and the DSPF
is *DEVD with no CHRID DDS keyword usage, then the intended 1148 | will
display as the ! as we have converted from the database x'BB' to the job
x'4F' and then sent the x'4F' as-is to the device (logically | to ] to !).

If the DSPFD CHRID is *JOBCCSID then code point conversion will occur when
the job CCSID and device configuration do not agree. So the 1148 x'BB' will
convert to the 37 x'4F' and workstation data management will then convert
the x'4F' back to x'BB'.

As you mention "constants" but I'm not sure how you might be defining them,
I'll also point you to this discussion
<
https://www.ibm.com/support/knowledgecenter/en/ssw_ibm_i_71/nls/rbagsccsiddispfilesup.htm

.

As it relates to CCSID defaults CCSID tagging didn't exist until (and I may
have this wrong) V2R1.1 and so the default (for compatibility reasons) was
65535. Job default CCSID then came along in V3R1 as most systems didn't
change the CCSID from 65535 to a more correct value and IBM really, really
wanted CCSID conversions to start taking place. For many, many systems
changing the CCSID/CHRID default values to a more meaningful value (like
*JOBCCSID) will have no impact. But it will absolutely mess up any systems
that were "internationalized" prior to the AS/400 (which I think was the
name back in V2 days) supporting CCSIDs -- and so the defaults are selected
to maintain compatibility.

On Mon, Jan 21, 2019 at 9:57 AM Paul Nicolay <paul.nicolay@xxxxxxxxxx>
wrote:

Rob,

Changing QCCSID is not my main concern anymore... I do understand the
implications of that (I guess).

The questions came from digging deeper into CCSID stuff... and got in
doubt about QCHRIDCTL and CHRID on CRTDSPF.

Playing with these as well as the emulator and job CCSID I get varying
results in what I see on my screen.

For example...
DSPF (CHRID *DEVD)
QCCSID 1148, Database 1148, Emulator 1148, Job 1148… data looks correct.
QCCSID 1148, Database 1148, Emulator 1148, Job 37… data looks NOT

correct.

QCCSID 1148, Database 1148, Emulator 37, Job 1148… data looks NOT

correct.

QCCSID 1148, Database 1148, Emulator 37, Job 37… data looks correct.

DSPF (CHRID *JOBCCSID)
QCCSID 1148, Database 1148, Emulator 1148, Job 1148… data looks correct.
QCCSID 1148, Database 1148, Emulator 1148, Job 37… data looks correct.
QCCSID 1148, Database 1148, Emulator 37, Job 37… data looks correct.
QCCSID 1148, Database 1148, Emulator 37, Job 1148… data looks correct.

With the first set of test, constants on the screen only show up

correctly

with emulator and job set at 1148 (using SDA they however look correct in
other combinations as well)... in the second set of tests constants show

up

correctly always !

So why is *JOBCCSID not the default, is there any risk and why can't find
any recommendation on it ?

Kind regards,
Paul
________________________________________
From: MIDRANGE-L <midrange-l-bounces@xxxxxxxxxxxxxxxxxx> on behalf of

Rob

Berendt <rob@xxxxxxxxx>
Sent: Monday, January 21, 2019 16:32
To: Midrange Systems Technical Discussion
Subject: RE: The complete CCSID Q&A

Many of us have changed the system value QCCSID during the middle of the
day with no ill repercussions.
Chuck Pence once published a concern with that but the list's archive
searches are just not what they used to be and I cannot find that
discussion.

Search this for QCCSID

https://www.ibm.com/developerworks/community/wikis/home?lang=en-gb#!/wiki/IBM%20i%20Technology%20Updates/page/CGI%20frequently%20asked%20questions

http://www-01.ibm.com/support/docview.wss?uid=nas8N1021732

https://www.ibm.com/support/knowledgecenter/ssw_ibm_i_73/ddp/rbal1apendixqccsid.htm

<snip>
The default CCSID value in a user profile is *SYSVAL. This references the
QCCSID system value. You can change the QCCSID system value that is used

by

all user profiles with the Change System Value (CHGSYSVAL) command. If

you

do this, you would want to select a CCSID that represents most (if not

all)

of the users on your system
</snip>

https://www.ibm.com/support/knowledgecenter/ssw_ibm_i_73/nls/rbagsccsidgenrecom.htm

-----Original Message-----
From: MIDRANGE-L <midrange-l-bounces@xxxxxxxxxxxxxxxxxx> On Behalf Of
Paul Nicolay
Sent: Monday, January 21, 2019 10:04 AM
To: Midrange Systems Technical Discussion <midrange-l@xxxxxxxxxxxxxxxxxx

Subject: The complete CCSID Q&A

Hi,

While the title maybe a bit misleading, I hope the final result will be
more or less like that.

The facts, currently I see various systems at customer locations that
still have QCCSID set to 65535 of which I know it is a bad thing so I'm
trying to find out what the implication of changing this to a normal

value

is. In my investigation I found already a few things;

The database currently does have a (correct or at least a non-65535)

CCSID

so having QCCSID set to 65535 or not doesn't seem to make any difference

as

far as FTP and ODBC is concerned. This means the main issue is how

screens

handle the data (which indirectly impacts FTP/ODBC however by themselves
they aren't the issue).

Setting the emulator codepage and/or job CCSID does have an impact on

what

I see, but this is again is impacted by whether the DSPF is compiled with
CHRID(*DEVD or *JOBCCSID). While the last one is not the default, it
however provides the best results, not only for database fields, but as
well for constants on the screen (all my tests are based on just both the
square brackets, ie. [] ) So the question is, should I change the
compile option on CRTDSPF ?

This brings me to the next system value... QCHRIDCTL which is also set by
default to *DEVD ? Would I need to change this one to *JOBCCSID (and
change my CRTDSPF to CHRID(*CHRIDCTL) ?), or can this cause other issues.
I already read some conflicting things about *JOBCCSID... some IBM
documents say that panel-groups don't support it, others explicitely
mention panel-groups ?

Whats facinates me is that when using CHRID(*JOBCCSID) on the display

file

it doesn't matter how my emulator is set, nor how the job is set... so I
tested with 37 and 1148 but all four combinations show the data correctly
for the database and the constants ? Using CHRID(*DEVD) there's only one
combination, being emulator AND job set at 1148 to have everything

correct ?

For people that use the default of *DEVD, I wonder how you make sure that
every emulator in the company is set correctly as the smallest mistake
results in corrupting your data. Would you check if the device codepage

is

different from the QCCSID value... and terminate the logon ?

Digging deeper and deeper in all this makes me only more confused... the
question remains, what is the ideal setting of all this and why (or why
shouldn't I use certain settings) ?

Thanks in advance,

Paul

PS. The 621 pages IBM i Globalization manual didn't help me !

--
This is the Midrange Systems Technical Discussion (MIDRANGE-L) mailing
list To post a message email: MIDRANGE-L@xxxxxxxxxxxxxxxxxx To

subscribe,

unsubscribe, or change list options,
visit: https://lists.midrange.com/mailman/listinfo/midrange-l
or email: MIDRANGE-L-request@xxxxxxxxxxxxxxxxxx
Before posting, please take a moment to review the archives at
https://archive.midrange.com/midrange-l.

Please contact support@xxxxxxxxxxxx for any subscription related
questions.

Help support midrange.com by shopping at amazon.com with our affiliate
link: https://amazon.midrange.com
--
This is the Midrange Systems Technical Discussion (MIDRANGE-L) mailing

list

To post a message email: MIDRANGE-L@xxxxxxxxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: https://lists.midrange.com/mailman/listinfo/midrange-l
or email: MIDRANGE-L-request@xxxxxxxxxxxxxxxxxx
Before posting, please take a moment to review the archives
at https://archive.midrange.com/midrange-l.

Please contact support@xxxxxxxxxxxx for any subscription related
questions.

Help support midrange.com by shopping at amazon.com with our affiliate
link: https://amazon.midrange.com

--
This is the Midrange Systems Technical Discussion (MIDRANGE-L) mailing

list

To post a message email: MIDRANGE-L@xxxxxxxxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: https://lists.midrange.com/mailman/listinfo/midrange-l
or email: MIDRANGE-L-request@xxxxxxxxxxxxxxxxxx
Before posting, please take a moment to review the archives
at https://archive.midrange.com/midrange-l.

Please contact support@xxxxxxxxxxxx for any subscription related
questions.

Help support midrange.com by shopping at amazon.com with our affiliate
link: https://amazon.midrange.com

--
Thanks and Regards,
Bruce
931-505-1915
--
This is the Midrange Systems Technical Discussion (MIDRANGE-L) mailing list
To post a message email: MIDRANGE-L@xxxxxxxxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: https://lists.midrange.com/mailman/listinfo/midrange-l
or email: MIDRANGE-L-request@xxxxxxxxxxxxxxxxxx
Before posting, please take a moment to review the archives
at https://archive.midrange.com/midrange-l.

Please contact support@xxxxxxxxxxxx for any subscription related
questions.

Help support midrange.com by shopping at amazon.com with our affiliate
link: https://amazon.midrange.com
--
This is the Midrange Systems Technical Discussion (MIDRANGE-L) mailing list
To post a message email: MIDRANGE-L@xxxxxxxxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: https://lists.midrange.com/mailman/listinfo/midrange-l
or email: MIDRANGE-L-request@xxxxxxxxxxxxxxxxxx
Before posting, please take a moment to review the archives
at https://archive.midrange.com/midrange-l.

Please contact support@xxxxxxxxxxxx for any subscription related
questions.

Help support midrange.com by shopping at amazon.com with our affiliate
link: https://amazon.midrange.com