Re: Allowing accented characters in the database

Hi Patrik


You ask about why mess with UTF-8 - I think one reason is its nearly ubiquitous use in json and xml. We have service reports sent in by our associates in the field, they are either JSON or XML coming from an iPhone application. Emojis such as you showed do not convert into CCSID 37 (we are in the US). And Apple, etc., are adding more all the time into UTF-8 space.


We went to using the XMLTABLE SQL table function instead of XML-INTO, because it substitutes X'3F' for unconvertable characters - no errors. We find that it also can flatten hierarchical structures with great flexibility.


At any rate, UTF-8 is something we have to mess with here.


Cheers
Vern

On Tue, 29 Aug, 2023 at 7:28 AM, Patrik Schindler <poc@xxxxxxxxxx> wrote:


To: midrange systems technical discussion

Hello Barbara,

Am 29.08.2023 um 01:57 schrieb Barbara Morris <bmorris@xxxxxxxxxx<mailto:bmorris@xxxxxxxxxx>>

1. If the job CCSID is 65535, database doesn't do the conversion, but RPG (by default) assumes that the conversion was done.

Which is a side effect of the default QCCSID system variable never being touched, if I understand correctly, yes?

2. The UTF-8 data used by the current job should all be able to be converted to the current job CCSID.

Example: a 👍 is never to be found in the 37 code page. Although I think it would be awesome beyond belief to actually see this in a 5250 session. :-D

Again, this is only an issue with UTF-8 data in database files. There is no similar issue with UCS-2 and UTF-16 data in database files.

Why should one mess with UTF-8 in the i database then? As long as automatic conversion works towards the end user UI, I see no reason to introduce workarounds to prohibit any conversion just for the sake of "UTF-8 in the database"? To me, this smells quite a bit like the ever-touched-QCCSID issue.

:wq! PoC