× The internal search function is temporarily non-functional. The current search engine is no longer viable and we are researching alternatives.
As a stop gap measure, we are using Google's custom search engine service.
If you know of an easy to use, open source, search engine ... please contact support@midrange.com.



Hi Vern,

As so often Scott is fast like the wind and has already fixed CSVR4.

I think that should help Greg as much as possible. CSVDEMO4 explains well how to do it. One could even filter invalid Unicode characters by a range of hexadecimal values in order to remove or replace them.

Regards,

Thomas.

Am 06.05.2020 um 00:24 schrieb Vernon Hamberg:
You're right, Thomas - I found SQL's XMLTABLE (and probably the same for the equivalent JSON UDTF) to yield the best (most reliable?) results - trying to replace a non-printable UTF-8 character would be nice, maybe, and I hope my colleague has some luck with that. As I see, it you'd need a table of the UTF-8 glyphs that are not in your version of EBCDIC, then provide an alternative. I hope that is not a fool's errand!

One alternative is to put everything in your tables into UTF-8 or UTF-16.

Vern

On 5/5/2020 1:15 PM, Tools/400 wrote:
I do not think that iconv() could help, because there are no counterparts in EBCDIC for a large number of characters that are available in UTF-8.

So all these missing characters would end up in x'3F', right?

Thomas.

Am 05.05.2020 um 19:58 schrieb Vernon Hamberg:
I think probably, Sam - I suspect that the i is using that in WRKLNK, option 5 (also known as DSPF). And probably equally the SQL engine does.

Unless we can specify that the XML-SAX and XML-INTO opcodes can have the CCSID set to 1208 - you CAN have it set to UCS2, but that's not enough to prevent errors when the UTF-8 character is not in the UCS2 character set - emojis certainly are in this category, things like ellipsis and em dash get set to that X'3F' again.

But it's too late to use iconv, I am guessing, when you use those opcodes.

Maybe for consuming XML (not using SQL's capabilities), the DATA-INTO can be used and then something that uses iconv?

A very challenging task, we have, don't we?

Vern

On 5/5/2020 12:38 PM, Sam_L wrote:
I admit my lack of experience or knowledge on this, but would the iconv()--Code Conversion API be of any use?

Sam


On 5/5/2020 12:10 PM, Vernon Hamberg wrote:
Oh yeah - and I have a presentation on this problem in the iNSIGHT virtual COMMON conference that started today - I believe everything will be available for 60 days, one can still register for it. Have to be a COMMON member, probably.

Vern

On 5/5/2020 9:17 AM, Greg Wilburn wrote:
So I changed the attributes of the file from 819 to 1208.  It displays better in WRKLNK.  The Right Single Quote is displayed as a blank instead of the other characters.

I'm wondering if this will help (at least some).






As an Amazon Associate we earn from qualifying purchases.

This thread ...

Follow-Ups:
Replies:

Follow On AppleNews
Return to Archive home page | Return to MIDRANGE.COM home page

This mailing list archive is Copyright 1997-2024 by midrange.com and David Gibbs as a compilation work. Use of the archive is restricted to research of a business or technical nature. Any other uses are prohibited. Full details are available on our policy page. If you have questions about this, please contact [javascript protected email address].

Operating expenses for this site are earned using the Amazon Associate program and Google Adsense.