× The internal search function is temporarily non-functional. The current search engine is no longer viable and we are researching alternatives.
As a stop gap measure, we are using Google's custom search engine service.
If you know of an easy to use, open source, search engine ... please contact support@midrange.com.



David FOXWELL wrote:

Our system is set to CCSID 65535. A programmer wrote some code containing the <!!> characters to replace <concat>. [snip] The program actually running the SQL instructions saw something other than <!!>.

David:

I _think_ that your fundamental problem is buried in those words.

AFAIK, the earlier programmer placed '||' (vertical bars; "pipes"; x'7C7C'?)into the SQL statement and those are what are in the SQL statement now. I _think_ that a hex dump of the physical bytes should show the correct hex values. If '!!' (exclamation points or 'marks'; x'2121'?) had actually been in the physical statements, they would have been rejected as syntax errors. However...

When the bits that make up those two bytes get converted into visible characters for you to look at, either on a display or printer, the bits get converted according to whatever conversion tables are in effect for the display or print operation. The earlier programmer could have had the same setup as you have; he would indeed have typed the exclamation points even though they would be stored as x'7C7C'. The same inappropriate conversion could have happened back then.

With a system QCCSID of 65535, conversions are allowed to look like almost anything. A display device that has a particular keyboard/language setting being used by a user profile with particular language identifier, country/region identifier, CCSID, etc., might show different character "shapes" for a given sequence of (65535=no conversion) bits than what would be _seen_ on a display device sitting next to it. Exact same bits, but different "shapes" drawn on the two screens.

One user in a given set of circumstances gets x'7C' drawn as a vertical bar; the next user sees the same bits drawn as an exclamation point.

It's not particularly different from using FTP to do a BINARY transfer of an ASCII text file and then trying to view it under an EBCDIC CCSID. The BINARY transfer avoided doing the appropriate translation/conversion at the bit level when the bits were written to storage.

The hex values are the same. But when viewed through an inappropriate character set/code page conversion to visual representation, the characters become... garbage.

You could probably see the characters as vertical bars if you adjusted your display session appropriately. Maybe it's in the *DEVD that you're using; maybe it's in the conversions defined in your emulator settings; maybe it's even in the country/region or language settings on your PC. There are a lot of possible points depending on what all the bits travel through.

If humans would read everything as hex or binary values, there might not be such a thing as 'variant' and 'invariant' characters. But because we require our machines to draw bits as visible characters with different shapes in different languages, just for our convenience, we have to make sure that we tell it the right ways to do the drawings (or light up pixels or deposit ink droplets).

We do that by ensuring that CCSID tags go along with the collections of bits. The machines can then manipulate bits at all input and output points, as well as during all collating/sorting/comparison points, wherever those bits get processed.

Having a major intermediate process that skips conversion (QCCSID 65535) is trouble. As soon as a device gets attached that expects a different interpretation of bit patterns, things get ugly.

Now that I've confused everything beyond hope, I'll go back to work... I'm sure that some discussion will bring clarity eventually.

Tom Liotta


As an Amazon Associate we earn from qualifying purchases.

This thread ...


Follow On AppleNews
Return to Archive home page | Return to MIDRANGE.COM home page

This mailing list archive is Copyright 1997-2024 by midrange.com and David Gibbs as a compilation work. Use of the archive is restricted to research of a business or technical nature. Any other uses are prohibited. Full details are available on our policy page. If you have questions about this, please contact [javascript protected email address].

Operating expenses for this site are earned using the Amazon Associate program and Google Adsense.