RE: Possible dumb question: ASCII, EBCDIC, or Unicode?

Fred --

I stand corrected.  I actually ran my examples on OS/390 which behaves the way 
I described.  The output on OS/390 shows:

Ascii String: çÁ%%?
Ebcdic String: Hello

Apparently, the JVM writers for OS/390 have decided to use this "really 
non-portable way to deal with Java".

The source of the difference appears to be in the value chosen by the JVM for 
the "file.encoding" system property.  On iSeries, a job CCSID of 37 (US EBCDIC) 
maps to a file.encoding value of ISO8859_1 which is ASCII.  This means that 
when byte arrays are converted to Strings, the byte arrays are assumed to be 
encoded in ASCII.

While this works fine for sockets, this causes problems when using the java.io 
Stream classes on the iSeries since it is assumed that data in byte arrays is 
ASCII while most iSeries files are actually stored in EBCDIC.  For example, the 
following does not work on the iSeries although it does work on Windows and on 
zSeries:

  FileInputStream fis = new 
FileInputStream("/QSYS.LIB/SOMELIB.LIB/SOMEFILE.FILE/SOMEMBR.MBR");
  byte[] ba = new byte[35];
  int read = fis.read(ba);
  String reads = new String(ba);
  System.out.println(reads);

So, by choosing ASCII as the platform default encoding, the iSeries JVM writers 
have made it easy to interface with sockets, for example, but more difficult to 
use native iSeries files with the java.io package.  This same problem occurs 
with Readers and Writers which assume an ASCII default encoding.  Hopefully, 
most people are using either the JDBC driver or the Toolbox classes to deal 
with iSeries native files.

In contrast, the zSeries default "file.encoding" value is "Cp037" which is US 
EBCDIC.

So, David, I am now puzzled as to the source of your original problem unless 
you're overriding the file.encoding system property somewhere.

Gary

> -----Original Message-----
> From: java400-l-admin@midrange.com
> [mailto:java400-l-admin@midrange.com] On Behalf Of Fred Kulack
> Sent: Tuesday, September 03, 2002 8:43 AM
> To: java400-l@midrange.com
> Cc: java400-l-admin@midrange.com
> Subject: RE: Possible dumb question: ASCII, EBCDIC, or Unicode?
>
>
>
> On 09/02/2002 at 12:39:05 PM, java400-l-admin@midrange.com
> wrote: Remember the text is received into a byte array.  If
> that happens to be in ASCII, which is probably is, and the
> program says to convert that to Unicode but interpret the
> bytes as if they're EBCDIC, then the conversion will not be
> properly performed.  The statement
>
>   String myString = new String(socketByteArray)
>
> will assume that the byte array contains ASCII on a
> non-iSeries machine but EBCDIC on an iSeries machine.
> --- end of excerpt ---
>
> No, that's not true.
> That would be a really non-portable way to deal with Java on
> an iSeries machine. Java usually interfaces to the native
> machine in unicode or ASCII just like it does on other
> systems. I.e. if you write an output file or output socket
> stream in Java, that data by default should be in ASCII.
>
> Note that the problem then typically comes in when native
> iSeries programs send binary data (EBCDIC) to a Java opened
> streams (which expect them to be ASCII). There are some
> tricks that occur in some circumstances (which I don't
> exactly recall), in which the data encoding is changed upon
> read/write (i.e. think standard I/O)
>
>
> Run this example:
> public class ByteEncoding {
>    public static void main(String args[]) {
>       byte     asciiHello[] = new byte[]{ (byte)0x48, (byte)0x65,
> (byte)0x6c, (byte)0x6c, (byte)0x6f };
>       byte     ebcdicHello[] = new byte[]{ (byte)0xC8, (byte)0x85,
> (byte)0x93, (byte)0x93, (byte)0x96 }; // CCSID 37
>
>       String   sAscii = new String(asciiHello);
>       String   sEbcdic = new String(ebcdicHello);
>       System.out.print("Ascii String: ");
>       System.out.println(sAscii);
>       System.out.print("Ebcdic String: ");
>       System.out.println(sEbcdic);
>    }
> }
>
>
> You'll see this output on your Windows machine:
> Ascii String: Hello
> Ebcdic String: ╚àôôû
>
> You'll see this output on your iSeries machine:
>  Ascii String: Hello
>  Ebcdic String: È
>
> Note that the ASCII array was 'correctly' converted to a
> string, while the EBCDIC one was not.
>
>
>
> "The stuff we call "software" is not like anything that human society
>   is used to thinking about. Software is something like a machine, and
>   something like mathematics, and something like language, and
>   something like thought, and art, and information...
>   but software is not in fact any of those other things."
> Bruce Sterling - The Hacker Crackdown
>
> Fred A. Kulack - IBM eServer iSeries - Enterprise Application
> Solutions ERP, Java DB2 access, Jdbc, JTA, etc...
> IBM in Rochester, MN  (Phone: 507.253.5982   T/L 553-5982)
> mailto:kulack@us.ibm.com   Personal: mailto:kulack@magnaspeed.net
> AIM Home:FKulack  AIM Work:FKulackWrk
> MSN Work: fakulack@hotmail.com
> _______________________________________________
> This is the Java Programming on and around the iSeries /
> AS400 (JAVA400-L) mailing list To post a message email:
> JAVA400-L@midrange.com To subscribe, unsubscribe, or change
> list options,
> visit: http://lists.midrange.com/cgi-bin/listinfo/java400-l
> or email: JAVA400-L-request@midrange.com
> Before posting, please take a moment to review the archives
> at http://archive.midrange.com/java400-l.
>