Re: using RDP to edit UTF-8 documents on the IFS

Follow up.

I changed the "default transfer type for .xml docs" from text to binary.
Now the file has been translated correctly - on the IFS - to UTF-8.

Now, instead of more "live parsing" which is really neat but not very
helpful if the cursor keeps hanging and hanging, it would be nice for IBM
to squash some bugs (like selecting all text sometimes which is in there
for more than 10 years already), or a nice feature (it's RD... for "i"
therefore you pay 800 bucks) it would read the CCSID setting on the IFS.
But that isn;t a nice feature to show off on some demo i suppose.


End of ranting. Sorry about that,.





On Wed, Jul 10, 2013 at 12:19 PM, whatt sson <whattssonn@xxxxxxxxx> wrote:

Buck,
Thanks for testing.

I create a file with EDTF, or from RDi.
I change the CCSID to 1208 with option 13.
Etc

I didn't see the "properties" window where you can change the encoding of
the file.
When i go to RSE and right-click on the file i get a properties window
only showing some attributes such as encoding but i can't change it.
However, when i choose File->Properties the i get the full window where i
ca change the encoding. But i seem to get this window not always.
Sometimes it's disabled, sometimes a just get the small properties window.
Pff

However, i'm trying all things but it keep saying to me "just use another
tool for this".
Howver, 800 bucks a year is a lot of money for something you can't even
use to edit xml docs.

The locally cached file is encoded ok in UTF-8, after i save the document.
For example ř is translated to C5 99.
But the file on the IFS is not, although it's CCSID is 1208.

This also happens when i create the file from RDP.
Another file with another name, no avail, still having the same problems.

Clearing the cache does not help.
I clear the cache regularly because i had some problems before that RDP
wouldn't load the - NEWER - remote source member but instead loaded the old
source in the cache.


So nothing helps, local file is encoded ok, but remote is not, no matter
what i do.
I can't even reproduce the situation from yesterday that i create a new
file (with EDTF file or whatever) and let it save explicitly to UTF-8.
Local file is ok but remote file is not.
The ř is translated to C5 99.
But the file on the IFS is not, although it's CCSID is 1208.
It's translated to a question mark (3F).
And the other characters (ö and ä which have a representation in Cp1252)
are translated to EF BF BD which is the UTF-8 replacement character.
So it seems after translating correctly to UTF-8 locally, when it uploads
the file it then translates to characters in the 437 codepage but with
UTF-8 representation. So if a character does not exist it becomes a
question mark or replecament character.
Complicate all this... yes.. but thats the situation.

But now comes a surprise. Whe i create a file on the IFS from RDP 0 but
WITHOUT .xml extension, then it does encode correctly in UTF-8 on the IFS.

So maybe there some config setting associated with .xml but all settings
specify UTF-8.

I give up.

I use another free xml tool to create them on the PC and then upload
binary to the IFS.
That will work.

But what a shitty product, RDP, really.




I did some more testing but it's really annoying.





On Tue, Jul 9, 2013 at 7:10 PM, Buck Calabro <kc2hiz@xxxxxxxxx> wrote:

Replies inline.

On 7/9/2013 11:49 AM, whatt sson wrote:

I'm trying to edit an xml document on the ifs using RDP (8.5) but it

seems

to have some trouble converting to/from UTF-8.

I have created the file on the IFS with CCSID 1208.

How? I used EDTF to create a file with a blank line, save it, exit
EDTF, re-run EDTF, F15 option 3 CCSID 1208, then paste <?xml
version="1.0" encoding="utf-8" ?> into the first line and saved it.
WRKLNK shows CCSID 1208.

Then i edit the file in RDP and put some text in it with latin-1
characters, such as ä.
Then i save the file but RDP converts to 1252 instead of 1208.

I opened my test.xml in RDP 8.5.1, changed to Source view and added
this: <test>Hélène says bonjour.</test> I am in the US, with a US
keyboard, so I typed the accented letters by holding down the Alt key
and pressing the numbers on the number pad: Alt+130 for é and Alt+138
for è. Then I saved the file. No warnings in RDP.

WRKLNK still shows CCSID 1208. EDTF shows a warning:
CPF9897 1 INVALID CHARCTERS FOR CCSID 01208 ENCOUNTERED IN DISPLAY
RECORD 3. IF THE RECORD IS CHANGED, THEY WILL BE REPLACED WITH
BLANKS.
Cause . . . . . : No additional online help information is available.
Exit EDTF.

Back to RDP, in RSE, right click the XML file -> Properties. I see that
file encoding is set to Default(Cp1252). I set it to Other UTF-8.
Apply, OK. I see funny characters in my RDP editor where my accents
used to be. I fix them and save.

WRKLNK shows 1208 and EDTF shows the accented characters properly. In
hex they are:
<test>
3C746573743E
Hélène
48C3A96CC3A86E65

When i put a latin-2 character in the text such as ř and then save to

text

RDP complains and asks to convert it to UTF-8 (which i want it to do).

When I paste that character into my RDP editor and save, I see no
warnings in RDP. WRKLNK shows 1208, EDTF shows a non-displayable
character on my US English 5250 (CCSID 37) session. In hex it is:

<test2>
3C74657374323E
ř
C599
</test2>
3C2F74657374323E

How can i set RDP to always save in UTF-8 format.
Or even better, to respect to CCSID attribute of the IFS file.

I did not see a preference for this in regular RDP 8.5.1. Maybe it's
because RDP is not the web developer version - I think that is Rational
Business Developer? Just a guess.

In the general settings i have set the default CCSID to 1208 for XML

files

(*.xml).
But this does not have any effect it seems.

General -> Appearance -> Content types. My XML type was set to UTF-8
(which is the default).

XML -> XML Files has what to do when /creating/ an XML file. Mine was
set to UTF-8 (which is the default).

RDP also behaves rather quirky with this.
When i explicitly choose to save to UTF-8 (when asked for it) it

converts

incorrectly to the existing file (CCSID 1208).
When i create a new file with CCSID 1208 and save the same text then it
does convert to UTF-8.

RDP (or RDi) seems to be completely broken on this.

I'm not sure what is different between our setups. In my case, I must
remember to change the properties to UTF-8. That's clumsy but I
wouldn't call it completely broken. I hope this was some help; if only
to show where RDP ends and RBD begins.
--buck


--
This is the Rational Developer for IBM i / Websphere Development Studio
Client for System i & iSeries (WDSCI-L) mailing list
To post a message email: WDSCI-L@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/wdsci-l
or email: WDSCI-L-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/wdsci-l.