Re: Using iconv to convert data FROM file READ from IFS

Scott

Thanks; I will not make excuses for my ignorance; but I feel 'tis better to try 
new things than to carry on with the same old same old. I was writing this 
programme to replace a CL with a cpyfrmpcd followed by a dodgy rpg programme 
(yuck!).

By the way, I have made an amendment to your ifsio_h copy source member on our 
system to add the ccsid:

     D  st_codepag                    5U 0 
MS#1 D  st_ccsid                      5U 0 
     D**st_resv11****MS#1------------62A   
MS#1 D  st_resv11                    60A   

We over here in Europe have lots of fun (NOT!) with ccsids, so this is of use 
to us.

Thanks again, and to Barbara.

And I do understand all your and Barbara's comments about iconv; just itching 
to use it in anger now!

Michael

> Message Received: May 18 2006, 08:08 AM
> From: "Scott Klement" 
> To: "RPG programming on the AS400 / iSeries" 
> Cc: 
> Subject: Re: Using iconv to convert data FROM file READ from IFS
> 
> 
> 
> > I read the file onto a pointer.
> 
> I think I can safely assume that you're reading teh file into the memory 
> that the pointer points to... Obviously, a pointer can't store file 
> data, it can only store an address.
> 
> 
> > I open iconv with a to of 0 for the job and a from of the file's CCSID
> > Does not return an error.
> 
> The IFS APIs will do this automatically if you pass the O_TEXTDATA flag. 
> You don't need to use iconv()...
> 
> > I iconv the data on the pointer for the length of the data on the 
> > pointer back onto itself. Again this does not return an error, but the 
> > pointer is cleared and the length is set to zero. What am I doing wrong?
> 
> It sounds to me like the only problem here is that you don't understand 
> how the iconv() API works. What you describe could very well mean that 
> iconv() was successful! I'll explain...
> 
> When you call iconv(), you do something like this:
> 
> iconv(cd: PtrToInput: InputLeft: PtrToOutput: OutputLeft)
> 
> As iconv() converts each character, it changes PtrToInput to point to the 
> next character that needs to be convered, and reduces the number of bytes 
> left to convert.
> 
> Likewise, it changes PtrToOutput to point to the space where the next 
> output character will be placed, and decreases OutputLeft to reflect the 
> amount of space left in the buffer.
> 
> That way, if an untranslatable character should be found, the pointers 
> will be pointing to that character and where it goes in the output buffer. 
> Your program could add logic to do something special with that character. 
> Or perhaps log an error message showing the unconvertable character. It 
> could then add 1 to the input pointer, subtract one from the bytes left, 
> and re-call iconv() and it'd pick up where it left off.
> 
> When iconv() is done, the PtrToInput and PtrToOutput will be pointing to 
> the next charater to translate, or to undefined locations if all 
> translation has been done. You should not try to use them again.
> 
> And if all went well, InputLeft will be 0, since the whole input string 
> has been translated. OutputLeft will be the amount of unused space in the 
> output buffer, or if the whole thing was used, it'll also be 0.
> 
> So when you call iconv(), you should make sure of the following:
> 
> a) Don't pass the actual pointer to the start of your data as the 
> PtrToInput parameter. Instead, make a new pointer, and set it to the same 
> address as the start of your data. That way, when iconv() changes the 
> pointer, it won't affect your "start of buffer" pointer.
> 
> Same with the output buffer.
> 
> 
> b) Same thing with the length... have a separate length var for "bytes 
> left to translate". Before calling iconv(), set it to the length of the 
> data, when it's done, subtract the "bytes left" field from the original 
> length to find out how much was translated. Subtract the OutputLeft field 
> from the original output length to find out the length of the data that 
> iconv() wrote out.
> 
> When converting from single-byte character sets to double-byte, the input 
> and output lengths won't match. Also, make sure you use separate input 
> and output buffers for those, or you'll end up with a corrupted mess.
> 
> This also applies to unicode character sets, including (but not limited 
> to) UTF-8.
> 
> > I could use QDCXLATE but that would be admitting defeat.
> 
> Not to mention, it would lead to a solution that's technically incorrect. 
> You might be able to get it to work with a particular from/to CCSID, but 
> it wouldn't work universally with any file.
> 
> Once you understand iconv(), there's no reason why your solution shouldn't 
> work -- however, I'm a bit puzzled by the whole thing, since the open(), 
> read(), and write() APIs do all of this translation for you, so there's 
> absolutely no need to code iconv() or QDCXLATE() unless you're doing 
> something that's a bit more custom than what the IFS APIs do. Just pass 
> O_TEXTDATA, and you don't need all of this other stuff.
> 
> -- 
> This is the RPG programming on the AS400 / iSeries (RPG400-L) mailing list
> To post a message email: RPG400-L@xxxxxxxxxxxx
> To subscribe, unsubscribe, or change list options,
> visit: http://lists.midrange.com/mailman/listinfo/rpg400-l
> or email: RPG400-L-request@xxxxxxxxxxxx
> Before posting, please take a moment to review the archives
> at http://archive.midrange.com/rpg400-l.
> 
> 
>