Re: Encryption Problem. -- MIDRANGE-L

Hello, Eduard:
**
IBM i, i5/OS and OS/400 are EBCDIC systems, like the IBM mainframes (z/OS, z/VM, z/VSE...). Almost all other platforms are ASCII-based.

This means you need to be aware of this and take appropriate steps.

I did not see in your code snippets where you translate the "key" from EBCDIC to ASCII, before you use it with the decryption algorithm.

Remember, when you encrypted the data on an ASCII system, the key was in ASCII, as was the data. So, when you do the corresponding decrypt on IBM i, unless you take appropriate steps to control this, the key will most likely end up as EBCDIC.

And, of course, an EBCDIC key is not the same value as the same key in ASCII, and so, you will never get the correct results.

This probably also explains why "it worked" when you ran both the encrypt and recrypt on the IBM i ...

I hope this helps. Let us know how it goes. ;-)

All the best,

Mark S. Waterbury

> On 12/7/2010 7:15 PM, Eduard Sluis wrote:

Hi all,

We have problems to get a correct result when Decrypting on the IBM i with RC4.
What is the case?

We are communicating with an external device (not an IBM i).
That device Encrypt with RC4.
The results is made Base64 to get a string that can be communicated.
That string is send to the IBM i.
We decode on the IBM i and get the same Hex code as the device had before the
Base 64 step.
We decrypt the RC4 and make the result EBCDIC on the IBM i but we do not get the
original input back.

So the steps we take are:
On the device:
Have the Clear text.
Encrypt with RC4.
Encode with Base64.
Send to the IBM i.
On the IBM i:
Decode the Base64.
Decrypt the RC4.
Translate to EBCDIC.

Unluckily after the decryption on the IBM i we do not get the correct result.
We checked and tried a lot of different things but we could not come to a
resolution.
We also tried the two different decryption functions for RC4 we have found.-
Qc3DecryptData
- Cipher( p_ENCOPN:RC4_Controls:p_ENCCLS);To our surprise they gave a different
results that both are not the original input.
Sure we are doing something wrong but we cannot figure out what and resolve it
as we normally can and do.

I hope some of you has more experience and can point out what we are doing
wrong.
The code snippets of the Decryption we have tried are:

key_ctx = *ALLx'00'; %subst(key_ctx.stream:1:%len(key)) = key; key_ctx.len = %len(key); RC4_Controls = *ALLx'00'; RC4_Controls.funct_id = x'0013'; RC4_Controls.datalen = %size(data); RC4_Controls.operation = x'00'; // 0=Encrypt,1=Decrypt RC4_Controls.p_key_ctx = %addr(key_ctx); p_recv = %addr(encrypted); p_src = %addr(data); cipher( p_recv: RC4_Controls: p_src);
And:

Qc3DecryptData(encdata:nRTNlen:MyAlgo:'ALGD0300':myKey:'KEYD0200':
ANY_CRYPTO_SRV:CRYPYO_SRV:szData:%size(szDate):nRtnLen:apiError);

If we follow the sequence of the steps we have:
- Clear text on the device: - Hex Input text:
This iS a Test!
54 68 69 73 20 69 53 20 61 20 54 65 73 74 21- Clear text encrypted with key:
12345678901234567890

- Hex Encrypted text:9E 44 9C 72 78 33 41 61 A5 A9 4D 0C 7B C5 FADrx3Aa¥©M{Åú
- Base 64 Encoded:
nkSccngzQWGlqU0Me8X6
- Hex Base 64 Encoded:95 92 E2 83 83 95 87 A9 D8 E6 C7 93 98 E4 F0 D4 85 F8 E7
F6
- Base 64 Encoded received on IBM i:
nkSccngzQWGlqU0Me8X6
- Hex as recieved on IBM i (see is same as sent):95 92 E2 83 83 95 87 A9 D8 E6
C7 93 98 E4 F0 D4 85 F8 E7 F6
- Result of Base 64 decoding
ÆàæÊÌ /vz(#E³
- Hex of Base 64 Decoding (see is same as Hex input to Base 64 Encoding on
device)!9E 44 9C 72 78 33 41 61 A5 A9 4D 0C 7B C5 FA
- Clear text result of Xßü;QX¹Õ#X is hex:
E7 59 DC 5E 2D D8 2E 2D E7 DA EF FF 7B E7 24- Clear text result
of Cypher:Qc3DecryptData:
2{k6âÈü¥n^äx is hex:
F2 0A C0 92 F6 42 74 DC 29 B2 95 B0 2F 43 A7
Both differ from each other!! and both differ dearly from the the expected:
9E 44 9C 72 78 33 41 61 A5 A9 4D 0C 7B C5 FA
Clearly we have not tried the way back yet....
When we do all steps on the IBM i we do get the correct results (with both
methods but not intermingled).
We would like to assume that RC4 is RC4 and equal input would create equal
output.
Even with the two IBM i methods that seems not the case.....
If we use different tools on the Internet to Encrypt (with one tool) and Decrypt
(with another tool) we get the same results.
We do not believe that the problem is created on the Device.
Even if that would be the case we would need to get the same wrong result with
the two decryption methods. Who will be able to help us??
Kind regards,
Eduard Sluis.