RE: Encryption Problem. -- MIDRANGE-L

Mark,

Also precaution needs to be done to make sure the CSSID you're using
converts as you're expecting, especially if you're using special characters
in the key...

DR2

===============

-----Original Message-----
From: midrange-l-bounces@xxxxxxxxxxxx
[mailto:midrange-l-bounces@xxxxxxxxxxxx] On Behalf Of Mark S. Waterbury
Sent: Tuesday, December 07, 2010 7:49 PM
To: Midrange Systems Technical Discussion
Subject: Re: Encryption Problem.

Hello, Eduard:
**
IBM i, i5/OS and OS/400 are EBCDIC systems, like the IBM mainframes
(z/OS, z/VM, z/VSE...). Almost all other platforms are ASCII-based.

This means you need to be aware of this and take appropriate steps.

I did not see in your code snippets where you translate the "key" from
EBCDIC to ASCII, before you use it with the decryption algorithm.

Remember, when you encrypted the data on an ASCII system, the key was in
ASCII, as was the data. So, when you do the corresponding decrypt on
IBM i, unless you take appropriate steps to control this, the key will
most likely end up as EBCDIC.

And, of course, an EBCDIC key is not the same value as the same key in
ASCII, and so, you will never get the correct results.

This probably also explains why "it worked" when you ran both the
encrypt and recrypt on the IBM i ...

I hope this helps. Let us know how it goes. ;-)

All the best,

Mark S. Waterbury

On 12/7/2010 7:15 PM, Eduard Sluis wrote:
Hi all,

We have problems to get a correct result when Decrypting on the IBM i with

RC4.

What is the case?

We are communicating with an external device (not an IBM i).
That device Encrypt with RC4.
The results is made Base64 to get a string that can be communicated.
That string is send to the IBM i.
We decode on the IBM i and get the same Hex code as the device had before

the

Base 64 step.
We decrypt the RC4 and make the result EBCDIC on the IBM i but we do not

get the

original input back.

So the steps we take are:
On the device:
Have the Clear text.
Encrypt with RC4.
Encode with Base64.
Send to the IBM i.
On the IBM i:
Decode the Base64.
Decrypt the RC4.
Translate to EBCDIC.

Unluckily after the decryption on the IBM i we do not get the correct

result.

We checked and tried a lot of different things but we could not come to a
resolution.
We also tried the two different decryption functions for RC4 we have

found.-

Qc3DecryptData
- Cipher( p_ENCOPN:RC4_Controls:p_ENCCLS);To our surprise they gave a

different

results that both are not the original input.
Sure we are doing something wrong but we cannot figure out what and

resolve it

as we normally can and do.

I hope some of you has more experience and can point out what we are doing
wrong.
The code snippets of the Decryption we have tried are:

key_ctx = *ALLx'00';
%subst(key_ctx.stream:1:%len(key)) = key;
key_ctx.len = %len(key);

RC4_Controls = *ALLx'00';
RC4_Controls.funct_id = x'0013';
RC4_Controls.datalen = %size(data);
RC4_Controls.operation = x'00'; // 0=Encrypt,1=Decrypt
RC4_Controls.p_key_ctx = %addr(key_ctx);

p_recv = %addr(encrypted);
p_src = %addr(data);

cipher( p_recv: RC4_Controls: p_src);

And:

Qc3DecryptData(encdata:nRTNlen:MyAlgo:'ALGD0300':myKey:'KEYD0200':

ANY_CRYPTO_SRV:CRYPYO_SRV:szData:%size(szDate):nRtnLen:apiError);

If we follow the sequence of the steps we have:
- Clear text on the device: - Hex Input text:
This iS a Test!
54 68 69 73 20 69 53 20 61 20 54 65 73 74 21- Clear text encrypted with

key:

12345678901234567890

- Hex Encrypted text:9E 44 9C 72 78 33 41 61 A5 A9 4D 0C 7B C5

FADrx3Aa¥©M{Åú

- Base 64 Encoded:
nkSccngzQWGlqU0Me8X6
- Hex Base 64 Encoded:95 92 E2 83 83 95 87 A9 D8 E6 C7 93 98 E4 F0 D4 85

F8 E7

F6

- Base 64 Encoded received on IBM i:
nkSccngzQWGlqU0Me8X6
- Hex as recieved on IBM i (see is same as sent):95 92 E2 83 83 95 87 A9

D8 E6

C7 93 98 E4 F0 D4 85 F8 E7 F6
- Result of Base 64 decoding
ÆàæÊÌ /vz(#E³
- Hex of Base 64 Decoding (see is same as Hex input to Base 64 Encoding on
device)!9E 44 9C 72 78 33 41 61 A5 A9 4D 0C 7B C5 FA
- Clear text result of Xßü;QX¹Õ#X is hex:
E7 59 DC 5E 2D D8 2E 2D E7 DA EF FF 7B E7 24- Clear text result
of Cypher:Qc3DecryptData:
2{k6âÈü¥n^äx is hex:
F2 0A C0 92 F6 42 74 DC 29 B2 95 B0 2F 43 A7

Both differ from each other!! and both differ dearly from the the

expected:

9E 44 9C 72 78 33 41 61 A5 A9 4D 0C 7B C5 FA

Clearly we have not tried the way back yet....
When we do all steps on the IBM i we do get the correct results (with both
methods but not intermingled).
We would like to assume that RC4 is RC4 and equal input would create equal
output.
Even with the two IBM i methods that seems not the case.....

If we use different tools on the Internet to Encrypt (with one tool) and

Decrypt

(with another tool) we get the same results.
We do not believe that the problem is created on the Device.
Even if that would be the case we would need to get the same wrong result

with

the two decryption methods.

Who will be able to help us??

Kind regards,
Eduard Sluis.