Re: SSL client connection error - SSL_Handshake(): Peer not recognized or badly formatted message received.

Paul,

This question would be better answered by Brad Stone as he uses the SSL APIs regularly. I prefer the GSKit ones, myself...

But, I assume that GSKit and SSL APIs work the same, here, just have a different syntax. (That's pretty much the norm with these...)

The stuff you set in the system values are the only protocols that are allowed, the system won't let the APIs override these settings. But, the APIs can further restrict the choices.

So, for example:

Sysval allows TLS 1.1 and 1.2
API specifies TLS 1.0, TLS 1.1 and TLS 1.2
Result: Only TLS 1.1 or 1.2 will work. If client doesn't support these, you get 'Peer not recognized'

Sysval allows SSL 2 and 3, TLS 1.0,1.1 and 1.2
API specifies TLS 1.2
Result: Only TLS 1.2 will work, if client doesn't support it, 'Peer not recognized'

Sysval allows TLS 1.0, 1.1 and 1.2
API specifies: SSL 3
Result: Nothing will work, always get 'Peer not recognized'


The error 'Peer not recognized' can occur for other reasons besides these protocol mismatches. For example, if you try to connect with an SSL client to a plaintext server (one that doesn't support SSL) then it will also produce this same error. This is something I do all the time by accident :-)

The error just means that the data it received doesn't fit any SSL/TLS settings that you have enabled. So if it gets a plaintext message, that would look like an 'unrecognized' SSL message. But also, if it's in a format it's not configured for, it also wouldn't recognize it.

-SK

On 6/26/2015 2:21 PM, Steinmetz, Paul wrote:

Scott,

We are using the SSL APIs.

Do these SSL APIs use the 3 system value settings, QSSLCSL, QSSLCSLCTL, QSSLPCL?
Or
Do they have their own settings?
Or
Are they hardcoded?

What needs to be done to fix this?

Paul



-----Original Message-----
From: MIDRANGE-L [mailto:midrange-l-bounces@xxxxxxxxxxxx] On Behalf Of Scott Klement
Sent: Friday, June 26, 2015 11:11 AM
To: Midrange Systems Technical Discussion
Subject: Re: SSL client connection error - SSL_Handshake(): Peer not recognized or badly formatted message received.

Since he references SSL_Handshake(), I'm assuming that he's using the SSL APIs.

But if someone does need info about how to configure the versions in GSKit, let me know, I can provide that...



On 6/26/2015 9:10 AM, Bradley Stone wrote:

Hi Paul.

What are you using to connect/communicate? Can you get the return code?

Do you know if the GSKit APIs are used or the standard SSL APIs are being
used for the connect?

I ran into an issue with a customer on V7R1 that was trying to use V7R1 and
up and the SSL APIs weren't really doing things right, so on the SSL
Handshake API we had to tell it by sending it the proper code and that
cleared things up.

Here's a link to an article I wrote about it.. it refers to GETURI but it
would also apply to any client application that uses the SSL APIs. (the
GSKit APIs may have a different setting).

http://www.fieldexit.com/forum/display?threadid=170

Brad
www.bvstools.com

On Fri, Jun 26, 2015 at 8:06 AM, Steinmetz, Paul <PSteinmetz@xxxxxxxxxx>
wrote:

I'm receiving this error when trying to connect to a remote server.

SSL_Handshake(): Peer not recognized or badly formatted message received.

V7R1, TR10, latest CUM 15142 and all groups

I've confirmed DCM has proper CA, both root and intermediate.
Remote server has TLS1.0 disabled, TLS1.2 is currently being used for
other connections to that server.
I'm thinking this is either a SSL protocol issue or cipher issue.
I know when the I is the server, the DCM application defaults need to be
changed to allow TLS1.2 , TLS1.1 and disable SSL 3.0, SSL2.0
Also cipher defaults need to be changed.

Are there similar settings for when the I is the client?
I've seen other posts with this error, but did not see the final
resolution.


- - - - - - - - - - - - - - - - - - - - - - - C O N N E C T I O N F E E
D B A C K -
About to connect() to XXXXXX-web.XXX.net port 443 (#0)
Trying XXX.XXX.XXX.X... connected
SSL_Handshake(): Peer not recognized or badly formatted message received.
Closing connection #0
SSL connect error
************End of Data********************


Thank You
_____
Paul Steinmetz
IBM i Systems Administrator

Pencor Services, Inc.
462 Delaware Ave
Palmerton Pa 18071

610-826-9117 work
610-826-9188 fax
610-349-0913 cell
610-377-6012 home

psteinmetz@xxxxxxxxxx
http://www.pencor.com/

--
This is the Midrange Systems Technical Discussion (MIDRANGE-L) mailing list
To post a message email: MIDRANGE-L@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/midrange-l
or email: MIDRANGE-L-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/midrange-l.