RE: SSL client connection error - SSL_Handshake(): Peer not recognized or badly formatted message received.

Paul.

Recently I was involved in a project with a customer
who has a app. using sockets between a windows box and a iSeries.
The project was to change to SSL.

Windows is third party app and no source programs.
we only know the windows app was ready to use SSL.
No information about TLS or cypherSuite used by Windows app.

Book "iseries socket programming" has a very good examples
using GSKit and SSL_API, but written in "C", also has a flow
of events for socket and for SSL (Chapter 10).

We used these examples to test the communications with
the windows box and eventually we received the
"Peer not recognized or badly formatted message",
we were changing the ciphersuite and the TLS, and finally
found the ciphersuite and TLS used by the windows app.

In short what we did was:
.- Create a local certificate authority(CA), with DCM.
.- Issue a digital certificate.
.- Create (define) a app in DCM(TESTSSL).
.- Assign the certificate to TESTSSL.
.- We modify these examples to test the communications
with the windows box.
.- Export the certificate to the IFS with the ",cer" extension.
.- Import the certificate to the windows box(done by the customer).
.- Test...test... test.. Eventually we received the
"Peer not recognized or badly formatted message", we were changing
the ciphersuite, the TLS protocol and finally found the ciphersuite and
the TLS used by
the windows app.
.- Finally works.

The app in iseries side was changed to use SSL_API(no GSKit).

Saludos.
Alf.

-----Mensaje original-----
De: MIDRANGE-L [mailto:midrange-l-bounces@xxxxxxxxxxxx] En nombre de
Steinmetz, Paul
Enviado el: lunes, 29 de junio de 2015 09:44 a.m.
Para: 'Midrange Systems Technical Discussion'
Asunto: RE: SSL client connection error - SSL_Handshake(): Peer not
recognized or badly formatted message received.

Scott,

1) How do we know if an app is using GSKit for SSL?
2) If we do have an app using GSKit for SSL, I would like the configuration
info your referring to?

Paul

-----Original Message-----
From: MIDRANGE-L [mailto:midrange-l-bounces@xxxxxxxxxxxx] On Behalf Of Scott
Klement
Sent: Friday, June 26, 2015 11:11 AM
To: Midrange Systems Technical Discussion
Subject: Re: SSL client connection error - SSL_Handshake(): Peer not
recognized or badly formatted message received.

Since he references SSL_Handshake(), I'm assuming that he's using the SSL
APIs.

But if someone does need info about how to configure the versions in GSKit,
let me know, I can provide that...



On 6/26/2015 9:10 AM, Bradley Stone wrote:

Hi Paul.

What are you using to connect/communicate? Can you get the return code?

Do you know if the GSKit APIs are used or the standard SSL APIs are
being used for the connect?

I ran into an issue with a customer on V7R1 that was trying to use
V7R1 and up and the SSL APIs weren't really doing things right, so on
the SSL Handshake API we had to tell it by sending it the proper code
and that cleared things up.

Here's a link to an article I wrote about it.. it refers to GETURI but
it would also apply to any client application that uses the SSL APIs.
(the GSKit APIs may have a different setting).

http://www.fieldexit.com/forum/display?threadid=170

Brad
www.bvstools.com

On Fri, Jun 26, 2015 at 8:06 AM, Steinmetz, Paul
<PSteinmetz@xxxxxxxxxx>
wrote:

I'm receiving this error when trying to connect to a remote server.

SSL_Handshake(): Peer not recognized or badly formatted message received.

V7R1, TR10, latest CUM 15142 and all groups

I've confirmed DCM has proper CA, both root and intermediate.
Remote server has TLS1.0 disabled, TLS1.2 is currently being used for
other connections to that server.
I'm thinking this is either a SSL protocol issue or cipher issue.
I know when the I is the server, the DCM application defaults need to
be changed to allow TLS1.2 , TLS1.1 and disable SSL 3.0, SSL2.0 Also
cipher defaults need to be changed.

Are there similar settings for when the I is the client?
I've seen other posts with this error, but did not see the final
resolution.


- - - - - - - - - - - - - - - - - - - - - - - C O N N E C T I O N F E E
D B A C K -
About to connect() to XXXXXX-web.XXX.net port 443 (#0)
Trying XXX.XXX.XXX.X... connected
SSL_Handshake(): Peer not recognized or badly formatted message received.
Closing connection #0
SSL connect error
************End of Data********************


Thank You
_____
Paul Steinmetz
IBM i Systems Administrator

Pencor Services, Inc.
462 Delaware Ave
Palmerton Pa 18071

610-826-9117 work
610-826-9188 fax
610-349-0913 cell
610-377-6012 home

psteinmetz@xxxxxxxxxx
http://www.pencor.com/

--
This is the Midrange Systems Technical Discussion (MIDRANGE-L)
mailing list To post a message email: MIDRANGE-L@xxxxxxxxxxxx To
subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/midrange-l
or email: MIDRANGE-L-request@xxxxxxxxxxxx Before posting, please take
a moment to review the archives at
http://archive.midrange.com/midrange-l.

--
This is the Midrange Systems Technical Discussion (MIDRANGE-L) mailing list
To post a message email: MIDRANGE-L@xxxxxxxxxxxx To subscribe, unsubscribe,
or change list options,
visit: http://lists.midrange.com/mailman/listinfo/midrange-l
or email: MIDRANGE-L-request@xxxxxxxxxxxx Before posting, please take a
moment to review the archives at http://archive.midrange.com/midrange-l.

--
This is the Midrange Systems Technical Discussion (MIDRANGE-L) mailing list
To post a message email: MIDRANGE-L@xxxxxxxxxxxx To subscribe, unsubscribe,
or change list options,
visit: http://lists.midrange.com/mailman/listinfo/midrange-l
or email: MIDRANGE-L-request@xxxxxxxxxxxx Before posting, please take a
moment to review the archives at http://archive.midrange.com/midrange-l.


---
El software de antivirus Avast ha analizado este correo electrónico en busca de virus.
https://www.avast.com/antivirus