Hi All,

Our QA department is implementing new remote commands initiated from a WebSphere (AIX) Application Server and they are reporting that 20%-30% of the remote command connections are failing with the response code:


<ResponseInfo code="6003" description="Connection Exception" message="Established connection failed to component:AS400_CMD:216.58.195.132" source="AS400_CMD"/>


I asked for a java log showing the error and received the following:

"2016-05-02 00:14:35.527 [ERROR] [WebContainer : 192] [1079c481-7768-4f48-a159-00bf23a09c8d] com.ABC.soa.msg.AppMsgImpl@d010db90[code=6003,description=Connection Exception,message=Established connection failed to component:AS400_CMD:216.58.195.132,system=AS400_CMD]
com.ABC.soa.exception.SystemException: com.ABC.soa.msg.AppMsgImpl@d010db90[code=6003,description=Connection Exception,message=Established connection failed to component:AS400_CMD:216.58.195.132,system=AS400_CMD]

at com.ABC.soa.icom.common.As400CommandCall.execute(As400CommandCall.java:289) ~[soa-icom-commons.jar:na]
at com.ABC.soa.icom.dao.AccountsReceivableDao2Impl.getActualDaysPastDue(AccountsReceivableDao2Impl.java:137) ~[soa-icom-commons.jar:na]
at com.ABC.soa.billing.dao.IcomsRetrieveBillingInfoV1x2DaoImpl.retrieveBillingInfo(IcomsRetrieveBillingInfoV1x2DaoImpl.java:173) ~[soa-billing-services.jar:na]
at com.ABC.soa.billing.dao.IcomsRetrieveBillingInfoV1x2DaoImpl.getBilling(IcomsRetrieveBillingInfoV1x2DaoImpl.java:142) ~[soa-billing-services.jar:na]
at com.ABC.soa.billing.service.RetrieveBillingInfoV1x2ServiceImpl.getBilling(RetrieveBillingInfoV1x2ServiceImpl.java:85) ~[soa-billing-services.jar:na]
at com.ABC.soa.billing.endpoint.RetrieveBillingInfoV1x2EndPoint.retrieveBilling(RetrieveBillingInfoV1x2EndPoint.java:64) ~[soa-billing-services.jar:na]
at sun.reflect.GeneratedMethodAccessor240.invoke(Unknown Source) ~[na:na]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37) ~[na:1.6.0]
at java.lang.reflect.Method.invoke(Method.java:611) ~[na:1.6.0]
at org.springframework.ws.server.endpoint.MethodEndpoint.invoke(MethodEndpoint.java:134) ~[spring-ws-2.1.3.RELEASE-all.jar:2.1.3.RELEASE]
at org.springframework.ws.server.endpoint.adapter.MarshallingMethodEndpointAdapter.invokeInternal(MarshallingMethodEndpointAdapter.java:140) ~[spring-ws-2.1.3.RELEASE-all.jar:2.1.3.RELEASE]
at org.springframework.ws.server.endpoint.adapter.AbstractMethodEndpointAdapter.invoke(AbstractMethodEndpointAdapter.java:53) ~[spring-ws-2.1.3.RELEASE-all.jar:2.1.3.RELEASE]
at org.springframework.ws.server.MessageDispatcher.dispatch(MessageDispatcher.java:233) [spring-ws-2.1.3.RELEASE-all.jar:2.1.3.RELEASE]
at org.springframework.ws.server.MessageDispatcher.receive(MessageDispatcher.java:173) [spring-ws-2.1.3.RELEASE-all.jar:2.1.3.RELEASE]
at org.springframework.ws.transport.support.WebServiceMessageReceiverObjectSupport.handleConnection(WebServiceMessageReceiverObjectSupport.java:88) [spring-ws-2.1.3.RELEASE-all.jar:2.1.3.RELEASE]
at org.springframework.ws.transport.http.WebServiceMessageReceiverHandlerAdapter.handle(WebServiceMessageReceiverHandlerAdapter.java:59) [spring-ws-2.1.3.RELEASE-all.jar:2.1.3.RELEASE]
at org.springframework.ws.transport.http.MessageDispatcherServlet.doService(MessageDispatcherServlet.java:239) [spring-ws-2.1.3.RELEASE-all.jar:2.1.3.RELEASE]
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:936) [spring-webmvc-3.2.3.RELEASE.jar:3.2.3.RELEASE]
at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:838) [spring-webmvc-3.2.3.RELEASE.jar:3.2.3.RELEASE]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:595) [javax.j2ee.servlet.jar:na]
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:812) [spring-webmvc-3.2.3.RELEASE.jar:3.2.3.RELEASE]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:668) [javax.j2ee.servlet.jar:na]
at com.ibm.ws.webcontainer.servlet.ServletWrapper.service(ServletWrapper.java:1230) [com.ibm.ws.webcontainer.jar:na]
at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:779) [com.ibm.ws.webcontainer.jar:na]
at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:478) [com.ibm.ws.webcontainer.jar:na]
at com.ibm.ws.webcontainer.servlet.ServletWrapperImpl.handleRequest(ServletWrapperImpl.java:178) [com.ibm.ws.webcontainer.jar:na]
at com.ibm.ws.webcontainer.filter.WebAppFilterManager.invokeFilters(WebAppFilterManager.java:1071) [com.ibm.ws.webcontainer.jar:na]
at com.ibm.ws.webcontainer.servlet.CacheServletWrapper.handleRequest(CacheServletWrapper.java:87) [com.ibm.ws.webcontainer.jar:na]
at com.ibm.ws.webcontainer.WebContainer.handleRequest(WebContainer.java:914) [com.ibm.ws.webcontainer.jar:na]
at com.ibm.ws.webcontainer.WSWebContainer.handleRequest(WSWebContainer.java:1662) [com.ibm.ws.webcontainer.jar:na]
at com.ibm.ws.webcontainer.channel.WCChannelLink.ready(WCChannelLink.java:200) [com.ibm.ws.webcontainer.jar:na]
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleDiscrimination(HttpInboundLink.java:453) [com.ibm.ws.runtime.jar:na]
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleNewRequest(HttpInboundLink.java:515) [com.ibm.ws.runtime.jar:na]
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.processRequest(HttpInboundLink.java:306) [com.ibm.ws.runtime.jar:na]
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.ready(HttpInboundLink.java:277) [com.ibm.ws.runtime.jar:na]
at com.ibm.ws.tcp.channel.impl.NewConnectionInitialReadCallback.sendToDiscriminators(NewConnectionInitialReadCallback.java:214) [na:CCX.CF [o1330.02]]
at com.ibm.ws.tcp.channel.impl.NewConnectionInitialReadCallback.complete(NewConnectionInitialReadCallback.java:113) [na:CCX.CF [o1330.02]]
at com.ibm.ws.tcp.channel.impl.AioReadCompletionListener.futureCompleted(AioReadCompletionListener.java:175) [com.ibm.ws.runtime.jar:na]
at com.ibm.io.async.AbstractAsyncFuture.invokeCallback(AbstractAsyncFuture.java:217) [com.ibm.ws.runtime.jar:na]
at com.ibm.io.async.AsyncChannelFuture$1.run(AsyncChannelFuture.java:205) [com.ibm.ws.runtime.jar:na]
at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1702) [com.ibm.ws.runtime.jar:na]
com.ibm.as400.access.ConnectionDroppedException: Disconnect request received, connection terminated.
at com.ibm.as400.access.AS400ThreadedServer.forceDisconnect(AS400ThreadedServer.java:237) ~[jt400.jar:JTOpen 7.4]
at com.ibm.as400.access.AS400ImplRemote.disconnectServer(AS400ImplRemote.java:453) ~[jt400.jar:JTOpen 7.4]
at com.ibm.as400.access.AS400ImplRemote.disconnect(AS400ImplRemote.java:425) ~[jt400.jar:JTOpen 7.4]
at com.ibm.as400.access.AS400.disconnectAllServices(AS400.java:1203) ~[jt400.jar:JTOpen 7.4]
at com.ibm.as400.access.ConnectionList.removeExpiredConnections(ConnectionList.java:648) ~[jt400.jar:JTOpen 7.4]
at com.ibm.as400.access.AS400ConnectionPool.cleanupConnections(AS400ConnectionPool.java:137) ~[jt400.jar:JTOpen 7.4]
at com.ibm.as400.access.PoolMaintenance.run(PoolMaintenance.java:86) ~[jt400.jar:JTOpen 7.4]"


looking at the classes in in the java logs, it appears that WAS makes a new connection request from the connection pool but immediately the ConnectionDroppedException class is invoked and then the iSeries cleans up the disconnected connection from the connection pool.


I have an open PMR with IBM on both sides but we are finding this difficult to troubleshoot. Can anyone give any advice for gathering more detailed insight into digging into this error?


Jim

As an Amazon Associate we earn from qualifying purchases.

This thread ...

Follow-Ups:

Follow On AppleNews
Return to Archive home page | Return to MIDRANGE.COM home page

This mailing list archive is Copyright 1997-2021 by midrange.com and David Gibbs as a compilation work. Use of the archive is restricted to research of a business or technical nature. Any other uses are prohibited. Full details are available on our policy page. If you have questions about this, please contact [javascript protected email address].

Operating expenses for this site are earned using the Amazon Associate program and Google Adsense.