Issue SocketTimeoutException – (Apache Mod-jk & JBoss)

Hi folks, from last week I am facing “SocketTimeoutException”.  JBoss servers getting down automatically getting above exception also there were spike in memory utilization.

This SocketTimeoutException is occurred when the system was idle for more time, and there is no request on applications and getting below errors, which is  observed on Apache(mod_jk) & JBoss instance logs.

Caused by: java.net.SocketTimeoutException: SocketTimeoutException invoking http://192.168.104.92:8081/Homeport/DecisionSetupService: Read timed out

Apache logs:

File name : mod_jk.log

****************************************

[Mon Aug 06 08:18:04 2018][1756:5104] [info] ajp_handle_cping_cpong::jk_ajp_common.c (913): timeout in reply cpong

[Mon Aug 06 08:18:04 2018][1756:5104] [info] ajp_send_request::jk_ajp_common.c (1562): (standaloneProd) failed sending request, socket -1 prepost cping/cpong failure (errno=60)

 [Mon Aug 06 08:18:16 2018]standaloneProd 192.168.104.91 17.753141

[Mon Aug 06 08:18:16 2018]standaloneProd www.middlewarebox.com 22.620435

[Mon Aug 06 08:18:29 2018][1756:1604] [info] ajp_handle_cping_cpong::jk_ajp_common.c (913): timeout in reply cpong

***************************************

JBoss logs:

file name : 1-instance (server.log)

**************************************

08:23:33,449 ERROR [Booking] (ajp-/192.168.104.92:8009-6) [com.griffin.homeport.reservation.booking.interceptors.BookingHistoryInterceptor.intercept:136] Exception in Booking History Interceptor: javax.xml.ws.WebServiceException: Could not send Message.

                at org.apache.cxf.jaxws.JaxWsClientProxy.invoke(JaxWsClientProxy.java:145) [cxf-rt-frontend-jaxws-2.4.6.jar:2.4.6]

                at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:931) [jbossweb-7.0.17.Final.jar:]

                at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_35]

Caused by: java.net.SocketTimeoutException: SocketTimeoutException invoking http://192.168.104.92:8081/Homeport/DecisionSetupService: Read timed out

   Caused by: java.net.SocketTimeoutException: Read timed out

                at java.net.SocketInputStream.socketRead0(Native Method) [rt.jar:1.6.0_35]

                at java.net.SocketInputStream.read(SocketInputStream.java:129) [rt.jar:1.6.0_35]

               

08:24:00,812 ERROR [stderr] (pool-10-thread-610) java.io.IOException: An established connection was aborted by the software in your host machine 

****************************************

This issue is happening due, to long time idle AJP connections, for removing these idle connections

need to add below configuration

1 : On JBoss instance (standalone.xml).

**********************************

<system-properties>

                 <property name="org.apache.coyote.ajp.DEFAULT_CONNECTION_TIMEOUT" value="600000"/>

                <property name="org.apache.coyote.http11.DEFAULT_CONNECTION_TIMEOUT" value="120000"/>

<system-properties>

**********************************

Default connection timeout :

AJP connectionTimeout to 600000 (600000 millseconds = 600 seconds = 10 minutes)

HTTP connectionTimeout to 120000 (120000 millseconds = 120 seconds = 2 minutes)

2: On Apache Mod_jk (worker.properties)

**********************************

worker.list=standaloneProd,status

worker.standaloneProd.port=8009

worker.standaloneProd.host=192.168.104.92

worker.standaloneProd.type=ajp13

worker.standaloneProd.ping_mode=A

worker.standaloneProd.socket_connect_timeout=10000

worker.standaloneProd.connection_pool_size=150

worker.standaloneProd.connection_pool_timeout=600

worker.status.type=status