java http长链接(keep-alive)导致的问题


两种由http长链接(keep-alive)导致的问题，当然这两种问题都有多种原因导致，这里只分析针对keep-alive相关而产生的异常。

1 SocketException: Connection reset

报错堆栈日志：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
Caused by: java.net.SocketException: Connection reset
        at java.net.SocketInputStream.read(SocketInputStream.java:209)
        at java.net.SocketInputStream.read(SocketInputStream.java:141)
        at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)
        at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153)
        at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:282)
        at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138)
        at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
        at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
        at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
        at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165)
        at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
        at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
        at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
        at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
        at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
        at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)
        at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:72)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:221)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:165)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:140)

关于SocketException在官方文档有描述：https://docs.oracle.com/javase/8/docs/technotes/guides/net/articles/connection_release.html

本质原因是服务器通过TCP协议给客户端返回了RST消息，表示已经完成了发送和接收，如果客户端此时从流中读取数据时会发生Connection reset，往流中写数据时就会发生Connection reset Connection reset by peer。注意Socket.close()语义和TCP FIN消息之间略有不匹配。
而至于为什么服务端会返回RST消息，那就是http keep-alive 导致的问题了。
如果是springboot的服务器，那么默认的keep-alive timeout是60s，如果客户端使用的是apache httpclient，默认的keep-alive是在
org.apache.http.impl.client.DefaultConnectionKeepAliveStrategy类设置的，代码如下：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
public long getKeepAliveDuration(final HttpResponse response, final HttpContext context) {
        Args.notNull(response, "HTTP response");
        final HeaderElementIterator it = new BasicHeaderElementIterator(
                response.headerIterator(HTTP.CONN_KEEP_ALIVE));
        while (it.hasNext()) {
            final HeaderElement he = it.nextElement();
            final String param = he.getName();
            final String value = he.getValue();
            if (value != null && param.equalsIgnoreCase("timeout")) {
                try {
                    return Long.parseLong(value) * 1000;
                } catch(final NumberFormatException ignore) {
                }
            }
        }
        return -1;
    }

可见，如果response header如果没有返回Keep-Alive,那么就会是-1，也就是无限的，https://hc.apache.org/httpcomponents-client-4.5.x/tutorial/html/connmgmt.html "If the Keep-Alive header is not present in the response, HttpClient assumes the connection can be kept alive indefinitely"

所以问题就比较明白了，springboot服务端没有返回Keep-Alive的header，客户端如果使用了apache httpclient，且没有设置Keep-Alive的话，就会导致服务端的超时是60s,客户端就认为是无限的，在某些情况下，服务端关闭了链接，客户端还会获取这个连接，就会导致上面的问题。

既然找到原因了那么就需要解决，httpclient文档中也给了说要设置默认的超时时间，即给一个自定义的实现。

2 NoHttpResponseException: xxx.xxx.xxx.xxx failed to respond

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
Caused by: org.apache.http.NoHttpResponseException: xxx.xxx.xxx.xxx failed to respond

at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:141)

at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)

at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)

at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)

at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165)

at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)

at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)

at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)

at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)

at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)

at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)

at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)

at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:72)

at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:221)

at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:165)

at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:140)

补充知识点：

1 'Connection: Keep-Alive' header头只用来在HTTP 1.0中， 而在HTTP 1.1中默认都是Keep-Alive的，所以不需要再添加'Connection: Keep-Alive'的头。所以在springboot1.5以上版本中，即使你在请求的header中加了'Connection: Keep-Alive'的头，在返回的header中是没有Connection的。相关链接：https://github.com/spring-projects/spring-boot/issues/4126

2 curl 命令可以使用--http1.0 来强制走http1.0协议。

3 springboot中 tomcat 默认的keepalive timeout是60s， https://github.com/spring-projects/spring-boot/issues/11955#issuecomment-364432662