使用httpclient时候，出现“Too many open files”问题

提示：扫一扫查出行
【扫一扫了解最新限行尾号】
复制提示

最近接触的sendHttpPostRequest的问题比较多，近期碰到了关于 "java.net.SocketException: Too many open files" 的问题

异常信息如下：

ERROR | 2016-01-12 03:19:36.642 | ERROR | IREAD | [Exception]exception stack info | com.huawei.bme.commons.util.debug.DebugLogImpl.doLog(DebugLogImpl.java:480)
java.net.SocketException: Too many open files
at java.net.Socket.createImpl(Socket.java:397)
at java.net.Socket.getImpl(Socket.java:460)
at java.net.Socket.bind(Socket.java:577)
at sun.reflect.GeneratedMethodAccessor26.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.commons.httpclient.protocol.ReflectionSocketFactory.createSocket(ReflectionSocketFactory.java:139)
at org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:125)
at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707)
at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361)
at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387)
at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)
at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323)
at com.huawei.iread.duibaintf.duiba.sns.SNSServer.sendPostRequestWithCode(SNSServer.java:296)
at com.huawei.iread.duibaintf.duiba.sns.SNSEngine.sendPostRequestWithCode(SNSEngine.java:92)
at com.huawei.iread.duibaintf.duiba.duibamall.GetDuibaExchangeResultNotifyImpl.getDuibaExchangeResultNotify(GetDuibaExchangeResultNotifyImpl.java:91)

其实也就是因为 sendPostRequestWithCode 和 getDuibaExchangeResultNotify 方法调用了 HttpClient.executeMethod 引起的。

查找linux服务器中open files文件数，发现好多是下面的情况，意思是socket资源泄漏，进程使用的sock未关闭：

COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
java 27185 duiba 100u sock 0,7 0t0 92280355 can't identify protocol
java 27185 duiba 101u sock 0,7 0t0 92282035 can't identify protocol
java 27185 duiba 102u sock 0,7 0t0 92280901 can't identify protocol
java 27185 duiba 103u sock 0,7 0t0 92282036 can't identify protocol
java 27185 duiba 104u sock 0,7 0t0 92280389 can't identify protocol
java 27185 duiba 105u sock 0,7 0t0 92280390 can't identify protocol
java 27185 duiba 106u sock 0,7 0t0 92280391 can't identify protocol
java 27185 duiba 107u sock 0,7 0t0 92280392 can't identify protocol

Java源代码中是这样写的：

try{
httpMethod = new PostMethod(url)
// 设置header信息，传输XML格式的
httpMethod.setRequestHeader(CONTENT_TYPE_NAME, CONTENT_TYPE_VALUE_XML_UTF_8);
httpMethod.setRequestEntity(entity); HttpClient httpClient = new HttpClient(new MultiThreadedHttpConnectionManager());//使用连接池技术创建
httpClient.executeMethod(httpMethod);
}catch{...}
finally{

if (null != httpMethod)
{
httpMethod.releaseConnection();//释放资源
}

}

从上面看，最后， httpMethod.releaseConnection();//释放资源。实际上，HttpClient建立Socket时，post.releaseConnection()并没有真正关闭连接，而是将该连接提交给 MultiThreadedHttpConnectionManager，等待复用。而http的连接是等待timeout才会自动断开的，所以，当用完系统的句柄后，自然会报Too many open files。

所以，应该添加下面的代码进行释放socket：

try{
httpMethod = new PostMethod(url)
// 设置header信息，传输XML格式的
httpMethod.setRequestHeader(CONTENT_TYPE_NAME, CONTENT_TYPE_VALUE_XML_UTF_8);
httpMethod.setRequestEntity(entity);
//请求头信息中添加关闭连接
httpMethod.addRequestHeader("Connection", "close"); HttpClient httpClient = new HttpClient(new MultiThreadedHttpConnectionManager());//使用连接池技术创建
httpClient.executeMethod(httpMethod);
//连发三次机制
httpClient.getParams().setBooleanParameter("http.protocol.expect-continue", false);
}catch{...}
finally{
if (null != httpMethod)
{
httpMethod.releaseConnection();//释放资源
}}

还有一种说法：

linux的文件打开数超过了默认的1024个文件，具体查看linux下面的文件打开最大值的命令ulimit -a，出现信息如下：

core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 69631
max locked memory (kbytes, -l) 32
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 10240
cpu time (seconds, -t) unlimited
max user processes (-u) 69631
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimite

其中open files (-n) 1024 默认为1024，
有说解决方法是加大文件打开的最大值，ulimit -n 8192
但是由于Httpclient引起的。当并发访问量过大的时候，这个方法只是治标不治本的。所有还是解决Httpclient远程请求所造成Socket没有释放，使用上面的方法较好。