HttpURLConnection: BindException when creating many connections

For testing/benchmarking purposes, I want to write a Java program that does the following tasks in a loop:

  • load data via HTTP GET from a server
  • ( generate an answer based on the received data - not important at this point )
  • send the answer via HTTP POST to the same server
  • This cycle runs on multiple threads at the same time.

    After having started, the program runs fine for a short period of time and is able to perform ~300 cycles per thread per second (the webserver runs on the same machine). But after 5-7 seconds, I'm getting BindException: Address already in use .

    Restarting the program after a 20-30 second cool-down time results in the same behavior; when I restart it immediately without waiting, it crashes immediately... so I suppose it could be a problem with bound resources.

    It's a quick and dirty approach using HttpURLConnection . The relevant parts:

    Getting data from the webserver

    public String fetchData() throws IOException {
    
            HttpURLConnection conn = (HttpURLConnection)url.openConnection();
            conn.setUseCaches(false);
            conn.setRequestMethod("GET");
    
            BufferedReader in = new BufferedReader(new InputStreamReader(conn.getInputStream()));
            String inputLine;
            StringBuffer response = new StringBuffer();
    
            while ((inputLine = in.readLine()) != null) {
                response.append(inputLine);
            }
    
            in.close();
            conn.disconnect();
    
            return response.toString();
        }
    

    Sending the answer

    public void sendData(byte[] data) throws IOException {
    
            HttpURLConnection conn = (HttpURLConnection)url.openConnection();
            conn.setUseCaches(false);
            conn.setRequestMethod("POST");
            conn.setDoOutput(true);
    
            OutputStream os = conn.getOutputStream();
            os.write(data);    
            os.close();
            conn.disconnect();
        }
    

    Calling both methods inside the thread

    @Override
    public void run() {
        while(true) {
            try {
                String data = fetchData();
                String answer = // ... generating answer
                sendData(answer.getBytes("UTF-8"));
            } catch (IOException e) {
                // ...
            }
        }
    }
    

    There is not a single URL object that is shared between threads - every thread has its own URL instance (however, each instance points to the same address).

    edit:

    Here's the stack trace of the very first exception that occurs:

    java.net.BindException: Address already in use: connect
        at java.net.DualStackPlainSocketImpl.connect0(Native Method)
        at java.net.DualStackPlainSocketImpl.socketConnect(Unknown Source)
        at java.net.AbstractPlainSocketImpl.doConnect(Unknown Source)
        at java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source)
        at java.net.AbstractPlainSocketImpl.connect(Unknown Source)
        at java.net.PlainSocketImpl.connect(Unknown Source)
        at java.net.SocksSocketImpl.connect(Unknown Source)
        at java.net.Socket.connect(Unknown Source)
        at java.net.Socket.connect(Unknown Source)
        at sun.net.NetworkClient.doConnect(Unknown Source)
        at sun.net.www.http.HttpClient.openServer(Unknown Source)
        at sun.net.www.http.HttpClient.openServer(Unknown Source)
        at sun.net.www.http.HttpClient.<init>(Unknown Source)
        at sun.net.www.http.HttpClient.New(Unknown Source)
        at sun.net.www.http.HttpClient.New(Unknown Source)
        at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(Unknown Source)
        at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(Unknown Source)
        at sun.net.www.protocol.http.HttpURLConnection.plainConnect(Unknown Source)
        at sun.net.www.protocol.http.HttpURLConnection.connect(Unknown Source)
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(Unknown Source)
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
        at PayloadProcessor.fetchData(PayloadProcessor.java:66)
        at PayloadProcessor.run(PayloadProcessor.java:32)
        at java.lang.Thread.run(Unknown Source)
    

    It occurs in the fetchdata method at the following line:

    in = new BufferedReader(new InputStreamReader(conn.getInputStream()));
    

    I removed the calls to disconnect() (as pointed out by Aaron) - unfortunately, the same problem still exists.


    Let's assume everything is closed correctly (not sure if this is the case here, just let assume it) - could it be the case that the program is just too fast? I found another post on stackoverflow, the "solution" was to add a simple Thread.sleep - but since it is a benchmark and should run as fast as possible, I don't like that. How do load testing tools handle this problem?


    I reduced the amount of threads to 1, even then the problem occurs.


    It sounds like you may be running out of local ports. Is it possible that there are too many connections being spawned in parallel and are not releasing their resources fast enough for other threads to reuse?

    I would get rid of the call to disconnect() as that will pretty much disable connection pooling for you. From the docs:

    Indicates that other requests to the server are unlikely in the near future. Calling disconnect() should not imply that this HttpURLConnection instance can be reused for other requests.


    I think you are running out of local port to check this: Run you program and run netstat command. Try using connection pooling for http connection HTTP connection pooling using HttpClient


    Looks like you are running out of local ports. To prevent this, you have to make sure that your connections are reused. Java will try to do this for you, but you need to read the entire response for every request.

    While fetchData function does read responses, sendData doesn't. While you don't need response body here, you should read it anyway (up to the end) and then close the corresponding InputStream . Then this connection might be reused.

    Also don't disconnect your connections.

    This question might be helpful.

    链接地址: http://www.djcxy.com/p/22950.html

    上一篇: jQuery如何根据数据查找元素

    下一篇: HttpURLConnection:创建多个连接时发生BindException