HTTPoison manually close connection

3
votes

This is a very simple module that keeps checking the requester's IP. I use backconnect proxies which means that it gets new IP on every http request.

defmodule Test do
  def go() do
    Enum.each(1..1, fn x ->
    Task.Supervisor.async_nolink(Scraper.TaskSupervisor, fn ->
      r = HTTPoison.get("https://api.ipify.org?format=json", [],
        [timeout: 10_000, recv_timeout: 10_000, proxy: "ip:port", ssl: [{:versions, [:'tlsv1.2']}]])

      case r do

        {:ok, %HTTPoison.Response{status_code: 200, body: body}} ->
          IO.inspect body |> Jason.decode
          :timer.sleep(1000)
          go()
      end
    end)
    end)

  end
end
Test.go()
:timer.sleep(2000000)

Problem? HTTPoison(hackney) doesn't release the connection as long as process is alive so IP is always the same. How would I manually close the connection inside:

{:ok, %HTTPoison.Response{status_code: 200, body: body}} ->
          IO.inspect body |> Jason.decode
2
elixirhttpoison

2 Answers

2
votes

From the HTTPoison README:

Normally hackney opens and closes connections on demand, but it also creates a default pool of connections which are reused for requests to the same host. If the connection and host support keepalive, the connection is kept open until explicitly closed.

HTTPoison does not export the close for the underlying socket, and there is an open issue for that since 2017.

Meanwhile, you might call hackney_pool:stop_pool(:default). This is obviously not the optimal solution by any mean and it has a huge overhead. So my advise would be to either provide a pull request to HTTPoison enabling close connection functionality (via delegating to ;hackney_response.close/1) or simply get rid of redundant HTTPoison and just use hackney.

2
votes

Simplest solution turned out bo be setting pool: false, the entire request looks like:

HTTPoison.get(url, [], [
  timeout: 5_000,
  recv_timeout: 5_000,
  proxy: proxy,
  hackney: [pool: false],
  ssl: [{:versions, [:'tlsv1.2']}]
])