Lua HTTP Request Timeout Hang

Question

I have inherited some Lua code that communicates with a server via HTTP Request/Response. This has been running in the field for several years but I recently noticed some hangs in the timeout of the HTTP requests from the system. Each individual message has the capability to define its own timeout, but if none is set a default timeout of 30 seconds is used. I noticed on a system a couple of weeks ago that there was a hang on a timeout for about 15 minutes before it recovered and processing continued. But I am currently looking at a system that has been hung for well over 3 hours on a 30 second timeout. Here is the setup for the requests:

local socket = require "socket"
local http = require "socket.http"
local ltn12 = require "ltn12"
local ssl = require "ssl"
local try = socket.try
local protect = socket.protect
...
function serverapi.http_request(request, timeout)
    ... (local variable setup and logging)
    socket.TIMEOUT = timeout
    socket.http.TIMEOUT = timeout

    result, status_code, content = socket.http.request {
        url = request.url,
        method = request.method,
        headers = request.header,
        source = ltn12.source.string(request_body),
        sink = ltn12.sink.table(response_body),
    }

    ... (receive response and process)

I should note that the hangs are erratic in terms of the messages types they hang on. So it is not consistently with one message. And, as I said, this has been deployed and running in the field for several years.

Anyone have any ideas here...? Even if it's just a way to help debug what's going on. I don't even know how to get any kind of logging in what's going on after the request is sent and it's waiting for timeout.

Thanks

What does the hang look like exactly? Where in the request is it hanging? Initial connection? Sending of the request? Waiting on the response to start? Waiting on the response to finish? — Etan Reisner
There is a log message immediately before the call to socket.http.request that logs the timeout and one immediately after printing the status code received. So the hang must be in that call. — linsek
can you capture the requests with wireshark then play them back to see if you get the same ones to hang? can you attach a debugger to the process? — Oliver

Keith Mashinter Keith Mashinter · Accepted Answer · 2014-07-13T02:12:07

I've had similar issues, and the socket library is old, last updated in 2007? I fix a bug in socket/http.lua, but it still occasionally hangs so that wasn't it. I tried luacurl and it also hangs, so it may be a problem at the compiled code level. I'm on Windows. nodejs has more momentum these days so I'm not sure how much I want to pursue Lua unless support improves.

while string.find(line, "^%s") do
    value = value .. line
    line, err = sock:receive() -- was line = sock:receive()
    if err then return nil, err end
end

Lua HTTP Request Timeout Hang

1 Answers