LuaSocket: Network support for the Lua language

Streaming with Callbacks

HTTP and FTP transfers sometimes involve large amounts of information. Sometimes an application needs to generate outgoing data in real time, or needs to process incoming information as it is being received. To address these problems, LuaSocket allows HTTP message bodies and FTP file contents to be received or sent through the callback mechanism outlined below.

Instead of returning the entire contents of a FTP file or HTTP message body as strings to the Lua application, the library allows the user to provide a receive callback that will be called with successive chunks of data, as the data becomes available. Conversely, the send callbacks should be used when data needed by LuaSocket is generated incrementally by the application.

receive_cb(chunk, err)

The callback provided by the user will be repeatedly called by the library whenever new data is available. Each time it is called, the callback receives successive chunks of downloaded data.

Chunk contains the current chunk of data. When the transmission is over, the function is called with an empty string (i.e. "") as the chunk. If an error occurs, the function receives nil as chunk and an error message as err.

The callback can abort transmission by returning nil as its first return value. In that case, it can also return an error message. Any non-nil return value proceeds with the transmission.

-- The implementation of socket.callback.receive_concat
function Public.receive_concat(concat)
    concat = concat or socket.concat.create()
    local callback = function(chunk, err)
        -- if not finished, add chunk
        if chunk and chunk ~= "" then
            concat:addstring(chunk)
            return 1
        end
    end
    return callback, concat
end

send_cb()

The callback provided by the user will be repeatedly called whenever the library needs more data to be sent.

Each time the callback is called, it should return the next part of the information the library is expecting, followed by the total number of bytes to be sent. The callback can abort the process at any time by returning nil followed by an optional error message.

Note: The need for the second return value comes from the fact that, with the HTTP protocol for instance, the library needs to know in advance the total number of bytes that will be sent.

-- The implementation of socket.callback.send_file
function Public.send_file(file)
    local callback
    -- if successfull, return the callback that reads from the file
    if file then
        -- get total size
        local size = file:seek("end") 
        -- go back to start of file
        file:seek("set")
        callback = function()
            -- send next block of data
            local chunk = file:read(Public.BLOCKSIZE)
            if not chunk then file:close() end
            return chunk, size
        end
    -- else, return a callback that just aborts the transfer
    else
        callback = function() 
            -- just abort
            return nil, "unable to open file"
        end
    end
    return callback, file
end