How to get updated value of table sent from C++ to Lua function?

Question

I'm trying to pass a float vector from C++ function to Lua function as a table parameter and then get the updated value of the vector after the Lua function call.

Here's the simple example code.

void myFunc(lua_State *L, std::vector<float> &vec) {

    lua_getglobal(L, "myFunc");
    lua_newtable(L);

    for (size_t i=0; i<vec.size(); ++i) {

        lua_pushinteger(L, i+1);
        lua_pushnumber(L, vec[i]);
        lua_settable(L, -3);
    }
    if (lua_pcall(L, 1, 0, 0) != 0) {

        std::cout << "Error : Failed to call myFunc" << std::endl;
    }
}

And then I can call this function as the following.

std::vector<float> vec = {1,2,3,4,5}; //an array that will be sent to Lua as table
    myFunc(L, vec); //call "myFunc" function in Lua and pass the array as an argument

    /* <Lua function which will be called>
     function myFunc(t)
        for i=1, #t do
            t[i] = t[i] * 2
        end
     end
     */

    //how to update elements of "vec" here so it now becomes {2,4,6,8,10}?

As I commented in the code, I would like to update the elements of vector<float> vec after calling the Lua function.

Is it possible to pass the array to Lua function as a reference? (like how it works in C++ functions)

If not, is it possible to get the values of Lua table(t) so I can write them back to the float vector in C++ after calling the function?

Thanks!

You have two options: (1) Have another C++ function to which you pass the updated table from Lua, which then updates the vector. (2) Pass the vector as userdata to Lua with a metatable which has overloaded __len and __index metamethods which call C++ functions which update the vector inside the userdata inplace (this comes closest to “pass an array by reference to Lua”). — Henri Menke
After lua_pcall you should insert another loop for (size_t i=0; i<vec.size(); ++i) to update the vector (a loop similar to the loop before lua_pcall) — Egor Skriptunoff
@HenriMenke Thanks. I think I found one solution. I changed lua_pcall to lua_pcall(L, 1, 1, 0) so it returns a table. and then added the following code after it. for (size_t i=0; i<vec.size(); ++i) { lua_pushinteger(L, i+1); lua_gettable(L, -2); vec[i] = lua_tonumber(L, -1); lua_pop(L, 1); } Is this similar to your option (1)? I wonder if there's any faster solution than this since it will be used to process audio. — Zack Lee

Henri Menke Henri Menke · Accepted Answer · 2018-05-30T08:42:51

Converting std::vector to and from Lua table

As discussed in chat it might be desirable to convert the function arguments from std::vector<float> to Lua tables and the return value from Lua table to std::vector<float>. The advantage of this is that it is completely transparent at the Lua end.

The function as_table creates a new table from a pair of iterators, from_table converts from a Lua table on top of the stack to a pair of iterators.

#include <algorithm>
#include <cassert>
#include <iostream>
#include <vector>

#include <lua.hpp>

template <typename T, typename U>
void as_table(lua_State* L, T begin, U end) {
    lua_newtable(L);
    for (size_t i = 0; begin != end; ++begin, ++i) {
        lua_pushinteger(L, i + 1);
        lua_pushnumber(L, *begin);
        lua_settable(L, -3);
    }
}

template <typename T, typename U>
void from_table(lua_State* L, T begin, U end) {
    assert(lua_istable(L,-1));
    for (size_t i = 0; begin != end; ++begin, ++i) {
        lua_pushinteger(L, i + 1);
        lua_gettable(L, -2);
        *begin = lua_tonumber(L, -1);
        lua_pop(L, 1);
    }
}

int main(int argc, char *argv[]) {
    if (argc != 2) {
        std::cerr << "Usage: " << argv[0] << " <script.lua>\n";
        return 1;
    }

    lua_State *L = luaL_newstate();
    luaL_openlibs(L);

    if (luaL_dofile(L, argv[1]) != 0) {
        std::cerr << "lua_dofile failed: " << lua_tostring(L, -1) << '\n';
        lua_close(L);
        return 1;
    }

    lua_getglobal(L, "perform");

    std::vector<float> iv(2000, 1);
    std::vector<float> ov(2000, 2);

    as_table(L, iv.begin(), iv.end());
    as_table(L, ov.begin(), ov.end());

    if (lua_pcall(L, 2, 1, 0) != 0) {
        std::cerr << "lua_pcall failed: " << lua_tostring(L, -1)
                  << '\n';
        lua_close(L);
        return 1;
    }

    std::vector<float> w(2000);
    from_table(L, w.begin(), w.end());

    assert(std::all_of(w.begin(), w.end(),
                       [](float p) { return p == 3.0f; }));
}

Here is the little Lua script which is to be used with the testcase above.

function perform(v1,v2)
    local n = math.min(#v1,#v2)
    local v = {}
    for i = 1,n do
        v[i] = v1[i] + v2[i]
    end
    return v
end

std::vector as userdata

Pushing the vector as a table is advantageous if most of the data manipulation is done on the Lua end, because Lua tables are actually pretty fast. If, however, most of the computations were done on the C++ end and the Lua end would only pass around the data between functions implemented in C++ this approach would add a large overhead, because we'd spend lots of time converting back and forth between Lua tables and std::vector. To this end, Lua provides userdata, a method to wrap C/C++ datastructures in such a way that they apprear as native Lua datatypes. The downside is, that when offering functions to inspect userdata from Lua, those are usually slow, because arguments have to be checked repeatedly and multiple nested functions have to be called. Combine this with metatables to have syntactic sugar for array access and length operations and you will find yourself in performance hell.

That said, I have constructed an example of pushing the vector as userdata and set its metatable. This process is also described in the chapter 28.1 – Userdata in the book “Programming in Lua” (read it!).

#include <iostream>
#include <vector>

#include <lua.hpp>

std::vector<float>& checkvector(lua_State *L, int index) {
    std::vector<float> *v = *static_cast<std::vector<float> **>(
        luaL_checkudata(L, index, "std::vector<float>"));
    luaL_argcheck(L, v != nullptr, index, "invalid pointer");
    return *v;
}

static int newvector(lua_State *L) {
    size_t size = luaL_checkinteger(L, 1);
    luaL_argcheck(L, size >= 0, 1, "invalid size");
    *static_cast<std::vector<float> **>(lua_newuserdata(
        L, sizeof(std::vector<float> *))) = new std::vector<float>(size);

    luaL_getmetatable(L, "std::vector<float>");
    lua_setmetatable(L, -2);
    return 1;
}

void pushvector(lua_State *L, std::vector<float> const &v) {
    std::vector<float> *udata = new std::vector<float>();
    *udata = v;
    *static_cast<std::vector<float> **>(lua_newuserdata(
        L, sizeof(std::vector<float> *))) = udata;

    luaL_getmetatable(L, "std::vector<float>");
    lua_setmetatable(L, -2);
}

static int deletevector(lua_State *L) {
    delete &checkvector(L, 1);
    return 0;
}

static int setvector(lua_State *L) {
    std::vector<float> &v = checkvector(L, 1);
    size_t index = luaL_checkinteger(L, 2) - 1;
    luaL_argcheck(L, index < v.size(), 2, "index out of range");
    luaL_argcheck(L, lua_isnumber(L, 3), 3, "not a number");
    float record = lua_tonumber(L, 3);

    v.at(index) = record;

    return 0;
}

static int getvector(lua_State *L) {
    std::vector<float> &v = checkvector(L, 1);
    size_t index = luaL_checkinteger(L, 2) - 1;
    luaL_argcheck(L, index < v.size(), 2, "index out of range");

    lua_pushnumber(L, v.at(index));

    return 1;
}

static int getsize(lua_State *L) {
    std::vector<float> &v = checkvector(L, 1);

    lua_pushinteger(L, v.size());

    return 1;
}

static int vectortostring(lua_State *L) {
    std::vector<float> &v = checkvector(L, 1);

    lua_pushfstring(L, "std::vector<float>(%d)", v.size());

    return 1;
}

static const struct luaL_Reg vector_float_lib[] = {
    {"new", newvector},
    {nullptr, nullptr} // sentinel
};

static const struct luaL_Reg vector_float_meta[] = {
    {"__tostring", vectortostring},
    {"__newindex", setvector},
    {"__index", getvector},
    {"__len", getsize},
    {"__gc", deletevector},
    {nullptr, nullptr} // sentinel
};

int luaopen_vector_float(lua_State *L) {
    luaL_newmetatable(L, "std::vector<float>");
    luaL_setfuncs(L, vector_float_meta, 0);
    luaL_newlib(L, vector_float_lib);
    return 1;
}


static int send_vector(lua_State *L) {
    std::vector<float> v = { 1, 2, 3, 4 };
    pushvector(L,v);
    return 1;
}

static int retrieve_vector(lua_State *L) {
    std::vector<float> &v = checkvector(L, 1);
    for (auto const &p : v) {
        std::cout << p << '\n';
    }
    return 0;
}

int main(int argc, char *argv[]) {
    lua_State *L = luaL_newstate();
    luaL_openlibs(L);

    luaL_requiref(L, "vector", luaopen_vector_float, 1);
    lua_pop(L, 1);

    lua_pushcfunction(L,send_vector);
    lua_setglobal(L,"send_vector");

    lua_pushcfunction(L,retrieve_vector);
    lua_setglobal(L,"retrieve_vector");

    if (argc != 2) {
        std::cerr << "Usage: " << argv[0] << " <script.lua>\n";
        return 1;
    }

    luaL_dofile(L, argv[1]);

    lua_close(L);
}

This can execute the following Lua script

local v = send_vector()
for i = 1,#v do
    v[i] = 2*v[i]
end
retrieve_vector(v)

Given a global function transform_vector, e.g.

function transform_vector(v)
    for i = 1,#v do
        v[i] = 2*v[i]
    end
    return v
end

one can call this function with a vector argument and retrieve a vector result just like with any other Lua function.

std::vector<float> v = { 1, 2, 3, 4 };
lua_getglobal(L,"transform_vector");
pushvector(L,v);
if (lua_pcall(L,1,1,0) != 0) {
    // handle error
}
std::vector<float> w = checkvector(L, -1);

How to get updated value of table sent from C++ to Lua function?

1 Answers

Converting std::vector to and from Lua table

std::vector as userdata