How to detect if a userdata has "environment table"?

Question

In Lua 5.1, one can associate an environment table with a userdata. This lets us add "fields" to individual userdatas.

The obvious approach would be to create this environment table, possibly an empty one, right when you create the userdata. However, in my application there'll be many userdatas that won't necessarily need this environment table and they come and go out of existence rapidly. I don't want the overhead of creating so many temporary empty tables that won't be used.

So, instead, I thought of creating the environment table only when I detect that the userdata doesn't already have one created by me.

The problem is that in Lua 5.1 the default environment table isn't nil. It's supposedly the global table, _G (I wonder how on earth this is useful). So, supposedly, I'd test for a non-initialized userdata by doing:

/* 'index' is where my userdata is */
lua_getfenv(L, index);
initialized = lua_rawequal(L, index, LUA_GLOBALSINDEX);

Now, my question:

Do I need to use LUA_ENVIRONINDEX, or LUA_GLOBALSINDEX? Or do I need to do something else? Is this test going to work in absolutely any scenario?

siffiejoe siffiejoe · Accepted Answer · 2014-05-26T19:42:57

Usually you will encounter either the global table _G or the package table as default environment for userdata depending on the function environment of the C function calling lua_newuserdata (see here). Userdata in extension modules loaded via require usually have the package table as default environment, because require has that table set as environment, and it is inherited by all registered C functions and the userdatas created within them. newproxy on the other hand sets _G as default environment.

In principle you could get any table as default environment if someone changes the function environment of C functions (which is pretty rare). In practice it's probably enough to check for _G and the package table. For that you need valid indices for the environment table of your userdata, the package table, and the globals table:

lua_getfenv( L, index ); /* pushes the environment of the ud to stack */
lua_getglobal( L, "package" ); /* pushes the package table to stack */
/* the globals table is always available at LUA_GLOBALSINDEX, so no need to push
 * anything ... */
hasdefaultenv = lua_rawequal( L, -2, -1 ) /* compare env to package table */
             || lua_rawequal( L, -2, LUA_GLOBALSINDEX ); /* compare env to _G */

You probably want to save the package table somewhere (e.g. in an upvalue), so that no-one can mess up your test by replacing the package table.

The most robust solution however would be to set one unique shared dummy table as environment when you create your userdatas (you could use _G for that), and check for and replace that dummy table later if necessary.

Btw., LUA_ENVIRONINDEX is for getting the environment of the currently running C function, not for getting the environment of a userdata.

How to detect if a userdata has "environment table"?

1 Answers