Compiling Lua lib for Android - success, but strange segfaults

Question

Sorry for long question. If you want, skip part about compiling Lua (which almost OK) and get straight to last question.

Let's compile Lua library like a static library for Android.

Download latest source and look into doc/readme.html - Building Lua on other systems section for list of files to compile.

And of course look into makefiles - see what in casual way we must set platform flag such is linux, bsd e.t.c. But of course there is no Android platform, so we have choice to set platform to ANSI, Linux, Posix or Generic.

First question: it builds ok (with one exception about llex.c which i will describe down below) even without any platform flag, so maybe this unnecessary?

I set ANSI flag.

Android.mk

LOCAL_PATH := $(call my-dir)
include $(CLEAR_VARS)
LOCAL_MODULE    := lua
LOCAL_CFLAGS    := -DLUA_ANSI
LOCAL_SRC_FILES := lapi.c lcode.c lctype.c ldebug.c ldo.c ldump.c lfunc.c lgc.c llex.c lmem.c lobject.c lopcodes.c lparser.c lstate.c lstring.c ltable.c ltm.c lundump.c lvm.c lzio.c lauxlib.c lbaselib.c lbitlib.c lcorolib.c ldblib.c liolib.c lmathlib.c loslib.c lstrlib.c ltablib.c loadlib.c linit.c
include $(BUILD_STATIC_LIBRARY)

Application.mk

APP_MODULES := lua
APP_PLATFORM := android-8
APP_OPTIM   := release
APP_ABI := armeabi

And got errors of course

Compile thumb  : lua <= llex.c
jni/llex.c: In function 'trydecpoint':
jni/llex.c:214:18: error: 'struct lconv' has no member named 'decimal_point'


#if !defined(getlocaledecpoint)
#define getlocaledecpoint() (localeconv()->decimal_point[0]) //Missing struct member
#endif

Fixing it in the most cheap way

#if !defined(getlocaledecpoint)
#define getlocaledecpoint() ('.') //Code-monkey style
#endif

There are some limitations about locale.h in Android NDK, so this error is not what surprising.

Also got errors about size_t, UCHAR_MAX, INT_MAX - adding llimits.h include into llex.c and all errors are gone now.

Only warnings now exist about "missing break at the end of case" and "no return in function returning non-void" in static int llex, but we don't mess with Lua source code no more because it's not what vital.

Second Question: am i going to programmer hell for such quick fixes?

Grab our fresh baked LuaLib in obj/armeabi directory and lets test it. Of course to load scripts from android file system, we need write some file loader with use of AssetManager class in Java, so let's do it far simple by pushing and reading lua global vars.

TestLua - Android.mk

LOCAL_PATH := $(call my-dir)

include $(CLEAR_VARS)
LOCAL_MODULE    := lua
LOCAL_SRC_FILES := liblua.a
LOCAL_EXPORT_C_INCLUDES := $(LOCAL_PATH)/lua-inc //Where .h files from lua src stored
include $(PREBUILT_STATIC_LIBRARY)

include $(CLEAR_VARS)
LOCAL_MODULE    := LuaLibTest
LOCAL_STATIC_LIBRARIES:= lua
LOCAL_SRC_FILES := LuaLibTest.c
LOCAL_LDLIBS    := -llog
include $(BUILD_SHARED_LIBRARY)

LuaLibTest.c

#include "LuaLibTest.h"
#include "lua-inc/lua.h"
#include "lua-inc/lauxlib.h"
#include <android/log.h>

#define INFO_TAG "[INFO]"
#define LOGI(...) __android_log_print(ANDROID_LOG_INFO, INFO_TAG, __VA_ARGS__)

 JNIEXPORT void JNICALL Java_com_lualib_test_NativeLib_testLua(JNIEnv* env, jclass _class)
{
 LOGI("HI FROM C");
 lua_State* L = luaL_newstate();
 luaL_openlibs(L);
 lua_pushstring(L, "Some string from Android C" );
 lua_setglobal(L, "TEST" );
 lua_getglobal(L, "TEST" );
 const char* res = lua_tostring(L, lua_gettop(L));
 LOGI("LUA TEST VAL: %s", res);
 lua_pop(L, 1);
 lua_close(L);
}

It also works if we read script file with AssetManager and put it contents in to string which feed into lual_dostring(). So yes, we build lua for Android (except lua i/o functions, which will not work, because stdio like printf don't work in Android NDK).

However this build have strange errors, for example - fps script update every frame, but it can fall at any moment with next error at function what updates fps with frame delta time

FPS.lua

FPS = {}

function initFPS()
FPS.fps = 0
FPS.last_fps = 0
FPS.frames_count = 0
FPS.frames_time = 0.0
local fps_msg = "FPS: " .. FPS.fps
c_set_fps(fps_msg);//Set some label in app - c function
end

function updateFPS(frameDeltaTime)
FPS.frames_count = FPS.frames_count + 1
FPS.frames_time = FPS.frames_time + frameDeltaTime
if FPS.frames_time >= 1000.0
then
    FPS.frames_time = 0.0;
    FPS.fps = FPS.frames_count;
    FPS.frames_count = 0;
    if FPS.last_fps ~= FPS.fps
    then
        local fps_msg = "FPS: " .. FPS.fps
        c_set_fps(fps_msg);
        FPS.last_fps = FPS.fps
    end
end
end

FPS.c

 void update_fps(const double* frame_delta_time) //SEGFAULT at this, at random time
 {
  lua_State* l = get_lua();
  lua_getglobal(l, "updateFPS");
  lua_pushnumber(l, *frame_delta_time);
  lua_call(l, 1, 0);
 }

And get next error message with whole app crash at random time (1 min - 3 min)

Last question (yay, you made it/skip boring part)
Why I get segfaults, why at random time? FPS script is just example, at most my every lua script has chance to crash whole app (more calls==better chance). So some player script which change its dir at new pos crash sometimes too.

I think it's because conflict of Android/Java garbage cleaner and Lua garbage cleaner, so something try to free already freed memory.

EDIT - FROM THERE DOUBLE POINTER COME AND WHY:

 #define MS_1_SEC 1000.0

 typedef struct time_manager
 {
   double _time;
   double delta_time;
 }time_manager;

 static double get_ms(s_time* time)//get time in ms
 {
  return MS_1_SEC * time->tv_sec + (double) time->tv_nsec / NS_1_SEC;
 }

 double get_time_now()
 {
 s_time time_now;
 clock_gettime(CLOCK_REALTIME, &time_now);
 return get_ms(&time_now);
 }

 void init_time_manager(time_manager* tm)
 {
 tm->_time = get_time_now();
 tm->delta_time = 0.0;
}

void update_time_manager(time_manager* tm)
{
 double time_now = get_time_now();
 tm->delta_time = time_now - tm->_time;
 tm->_time = time_now;
}


static time_manager TM;//Global static var for whole render module

In onInit() function

 init_time_manager(&TM);

In onDraw() function

 double* frame_time = &TM.delta_time;//get pointer to delta time
 update_ui(frame_time);//Pass it every function
 update_sprites(frame_time);
 update_fps(frame_time);
 ...
 draw_fps();
 update_time_manager(&TM);

Why i use pointer to double instead of just double? Well it saves 4 bytes of copying (every pointer has size of 4, double has size of 8) frame_delta_time param to every function like update_ui(), i do the same for every struct/type bigger than 4 bytes, const pointers instead of just struct x for read-only access. Is this bad thing?

Answer 1

Your changes look OK and appropriate for your system. The lua mailing list suggests that you make the changes to luaconf.h rather than llex.c ( http://lua-users.org/lists/lua-l/2012-08/msg00100.html ) but it shouldn't matter much. (So in short you're not going to hell for these changes ... ).

I'm guessing that the segfaults are happening because some of your C-lua bridge is doing something "bad". My guess would be some kind of lua stack over/under flow, or accessing outside the lua stack by using an invalid index. You may be able to track this down if you can build the bridge part on linux/os x and use valgrind. (Similar tools exist for windows too, but I'm not sure about native to android)

Answer 2

Look at this: http://comments.gmane.org/gmane.comp.security.nmap.devel/14966

static void trydecpoint (LexState *ls, SemInfo *seminfo) {
  char old = ls->decpoint;
  ls->decpoint = '.';   //ls->decpoint = getlocaledecpoint();  // try to fix error: 'struct lconv' has no member named 'decimal_point'   -------- look at here
  buffreplace(ls, old, ls->decpoint);  /* try new decimal separator */
  if (!buff2d(ls->buff, &seminfo->r)) {
    /* format error with correct decimal point: no more options */
    buffreplace(ls, ls->decpoint, '.');  /* undo change (for error message) */
    lexerror(ls, "malformed number", TK_NUMBER);
  }
}

Answer 3

Sometimes it helps to compile lua with -DLUA_USE_APICHECK. It will generate some error messages instead of segfaults.