C / C ++ Stacked Lua Session

Question

I need to safely start a Lua session in an embedded multi-threaded environment where the stream stack is pre-allocated and has a fixed size.

Allowed to have the script failed at the Lua level due to the high consumption of the C / C ++ stack.
This did not allow the entire application to crash at the C / C ++ level.
I cannot rely on the checks provided by LUAI_MAXCCALLS and MAXCCALLS.

What is the right way to check and prevent any potential stack overflows in a Lua session?
What is the right place to implement such a check?

Answer 1

You can wrap C functions for calling inside external C-closure by checking your C / C ++ stack. The closure will contain an upvalue referring to a valid C function that must be called:

CFunctionThunk(lua_state* l) { Check available stack space. if( available stack space < minimum stack space required ) { panic(...); } else { actualFunction = retrieve upvalue; actualFunction(l); } } 

This will add the thunk stack space needed for the stack space needed for the function to be called, but it should be small and constant. In addition, before calling Lua from the very beginning, you must make sure that the remaining C-stack space is at least large enough to contain your Lua VM and two frames of the C-function stack (thunk + actual). You can examine the stack pointer in the debugger to find the amount of space needed for thunk and Lua VM, which should be (more or less) constant. Checking the stack space can also be reduced to a local thread counter, which increases at the beginning of thunk and decreases at its end.

Another possibility is to enable Lua debugging on function calls, check C-functions there and, if necessary, check C-stacks. But this is likely to degrade runtime performance, as the hook will also be called for Lua-specific functions in your script.