Is it possible to call loadstring on a lua bytecode line that contains a link to a C function?

Question

We use the Love2d Lua engine, which provides api graphics for Lua. We are trying to serialize a giant hash table containing all the saved game data for the game world. This hash includes some functions, and some of these functions call Love2d C.

To serialize functions in a hash, we use string.dump and load them using loadstring. This works well for pure Lua functions, but when we try to serialize and then load it back into a function that calls a wrapped C function, such as one in Love2d api, loadstring returns nil.

Consider the following simple program that draws “hello world” on the screen using the Love2d graphics engine:

function love.load() draw = function() love.graphics.print('hello, world', 10, 10) end end function love.draw() draw() end 

We would like to be able to do this:

 function love.load() draw_before_serialize = function() love.graphics.print('hello, world', 10, 10) end out = io.open("serialized.lua", "wb") out:write('draw = load([[' .. string.dump(draw_before_serialize) .. ']])') out:close() require "serialized" end function love.draw() draw() end 

This operation is written to a Lua file on disk containing a combination of the Lua and Lua uncompiled bytecodes, which looks something like this:

 draw = load([[^[LJ^A^@ @main.lua2^@^@^B^@^B^@^D^E^B^B4^@^@^@%^A^A^@>^@^B^AG^@^A^@^Qhello, world print^A^A^A^B^@^@]]) 

This method works great with Lua functions that do not call C modules. We think this is a problem because this example works:

 function love.load() draw_before_serialize = function() print('hello, world') end out = io.open("serialized.lua", "wb") out:write('draw = load([[' .. string.dump(draw_before_serialize) .. ']])') out:close() require "serialized" end function love.draw() draw() end 

Instead of calling the graphical method Love2d, it performs printing on the console.

After more testing, we were confused by the fact that this example works:

 function love.load() draw_before_serialize = function() love.graphics.print('hello, world', 10, 10) end draw = load(string.dump(draw_before_serialize)) end function love.draw() draw() end 

Here we do not actually write the function to disk, but instead simply unload it and then immediately load it back. We thought that perhaps the culprit did not write data with the binary write mode flag ( "wb" ), but since we are on Linux, this flag has no effect.

Any ideas?

Answer 1

I think the problem is string formatting. Nicole Bolas may be right about the [[]] quotes associated with the bytecode dump, but this indicates a big problem; The bytecode can really be anything, but you see it as a regular line that can be written to and read from a text file. This problem is demonstrated by your last demo, where you load a discarded string without writing it to a file.

This is a serializer implementation for tables that include functions like what you want, I think, but I also think it broke (well, I couldn’t let this work anyway ...). In any case, this is on the right track. You need to format the bytecode and then write it to a file.

I am sure there is a better way to do this, but this works:

 1. binary = string.dump(some_function) 2. formatted_binary = "" 3. for i = 1, string.len(binary) do 4. dec, _ = ("\\%3d"):format(binary:sub(i, i):byte()):gsub(' ', '0') 5. formatted_binary = formatted_binary .. dec 6. end 

This loop goes through each character in the bytecode, formats them as escaped bytes (each line contains a code like "\ 097", which after interpolation will escape to "a").

Line 4 of this pattern is tight, so I will break it. At first,

 binary:sub(i, i) 

pulls the i-th character from the string. Then

 binary:sub(i, i):byte() 

returns the integer representation of the ascii of the i-th character. Then we format it using

 ("\\%3d"):format(binary:sub(i, i):byte()) 

which gives us a string like "\ 97", for example, if the character is "a". But this will not disappear properly, because we need “\ 097”, so we will replace gsub “with“ 0. "Gsub returns the result string and the number of substitutions performed, so we just take the first return value and put it in" dec ". I'm not sure why the format "% 3d" does not replace the spaces "0" by default ... oh well.

Then, to execute the formatted binary string, we need to avoid it and pass the result to "load". The quotation marks from [[]] in Lua do not escape flows, like "... I'm actually not sure that they have no escape at all. So, to create an Lua executable string that will return a function that will do what is in "some_function", we do the following:

 executable_string = 'load("' .. formatted_binary .. '")' 

Good - therefore, putting all this together, I think that we can make your test script work like this:

  1 function love.load() 2 draw_before_serialize = function() 3 love.graphics.print('hello, world', 10, 10) 4 end 5 6 binary = string.dump(draw_before_serialize) 7 formatted_binary = "" 8 for i = 1, string.len(binary) do 9 dec, _ = ("\\%3d"):format(binary:sub(i, i):byte()):gsub(' ', '0') 10 formatted_binary = formatted_binary .. dec 11 end 12 13 out = io.open("serialized.lua", "wb") 14 out:write('draw = load("' .. formatted_binary .. '")') 15 out:close() 16 17 require "serialized" 18 end 19 function love.draw() 20 draw() 21 end 

When I run this with Love, I get an OpenGL screen with a "hello world" printed in the corner. The resulting file "serialized.lua" contains the following:

 draw = load("\027\076\074\001\000\009\064\109\097\105\110\046\108\117\097\084\000\000\004\000\004\000\008\009\002\002\052\000\000\000\055\000\001\000\055\000\002\000\037\001\003\000\039\002\010\000\039\003\010\000\062\000\004\001\071\000\001\000\017\104\101\108\108\111\044\032\119\111\114\108\100\010\112\114\105\110\116\013\103\114\097\112\104\105\099\115\009\108\111\118\101\001\001\001\001\001\001\001\002\000\000")