SIDEBAR
»
S
I
D
E
B
A
R
«
Binding C++ to Lua, Part 3; Pushing C++ Objects to Lua
Oct 7th, 2010 by dholm

Part 3 of my Lua C++ binding series has been 95% ready for several months now and I finally got around to finished the last five percent.

In this part I will add code which enables us to return a class instance usable from Lua as a value from a method or function call. By calling the static member push we can put objects onto Lua’s stack (see section 3.1 – The Stack in the Lua reference manual).

First we need to rewrite our garbage collection code so that it is able to differentiate between objects that are managed by Lua and objects that are deleted elsewhere. We will control this by adding a new field to our class instance metatable called “unmanaged“. If the metatable has the unmanaged field we will not delete the C++ object instance when Lua is executing its garbage collector.

static int gcT(lua_State* l) {
    if (luaL_getmetafield(l, 1, "unmanaged")) {
        lua_pushvalue(l, 1);
        lua_gettable(l, -2);
        if (!lua_isnil(l, -1)) {
            return 0;
        }
    }

    Userdata* ud = reinterpret_cast(lua_touserdata(l, 1));
    T* obj = ud->pT;
    delete obj;
    return 0;
}

The allocUserdata allocates room for a light userdata item in the specified table of the specified size and returns a pointer to the data region. Light userdata is a pointer to a C object stored in Lua and we use it to store the pointer to our object instance.

static Userdata* allocUserdata(lua_State* l, Index table, void* key, std::size_t size) {
    Userdata* userdata = NULL;
    lua_pushlightuserdata(l, key);
    lua_gettable(l, table);
    if (lua_isnil(l, -1)) {
        lua_pop(l, 1);
        lua_checkstack(l, 3);
        userdata = reinterpret_cast<Userdata*>(lua_newuserdata(l, size));
        lua_pushlightuserdata(l, key);
        lua_pushvalue(l, -2);
        lua_settable(l, -4);
    }
    return userdata;
}

Weak tables are tables who reference data that is not to be garbace collected by Lua and they are explained in section 2.10.2 of the reference manual. The mode of the weak table defines whether the key, value or both are to be considered weak and protected from collection.

static void weaktable(lua_State* l, const char* mode) {
    lua_newtable(l);
    lua_pushvalue(l, -1);
    lua_setmetatable(l, -2);
    lua_pushliteral(l, "__mode");

The mode field is a string literal whose values are defined in section 2.10.2 of the Lua reference manual.

    lua_pushstring(l, mode);
    lua_settable(l, -3);
}

A subtable to the metatable of our object is used to store the mode setting for garbage collection.

static void subtable(lua_State* l, Index metatable, const char* name, const char* mode) {
    lua_pushstring(l, name);
    lua_gettable(l, metatable);
    if (lua_isnil(l, -1)) {
        lua_pop(l, 1);

If the table hasn’t been defined we create a new one otherwise we just reuse the old one.

        lua_checkstack(l, 3);
        weaktable(l, mode);

Three spaces are allocated on the stack and we put a weak table with the specified mode parameter on the first position.

        lua_pushstring(l, name);
        lua_pushvalue(l, -2);
        lua_settable(l, metatable);

Finally we set metatable[name] = weaktable so that our garbage collection setting is associated with the object instance.

    }
}

Now we have the necessary methods we need so let’s move on to the meat of it all, the push method.

static Index push(lua_State* l, T* instance, bool gc = false) {
    if (!instance) {
        lua_pushnil(l);
        return 0;
    }

First we add a safety measure so that if a NULL instance is pushed we put a nil on Lua’s stack.

    luaL_getmetatable(l, T::s_lunaClassName);
    if (lua_isnil(l, -1)) {
        luaL_error(l, "[Luna::%s] Class %s has not been commited!", __func__, T::s_lunaClassName);
        return 0;
    }
    Index metatable = lua_gettop(l);

Next we attempt to locate the metatable (the implementation) for the class type that we are going to push. If that particular class hasn’t been registered with Lua we raise an error.

    subtable(l, metatable, "userdata", "v");

Since we now have the metatable defining the class API we need to create a subtable to hold the user data (the instance pointer) which we can put on the stack and that makes sense to Lua.

    Userdata* userdata = allocUserdata(l, metatable, instance, sizeof(Userdata));

allocUserdata allocates an entry in metatable[instance] to hold a userdata (see 2.2 – Values and Types) instance for us to store the object instance pointer in.

Now we move on to the final part of the process which is to associate the object instance table with the class definition table and set the appropriate memory management type.

    if (userdata) {
        userdata->pT = instance;
        lua_pushvalue(l, metatable);
        lua_setmetatable(l, -2);

Here we push the class definition (var: metatable) onto the stack and associate the element at stack[-2] (which happens to be our userdata subtable) with it.

        if (!gc) {
            lua_checkstack(l, 3);
            subtable(l, metatable, "unmanaged", "k");
            lua_pushvalue(l, -2)
            lua_pushboolean(l, 1);
            lua_settable(l, -3);
            lua_pop(l, 1);
        }

If we push the object instance with garbage collection disabled we have to add the “unmanaged” subtable that we talked about when we implemented the new garbage collector which acts as a tag to inform it that the object instance should not be automatically garbage collected (i.e. someone has to call delete on it from C++).

    }
    lua_replace(l, metatable);
    lua_settop(l, metatable);
    return metatable;
}

Finally we replace the old metatable with our modified one containing the object instance, reset the stack and then returns the index of our metatable so that Lua knows where we put the object instance.

Microsoft .net vs Java Trailer
Oct 3rd, 2010 by dholm


A really funny norwegian Java trailer.

A 16-bit ALU implemented in Minecraft
Sep 29th, 2010 by dholm

httpv://www.youtube.com/watch?v=LGkkyKZVzug

This is an amazing video demonstrating a fully working 16-bit ALU implemented in the seminal online game Minecraft.

Binding C++ to Lua, Part 2; Binding Our First Class
May 23rd, 2010 by dholm

This part will be rather short and summarize part one by providing a very simple sample using our Luna implementation.
Let’s begin by creating a very simple class which we call Test along with a Lua binding LuaTest which will be the intermediate between C++ and Lua.

#include <iostream>

#include "Luna.hpp"

class Test {
public:
    Test(int x) : m_x(x) { }
    int getValue() const { return m_x; }
private:
    int m_x;
};

The test class is a simple getter that provides a method, getValue, which will return the value set in the constructor.
In order to make this class available from Lua we need to write a binder class which translates to and from C++ and Lua code, called LuaTest for the sake of simplicity.

class LuaTest {
public:
    LUNA_TYPES(LuaTest);

LUNA_TYPES(LuaTest); is a simple macro which injects the class with a couple of placeholder variables used by Luna.

LuaTest(lua_State* l) : m_test(luaL_checknumber(l, 1)) { }
    LuaTest(int x) : m_test(x) { }

We need to set up two constructors, one for Lua and one for C++ which emulates the same functionality as the C++ version. The C++ constructor for LuaTest is pretty straightforward but the Lua version is a bit more tricky.
All Lua calls only take a pointer to the actual Lua state since all values are pushed and pulled onto the stack. in this case we use luaL_checknumber(l, 1). This function call will return the first element on the top of the stack assuming it’s a number otherwise it will trigger an error.

int getValue(lua_State* l) {
    lua_pushnumber(l, static_cast<double>(m_test.getValue()));
    return 1;
}

This time we use lua_pushnumber(l, static_cast<double>(m_test.getValue())); to get the value from the stored Test object instance. Not that all numbers in Lua are in double so we need to cast them when expecting something else.

private:
    Test m_test;
};

LUNA_CLASS(LuaTest, Test, Test) = {
    LUNA_METHOD(LuaTest, getValue),
    {0}
};

The LUNA_CLASS macro is used to register the class along with exposing methods calls, if any, the list has to be terminated with a {0}. The first argument to LUNA_CLASS is the name of the binding class, the second the actual C++ class name and the last the identifier to be used from Lua scripts. This way you can for instance bind a boost class to Lua like: LUNA_CLASS(LuaDate, boost::gregorian::date, Date)

int main(int argc, char** argv)
{
    lua_State* l = lua_open();
    luaL_openlibs(l);
    Luna<LuaTest>::commit(l);
    luaL_dofile(l, argv[1]);
    lua_close(l);
    return 0;
}

We create a very simple main function, compile it and pass it the following Lua code:

print("begin")
test = Test(1)
print("Test=" .. test:getValue())
print("end")

The resulting output is left as an exercise to the reader.

In the next installment I will move on to some more advanced features such as instantiating specific C++ classes from within Lua scripts.

Binding C++ to Lua, Part 1; The Basics
May 8th, 2010 by dholm

I have been working on binding C++ to Lua and found that even though there are some good samples available on the lua-users wiki none of them really offers a complete or clean implementation except for Luabind which requires a lot of Boost components which might be unsuitable for embedded systems. Since I’ve never worked with Lua before this proved to be a good project to learn the rules and inner workings of the language.

If you haven’t worked with Lua before please read chapter 3 of the Lua 5.1 Reference Manual which provides a good introduction to the Lua C API. It’s important to understand that Lua is stack based and you work by pushing and popping values on and off the stack. In this first part I will show how to register a C++ class with Lua so that it can be instantiated and used from within a Lua script. This part is already well covered in LunaWrapper but I will attempt to break it down and explain each part.

Think of a Lua table as a sort of std::map to use C++ jargon. Class definitions in Lua live inside what is known as a metatable. The metatable is a table for which it is possible to modify the behavior of certain operations such as common binary operators and indexing. We need to use the metatable in order to let Lua know how to create and destroy C++ objects as well as binding object instances to method calls.

Enough talking, let’s have a look at some code. I will start out with how to set up the metatable and register your C++ class with it.

lua_newtable(l);
int methods = lua_gettop(l);

Create a new standard table in which we will store our class’ methods and save the index to it. Remember that Lua is stack based and we need to remember the indices of created variables if we need to reference them later. lua_gettop will return the absolute index of the item at the top of the stack, in this case the newly created table.

luaL_newmetatable(l, T::s_className);
int metatable = lua_gettop(l);

Create the metatable for our class and name if after our class (T::s_className).

lua_pushstring(l, T::s_className);
lua_pushvalue(l, methods);
lua_settable(l, LUA_GLOBALSINDEX);

Lua has three types of environments into which you can put values. We will put our our class metatables into the global environment in order to make the type available to all parts of our scripts. You can read about environments in chapter 2.9 of the reference manual.

We push the class name onto the stack followed by the index of our table of methods, which we will fill later. By calling lua_settable we tell Lua to store the method table in the global environment at the key “T::s_className“. In C++-speak this would be equal to having a std::map GLOBALSINDEX and doing GLOBALSINDEX[T::s_className] = methodArray;

lua_pushliteral(l, "__metatable");
lua_pushvalue(l, methods);
lua_settable(l, metatable);

Sets the metatable at the metatable stack index as the metatable of our method table. Meaning that Lua will now look for meta functions in “metatable” when calling methods from our method table.

lua_pushliteral(l, "__index");
lua_pushvalue(l, methods);
lua_settable(l, metatable);

Set up the indexing method of our metatable to get its values from the methods table. This will allow Lua to look for methods in the methods table whenever a call is made to the metatable (the class).

lua_pushliteral(l, "__gc");
lua_pushcfunction(l, gcT);
lua_settable(l, metatable);

Register a garbage collector for the metatable. Here we use lua_pushcfunction to let Lua know that whenever an instance of our class’ goes out of scope the function gcT should be called to clean up the object. This will allow us to execute our objects destructor and avoid any nasty resource leaks.

lua_newtable(l);
int methodTable = lua_gettop(l);
lua_pushliteral(l, "__call");
lua_pushcfunction(l, newT);
lua_pushliteral(l, "new");
lua_pushvalue(l, -2);
lua_settable(l, methods);
lua_settable(l, methodTable);
lua_setmetatable(l, methods);

In order for us to be able to create class instances in Lua we have to let Lua know how to instantiate a class. That is done by registering a C function, newT, which is called whenever someone performs a “function call” using our class name. You would recognize this as T::s_className().

Create a method table methodTable, set the function call to dispatch to the C function newT, which we will define later, and link it to the metatable of our table of methods.

In order to link methods names with the actual methods we need to set up a type to map them. Lets create a struct containing the name and a method type.

typedef int (T::*Method)(lua_State* l);

typedef struct {
    const char* name;
    Method method;
} MethodMap;

Note that we cannot bind our regular C++ classes directly to Lua. We will need an intermediate since it has to be able to marshall data to and from Lua. Since all arguments are marshalled each method call only needs to know which Lua state to push and pop data to. Our method calls will therefore only need a pointer to the Lua state and return the number of values it has pushed onto the stack.

for (MethodMap* m = classMethods; m->name; ++m) {
    lua_pushstring(l, m->name);
    lua_pushlightuserdata(l, reinterpret_cast<void*>(m));
    lua_pushcclosure(l, thunk, 1);
    lua_settable(l, methods);
}

For each method of our class we have to put the method map into the methods table together with a thunk as a C closure so that the object instance of the callee is passed to the method.

lua_pop(l, 2);

Finally pop the methods and class metatable off the stack and we’re done!

Ok, this wasn’t all that hard as long as you understand the stack concept. But we are missing a couple of things, the constructor, destructor and thunk so let’s move on and define them as well.

We need a type to store the pointer to our class instances in. Using Lua-speak we call them user data so we define a struct Userdata for them.

typedef struct { T* pT; } Userdata;

Now we need a “constructor” that creates instances of our objects.

int newT(lua_State* l) {
    lua_remove(l, 1);
    T* obj = new T(l);
    Userdata* ud = reinterpret_cast<Userdata*>(lua_newuserdata(l, sizeof(Userdata)));
    ud->pT = obj;
    luaL_getmetatable(l, T::s_className);
    lua_setmetatable(l, -2);
    return 1;
}

Remove the item at the top of the stack which is the name of the method call, “new“, which we already know since we have linked this specific function to that call. Create a new object of type T followed by a userdata entry in Lua into which we will put the pointer to our object. Get the metatable associated with the class name of T and connect it with our userdata so that Lua will use the data from our class metatable when working with our object instance. Return one to let Lua know that we put one item at the top of the stack, our object instance.

static int gcT(lua_State* l) {
    Userdata* ud = reinterpret_cast<Userdata*>(lua_touserdata(l, 1));
    T* obj = ud->pT;
    delete obj, obj = NULL;
    return 0;
}

This is the garbage collector for our class type. It will fetch the object instance from the top of the stack and call the C++ delete operator on it. Since we didn’t put anything on the stack this function returns zero.

int thunk(lua_State* l) {
    T* obj = reinterpret_cast<Userdata*>(luaL_checkudata(l, 1, T::s_className))->pT;
    lua_remove(l, 1);
    MethodMap* m = reinterpret_cast<MethodMap*>(lua_touserdata(l, lua_upvalueindex(1)));
    return (obj->*(m->method))(l);
}

Those of you who have ever written object-oriented code in a non object-oriented language will know that the first argument to a “method call” usually is a pointer to the object instance and that is the way it is done in Lua as well. We use luaL_checkudata to get the value at the top of the stack and verify that it is of userdata type. This call will actually return NULL if the item at the top of the stack isn’t of userdata type so we should check the return value but in order to cut down the code for the sake of this post I decided to trim out that particular check. lua_remove will remove the specified number of items from the top of the stack, in this case we remove the userdata from the top so that only the method arguments remain when we make the dispatch.

Since thunk is a C closure we can use lua_upvalueindex to get the method map from the methods table and use that to find out which method was being called. We return the result of the method call which again must be the number of items the method has pushed onto the Lua stack.

This concludes the first part of this series. In the next one I will show you how to use this code to register and use a simple C++ class from within a Lua script.

Luna source code, part 1

My very first Linux kernel patch submission
Jan 19th, 2010 by dholm

I’ve worked on kernel projects before but today marks the day I submitted a patch myself. I cannot take full or even half credit for it because it is a backport of a driver from Linux 2.6 for Sierra Wireless modems.

Even though that might sound simple it still took me quite a bit of work to get it to work properly due to the subtle changes in the underlying USB stack. I had to struggle quite a bit with some nasty instabilities related to interrupts which in the end turned out to be double frees on 2.4 but not on 2.6

My LKML submission is available at: http://marc.info/?l=linux-usb&m=126392883122598&w=2
The patch is available from my site since apparently GMail botched some of the lines: http://www.dholm.com/files/sierra-2.4_1.7.16-1.patch

Apparently 2.4 only accepts stability and security patches these days so I don’t think it will get merged but if someone ever needs it the work has already been done. We are running it on uClinux 2.4.26 but the patch was made against 2.4.37.

Git Grafts
Sep 2nd, 2009 by dholm

If you ever need to graft two git repositories, i.e. you want to restore history from your previous VCS, “Using git to stitch together a broken subversion repository” is a good starting point. Git is amazing when it comes two this task but in my opinion it is poorly documented at best.

AMD demos OpenCL on four six-core Opterons
Aug 10th, 2009 by dholm

Google Wave
Jun 1st, 2009 by dholm

A presentation on Google Wave which is Google’s next gen communications platform.

“The Renaissance of Craftsmanship” by Robert C. Martin
Mar 4th, 2009 by dholm

Uncle Bob” talks about what it means to be a software developer.

SIDEBAR
»
S
I
D
E
B
A
R
«
»  Substance:WordPress   »  Style:Ahren Ahimsa