A month or so ago, they covered "The Maw", an indie game created by 7 developers for Xbox Live Arcade. One of the 5 things that went wrong was their Lua and LuaBind implementation.
I thought I'd share it because it's an interesting read, but I'm kinda surprised they thought they could get away with having a separate Lua VM for EVERY object in-game.
Despite the great benefits of using Lua and Luabind in our engine has given us, it took us a few tries to get them working smoothly in our engine. Using Lua and Luabind for all of our gameplay, UI, and generally everything specific to The Maw has been a great help in development. It has helped shorten our iteration time and made it so that some of our designers are really doing the work of programmers. The flexibility and ease of use of Lua combined with the ease of binding C++ classes to Lua using Luabind really put the power of creating the game into the designer's hands. The problems came once we started paying attention to what was happening in memory.
At first, we went with a design that gave each actor its own virtual machine to run Lua. Our plan with this design was to keep each actor in its own isolated environment in order to enforce that all actors would communicate with each other only through our messaging system. This proved to be a mistake because there is a certain amount of overhead with having each VM plus there's the fact that many of the Lua scripts were duplicated across VMs which wasted alot of memory. The other big issue was the performance hit caused by having to run the garbage collector on each VM.
We realized halfway through the project that these issues needed to be addressed so we reworked our scripting system so that all actors and entities in the game used the same VM. This was a great boon to performance and memory usage; however there was still more work to be done. Because we had so many objects in the same VM, it took even longer for the garbage collector to check all of the objects for collection and perform the necessary work to clean up dead objects, not to mention all of the fragmentation caused by all of this.
To address this issue, we did some logging to see what was causing the bulk of the memory allocations and with a fairly small effort we were able to rework our scripts a bit so that many fewer allocations were occurring each frame. We went from about 350-450 allocations made by Lua and Luabind per frame to roughly 0 to 15 depending on what was going on at any given time. Obviously, this is a great improvement, but it still caused some issues.
We still had to run the garbage collector when too much memory was in use by Lua, even though this could easily cause a 40 second or more hang while the garbage collector did it's work. Obviously, this is unacceptable when a user is playing the game. We tried several things to help improve this, finally implementing a solution where we made modifcations to Lua's garbage collector so that it continuously runs on another of the Xbox 360's hardware threads, taking into account thread safety and minimizing the number of required critical sections.
We still had an issue with fragmentation, but roping off an area of memory just for use by Lua safely kept the fragmentation from contaminating the rest of the system.
