Devlog #009: Scripting System Progress Report

Introduction

After the previous devlog, I decided to take a break from developing Solar Lander to work on my other two projects. Solar Lander's development is being juggled between those other two projects. This break was about 3 months in length. And afterwards, I got back to working on Solar Lander again.

Just like before the break, I was initially stuck on a specific issue pertaining to mod validation. Actually, several issues. One of which was with edge cases surrounding Json files including other Json files. Another case was with included Json files sometimes not being read when instantiating spacecraft. A third was validating Lua scripts (more on that later). And the fourth issue being how I was going to properly sandbox the Lua environment.

The Mod Loading Process

Solar Lander will start by populating a list of every mod it can find, as well as a list of dependencies for each mod. It will then validate the dependency tree, and remove any mod from the list that has a missing or circular dependency. Then it will do a partial load of all of the mods that are left. This partial load will validate all of the mod's contents (or as much as it can). This mostly involves checking all of the references in all of the Json files. Missing or circular references will result in the mod failing validation and being removed from the list (which fails all of the mods in the dependency chain). Part of this check is making sure that the Lua scripts don't have any syntax errors.

Once a mod and all of its dependencies have passed validation, the mod can then be fully loaded for use within the game.

Script Validation

Validating models, audio, and Json files turned out to be trivially easy compared to validating Lua scripts. In fact, beyond simply using Lua's built-in syntax validation, validating Lua scripts to the full extent that I want is pretty much impossible to do with static analysis. At least, it goes beyond my current skillset. So what I will do instead is write some functions that, when called, will do the rest of the validation when the script is executed. If a script does something to trip these functions, then the script will immediately be terminated, and the mod will be flagged as having failed validation, and remove it from the list of available mods. Then the game will go through the entire mod list and revalidate all of the remaining mods, deactivating and removing mods that now have a missing reference.

As for the mod with the bad script, the game will remember that this mod is supposed to fail validation by adding a file somewhere (haven't decided where yet) giving information on what script tripped the validation functions and why the validation failed. It's not a perfect solution, but until I can figure-out a foolproof solution that does not require running the script to trigger validation failure, this is the best that I can do.

Script Sandboxing

The default lua.exe file runs in a non-sandboxed environment. This means that if you are not careful, it can cause very serious (and potentially permanent) damage to your system. This obviously won't fly in a game such as Solar Lander. So when Solar Lander decides that it's time to run a script, it will first set up a highly-restricted environment, then run the script.

Most of Lua's base library will be available, with notable exceptions being the [c]rawX()[/c], [c]loadX()[/c], functions being removed (among others). Other functions, such as [c]print()[/c], [c]tostring()[/c], and [c]require()[/c] will be modified to suit the needs of Solar Lander, and to strengthen the security of the sandboxed environment. For example, [c]tostring()[/c] will not print memory addresses and [c]print()[/c] will output to the Unity log. [c]require()[/c] will be restricted to referencing scripts either in the same mod as the script, or in a dependency mod (but not a dependency of a dependency). More importantly, [c]require()[/c] is one of those functions that can trip validation failure for a missing or circular reference.

For the standard libraries, you will only get access to the "math", "string", and "table" libraries. This should be sufficient for anything that you might want to do. You will not be getting access to "utf8" or "coroutine", simply because I don't think you'll need it. And for security reasons, you will also not be getting access to the "io", "os", "package", or "debug" libraries. I will probably be adding some game-specific libraries that any script can use, but I don't yet know what these will be.

A few other things that might be important is that the global environment is going to be pretty much locked down. Scripts will not be able to modify any of the default entries in the global environment. If you try, then the best case-scenario is that you simply hide part of the global environment, which can be unhidden by setting [c]field = nil[/c]. In the worst-case scenario, Lua will throw an error that terminates the script.

Each script in Solar Lander will have its own environment to make sure that scripts cannot interfere with each other. Different types of scripts will have different environments. For example, game mode scripts will basically have full control over the in-game environment, but flight computer programs will only have limited control over the craft that they are attached to. So while a game mode script can spawn and despawn crafts at will, teleport them, and directly manipulate their state, a flight computer program will have to send commands to the thrusters and engines to move the craft into the desired position.

Final Notes

Currently, I'm trying to decide how I will keep scripts isolated while also allowing [c]require()[/c] to work. And I'm also finalizing the protection of the global environment so that it cannot altered or lost. Once I get those details worked-out, I'll start working on the game's scripting API.