June 12, 2016


#1

We nudge ever closer to the public beta. This week has been about the dev tools, working on compatibility of save files, optimisation, and world feature concepts. This devlog I’ll be talking about the Editor and Serialisation.

Editor

I made some huge improvements to the internal editor – this is the tool I use to test things and prototype little areas. It lets you add blocks, objects, mobs, etc. I spent a few days making it more user-friendly so that Alex and I can more easily create those world features I talked about last week.

Here’s the block panel, which shows you all the blocks and offers a paint and fill tool. (It uses Dear ImGui for the GUI.) There’s a “filter” text box which lets you navigate the list more easily. The fill tool and bigger brush sizes are especially handy.

The “object mode” lets you select an object from the list and then click into the world to add it. The editor also tries to snap the object to the nearest valid point and place it. Similar modes exist for mobs and items, too.

At the moment these tools are developer only, but we’re having so much fun with them that I’m now seriously considering bundling them with the final game. It would probably be a feature that you’d have to unlock somehow, and to prevent spoilers it would only contain a subset of the content. Imagine creating a little level and then sharing it with your friends to play, I think that could be a lot of fun. This is a low-priority feature for now.

Serialisation

Not the most fun of topics but I think it’s an important one to mention. The game is currently very fragile in terms of save files: you can’t load games from different versions, and if something goes wrong with loading the game may just crash. I aimed to fix both of those this week. As we move into a content-creation phase of development we needed a stable save file system that is backwards and forwards compatible.

There are a few issues involved with compatibility:

  • File format
  • System architecture differences
  • Missing resources
  • Unknown/New resources

File Format: Consider that I want to add a new field “pants” to an object which I’ve saved. If your save file version is the same as your game version then this is all good, the game knows about pants and can read in that struct no problem.

struct DudeV1 { int x, y; }

struct DudeV2 { int x, y; bool pants; }

But what if you have V2 of the game and want to load a V1 save file (for instance if an update comes over steam and you want to continue playing your current game)? More rarely, what if you have V1 of the game and want to open a V2 save file? (This may happen if someone makes a level and sends it to you, but you don’t have the exactly-right-version of the game.)

One solution to this is to use a generic format like JSON/XML, and then read or write as much as you want – setting defaults when things are missing. E.g., if V2 reads in a V1 it may just set pants = true because that field would be missing in the V1 save. As long as you are strict about not removing any elements then all will be good.

If you’re working in a binary format, as I am, then you need to be a little more careful. If you’re writing out the Dude struct to a binary file, you’ll need to also write out the sizeof(Dude) (see below) and the version of Dude. This allows V1 to skip unknown V2 data, and V2 to only read in as much V1 data as it should.

I built a little system based around libs like Cereal and Boost::Serialisation to do this in Moonman. The Dude example looks like this in the engine:

However there will always be occasional big structural changes to the save system, and for those I will break compatibility. The save will have a structural “VERSION” number, which is changed only if the save format breaks compatibility. I expect this will happen rarely, but when it does you’ll lose your save files. A workaround for this is to provide a ‘save game converter’ function to map between old and new saves, but I don’t think this’ll be necessary given the nature of the game.

System architecture differences: Low-level languages are very susceptible to changes in architecture. If I write something like file.write(x); then that will write the exact bits of x as stored on the current system. Some typical example of architectural differences are little endian/big endian and 32-bit/64-bit. Floating point numbers may also be stored differently.

In C++ we also have a bigger problem, certain types or structs may have different sizes depending on which compiler and system you use. You write out a flat struct on MSVC and read it in on OSX? This may not work. To get around this you can use #pragma pack for structs and then sizeof() will be consistent, but this brings it’s own challenges, and so the most straightforward way is to just serialise all the members of a struct explicitly (as I did in the Dude example above). There are other approaches (e.g., Protobuf) but this is the method I’m using in Moonman.

For differences in type size I’m just restricting my serialisation to types with a constant size, e.g., int32. And for differences in endian-ness, well, this is where I’ve drawn the line – in the rare situation of a Big Endian port of Moonman comes up (Nintendo Wii?!), save files will simple be incompatible between endian-ness.

Missing resources: Assume you load a V1 save in a V2 game, but in V2 I’ve removed hats. Or I’ve removed the soil tile. Or something. Previously the system would have crashed because it didn’t recognise the old hat or soil. Now it happily continues on, skipping over the unknown data. If you were wearing a hat it won’t be there anymore - but that’s the worst case.

For blocks we also have another problem, the IDs of blocks may change between version. A stone block may have ID=3 in V1, but ID=4 in V2. This is due to how I generate the ids. Previously the game would have seen ID=3 and used it’s own mapping to determine the block (maybe wood has ID=3 in V2 and then all the stone would become wood!). Now I store a block database with the world itself and use this to maintain a compatible mapping.

Unknown/New resources:
Assume you load a V2 save in a V1 game, the save file may have kinds of unknown objects, items and blocks. The game now reports a warning but can skip over all the unknown data and load the level anyway. This is a rare situation, and if you got that warning and really wanted to play that level, then you’d probably want to update your game version anyway.

As always, I’m happy to answer any questions about this stuff. You can also subscribe to this subforum to get the devlog updates.


#2

I’m praying that the editor makes it into the final game. I can only IMAGINE the possibilities… imagine :moonman_squid:


#3

The save file type in my game is pretty similar, but I handle the versioning of specific entities a little differently. I personally like it a lot, but it has its own downfalls compared to Moonman’s.

Object IDs
I also store a list of all the object IDs, however for my game I store the ID-Object pairs for every individual save game. This takes up a little extra storage space, but the nature of my game allows players to add mods to existing save games. Different mods added at different times in different orders would cause different ID lists to be generated so each save file has to store it.

When the game first loads, it loads in the ID-object pairs, then loads in the raw binary data now that it knows what each ID represents. If, in an update, an object is removed from the game or a mod, the ID-object pair is changed to ID-removed and each time this ID shows up in the save file it will simply be ignored.

Another down-side to this is the ID-object pairs have to be sent to each client when joining a multiplayer game, but I think the trade-off will be worth it! (And now that I’m reading your post again I think this is how you’re doing it as well, so… cool!)

Attributes

This is the main difference in my system between other ones. This could be an awful way to go about things, but it’s worked fine for me so far and I like to try things out on my own! Each object is defined by its ID (a 16 bit integer), a 2d position (each 16 bit ints), and a list of attributes. Each entity is defined by a list of attributes, such as the pants in V2 of your dude.

Each entity is allowed 256 attributes, giving each one a single byte position. Every attribute knows how to write itself to a file as well, so a StringAttribute would write out the string while an IntAttribute would write out its integer value. In the case of my game, I would simply add the “pants” attribute to the “Dude” entity. When reading in the save files, that attribute would just get set to the default value if it is not found. The stream for each entity then looks like:
-(ID)-(x position)-(y position)-(number of attributes)-(ID of first attribute)-(value of first attribute)-(ID of second attribute)-…

The nice part about this system is I can add and remove attributes from entities with complete forward and backward compatibility with the saves (as long as I don’t reuse attribute IDs). I can also set specific attributes that do not get sent to clients in multiplayer games (to help combat cheating).
In multiplayer games, I can also send simple messages like “Change the value of this attribute on this entity to this value” to change the state of objects. In the in-game editor I provide people who run servers I can show a GUI that lists all the attributes and their values for the server-runner to change at will.

I’m basically just a fan of making things as easy as possible for myself xD

Anyways, fabulous work on your game so far! I’ve been following it for quite a while as I slowly hack away at my own game. If you have any other questions about how it works I’m happy to discuss it (I’m a little desperate for some game development friends, all mine are snobby software engineers :stuck_out_tongue:)


#4

Thanks for posting. Sounds like a straightforward system to me! Luckily I don’t have to deal with multiplayer or mods, they add a whole layer of complexity. :smile:

Btw, what game are you making? And do you have a Twitter we can follow? (I’m not sure if I know you already, sorry!)


#5

Yeah, it would be cool if you could download maps from friends or from the steam workshop.


#6

I’m making a game called Questica :slight_smile: my Twitter is @purenickery
I’ve been posting a devlog on TigSource as well if you want to check that out right here!


#7

Awesome, looks like a great project. Good luck! :smiley: