I'm continuing on with a project I am calling 'TiledStack' which is a set of libraries, web services and clients that assist in the development and running of games that utilise the Tiled map editor (opens new window).

This past week or so I have focused on 'developer workflow'. How developer(s) make changes to a map and how those changes get incorporated into the iteration and feedback loop workflow of the developer(s) involved.

If you've read my previous post on this library, you'll remember that I got simple messaging working between multiple MonoGame (opens new window) clients. This worked by all clients updating their map via messages received from the server and the server repeat each request to each client. I was generating these messages via a bound key, not exactly a real world scenario. What if we wanted to get changes to a map from an arbitrary source just by comparing the original instance and a new instance? Like updates from Tiled map editor itself?

Developer Workflow

Since the Tiled map editor is the common part to every developer using the Tiled map format, I want something that integrates with workflows that developers already have. This functionality originally came out of my own desire to not have to restart/rebuild my own game instance just to see how the game would play/look with the new map layout/map objects/layers/tilesets. This functionality might sound fairly insignificant, "It'll look the same as in the editor, get over it!". However, in my own experience, I found it a great source of motivation to be able to see the new changes running in the actual game, preferably on the actual device being targeted (eg, Android/Windows Phone/PC etc).

To get a simple proof of concept up and running, I'm using a simple console application to watch for file modifications in a specified directory that contains Tiled map json files. After sorting out some (apparently well known) issues with watching for file modifications in .NET using the 'Filewatcher' class, I had to be able to compare the old to the new and pick out the differences in the two.

Modifying Tiled

One of the main problems I came across when building the file watcher client was handling multiple deletes/creates and updates in one file change. This was due to, by default, there are no unique identifiers on arrays or layers, objects or tilesets. I tried quite a few ways of creating an id every time the map was loaded, but in the end it was based off the index number. I ended up getting the latest version of Tiled from github and changing these objects to produce a uuid on creation (Fork here (opens new window)). This has made solving the problem of difference checking much simpler and also gives an opportunity to manage objects differently at runtime. Below is a snippet of how I'm generating the different CRUD lists.

public List<TiledRequestBase> RequestsFromLayers(List<LayerDto> originalLayers, List<LayerDto> newLayers, string mapName)
{
    var result = new List<TiledRequestBase>();

    var ignoredLayers = new List<LayerDto>();
    var ignoredOrginalLayers = new List<LayerDto>();
    var updateOldLayers = new List<LayerDto>();
    var createLayers = new List<LayerDto>();
    var updateLayers = new List<LayerDto>();
    var deleteLayers = new List<LayerDto>();

    if (originalLayers == null && newLayers != null)
    {
        for (int i = 0; i < newLayers.Count; i++)
        {
            createLayers.Add(newLayers[i]);
        }
    }

    if (newLayers == null && originalLayers != null)
    {
        for (int i = 0; i < originalLayers.Count; i++)
        {
            deleteLayers.Add(originalLayers[i]);
        }
    }

    if (newLayers != null && originalLayers != null)
    {
        for (int i = 0; i < originalLayers.Count; i++)
        {
            for (int j = 0; j < newLayers.Count; j++)
            {
                var oldLayer = originalLayers[i];
                var nLayer = newLayers[j];
                if (oldLayer.guid == nLayer.guid && !oldLayer.Equals(nLayer))
                {
                    updateLayers.Add(nLayer);
                    updateOldLayers.Add(oldLayer);
                }

                if (oldLayer.guid == nLayer.guid && oldLayer.Equals(nLayer))
                {
                    ignoredLayers.Add(nLayer);
                    ignoredOrginalLayers.Add(oldLayer);
                }
            }
        }

        // layers that have copies on the originalLayers, refs from new map
        ignoredLayers = new List<LayerDto>(ignoredLayers.Distinct());
        // first get all the non equal layers from newLayers
        createLayers.AddRange(newLayers.Except(ignoredLayers));
        // then filter out ones with the same guid that are for updating -> gives you new to create
        createLayers = new List<LayerDto>(createLayers.Except(updateLayers));
        // get all non equal layers from originalLayers, ref from old map
        deleteLayers.AddRange(originalLayers.Except(ignoredOrginalLayers));
        // then filter out the ones with the same guid that are for updating -> gives you old to delete
        deleteLayers = new List<LayerDto>(deleteLayers.Except(updateOldLayers));
        // then filter out ones with the same guid that are for creating -> gives you new to update
        updateLayers = new List<LayerDto>(updateLayers.Except(createLayers));
    }

    if (createLayers.Count > 0)
    {
        foreach (var layerDto in createLayers)
        {
            result.Add(CreateLayerRequest(layerDto,mapName));
        }
    }

    if (updateLayers.Count > 0)
    {
        for (int i = 0; i < updateLayers.Count; i++)
        {
            var oldLayer = updateOldLayers[i];
            var nLayer = updateLayers[i];
            result.AddRange(ProcessLayerUpdates(oldLayer, nLayer,mapName));
        }
    }

    if (deleteLayers.Count > 0)
    {
        foreach (var deleteLayer in deleteLayers)
        {
            result.Add(DeleteLayerRequest(deleteLayer,mapName));
        }
    }

    return result;
}

This might not be the fastest way to check what has differences with the two instances of map, but being about to do these simple statements in LINQ makes short work of it.

So we have lists of differences, now we can create simple requests from the containing the new data.

private TiledRequestBase CreateLayerRequest(LayerDto layerDto, string mapName)
{
    var result = new LayerRequest();
    result.MapName = mapName;
    result.RequestType = TiledRequestType.Create;
    result.UserName = "Foo";
    result.LayerDto = layerDto;
    return result;
}

Now, if our map has 200 changes, sending out 200 requests is not a very efficient way of doing things, but for now this is what I'll be doing. A 'bulk' request functionality will be needed, but for now, it's a premature optimisation.

Send all the requests via the TiledStackProxy and ..

Updates straight from editor the running game 😃. This allows me to update tilesets, layers, map objects and properties and see how they look/act in the game at run time. Now that I'm running the latest version of Tiled which supports Map Object rotation, you'll notice some position problems. Positioning map objects before was accurate, so I'm going to leave my rendering code as is. Once I've updated Tiled itself to support turning these guid elements on and off in the UI, I hope to make my first pull request to the project. Hopefully this or similar functionality makes it into Tiled, as this will make passing the data around a lot simpler.

Persistence

Now that I have changes going to a MonoGame client, I want to be able to stop the game, fix some code and continue where the map left off or run the latest changes if some one else is updating the map. There are a number of problems to solve for this functionality, the first of which is persistence. I have started with MongoDB (opens new window) for saving the map changes and holding a copy for a user. MongoDB is really easy to get going and for what I am currently doing, does a nice job. That said, I actually plan to move to Redis (opens new window). This is because speed is going to mater and the source of truth of the map is not held solely remotely. The volatility of using a in memory database doesn't concern me in this situation and I've also been meaning to use Redis more in personal projects. This seems like a good fit.

For reference though, this is one of the services that is updating the map.

[IncomingTiledHub(Name = "mapUpdates", Method = "LayerDataMessage")]
public object Post(LayerRequest request)
{
    var collection = Database.GetCollection<MapModel>("maps");
    var map = collection.FindOne(Query<MapModel>.Where(model =>
        model.MapName == request.MapName && model.UserName == request.UserName));

    for (int index = 0; index < map.Map.Layers.Count; index++)
    {
        if (map.Map.Layers[index].guid == request.LayerDto.guid)
        {
            map.Map.Layers[index] = request.LayerDto;
            break;
        }
    }

    collection.Save(map);
    return new { Success = true };
}

HTML

I was writing a HTML 'map viewer' if you will that would also have the same live updates, but after trying to get 'MelonJS (opens new window)' to update the map for a few hours, I gave up on the idea for now. I was up and running with a static map of MelonJS very quickly, but found the code base quite difficult to extend (I might be coming from the wrong angle to do so though). I also very briefly tried 'canvasengine (opens new window)' , and although I didn't get canvas engine rendering, the project is very well architected and I may give it another go in the future.

I do want to have a HTML viewer (even if it is just a static latest version of the map) to help give distributed teams a common place to have a quick look at at the latest level looks like. I also want it to be a part of a larger set of web tools around hosted maps that people can share with the indie game dev community. If a developer decides to make a map public, people can have a go with other peoples map designs and maybe extend them. This is all in an ideal world and not sure how far I'll get with those features as I still have a lot to do around the basic developer workflow. For now, HTML will be stalled while I work on other features.

I'm currently working on 'versioning' via Tiled command, a way to mark and persist a version of the map as 'latest' for developers who want to use the latest version, but don't want objects popping into the game while they are debugging. I'll talk about those options and others in my next update.