Dev blog #13

So today went live the patch 0.4.1. Here are the changes.

• Snatchers are introduced as a new type of unit. They act as an alternative to grabbers. More info in Dev blog #12.

• Added Tab mode.

• Chests can now have their slots blocked to prevent snatchers/grabbers from filling them.

• Swarm trails can now be set to 'Wait until full', 'Wait until empty' or 'No restriction'.

• Added a 'Made in' section to crafting recipe tooltip.

• Modified the tutorial to account for the new snatcher mechanic.

• Added a tip that shows the player how to interact with buildings under units during the tutorial.

• Added a presets dropdown in the map generation menu.

• Depot now has a description.

• Coal has been renamed to Refined coal.

• Bug fix when placing a corner trail over a ghost corner trail.

• Intro text has been slightly slowed down.

• Bug fix for 'Show minimap' option that used to appear in the main menu.

• Bug fix for when player tries to pick up an item from a grabbers.

• Bug fix for rain now showing on certain players monitors. If this still happens to some of you, please let me know.

• Bug fix for placing a belt backwards while hitting a non belt obstacle causing a rotation issue.

Now last blog I talked about snatchers and how they are this cool new alternative to grabbers. Check it out to find out how they work because they are now the Swarm approved way of transferring items from one entity to another.

Today I want to talk about the technicals behind it and how the Swarm keeps growing without blowing up your CPU.

I like to think that one thing that sets this game apart from most other automation games is its technical implementation. I aim to utilize as efficiently as possible your computer to create a true Swarm simulation, which is why Swarmdustry's core gameplay loop is multithreaded. How do snatchers, which are entities that interact with many other types of entities, fit into this environment?

In ECS, doing computations over entities is usually lightning fast. You have highly memory optimized objects, or entities as they're known in ECS, stored in contiguous memory chunks. This dramatically improves CPU cache efficiency compared to your usual OOP design because of the way Systems work. (You can read more about it online if you're interested in more details). But this efficiency somewhat falls off when entities have to interact with other entities. That is because in order for entity A to, say, access the inventory of entity B, the CPU needs to jump in memory to wherever this entity B's data is. If it's not in the cache, it will result in a cache miss and slowing down performance.

No other type of entity interacts more with another type of entity than snatchers (and grabbers). Cache misses for these guys are going to be a given, so it's important that the data structures and jobs running on this data are very well optimized. And when I say very well optimized, I mean EXTREMELY WELL.

The first thing to do is data storage. Say you have a 2x2 building with a max snatcher range of 2 tiles away. That means you're gonna have a potential maximum of 32 world cells to scan for potential entities to interact with. So worst case, say 32 chests around a forge.

Do you:

A) Store those cells as world coordinates on your building. Memory footprint: float x, float y = 8 bytes in memory * 32 = 256 total bytes.

B) Store those cells as integers on your building and use a clever algorithm to assign an index to each world cell relative to the top left cell (that would be index = 0, then the cell to the right would be index =1 etc). Memory footprint: int index = 4 bytes in memory * 32 = 128 total bytes.

The answer is... neither! Come on, I said EXTREMELY WELL. That is not extreme enough. Why not turn the index into a byte? Boom, now you have byte index = 1 byte in memory * 32 = 32 total bytes. This is at least 8x times as fast as the naive A) implementation.

The second thing to do is to make sure that your CPU doesn't do any unnecessary work. For example, say you have a Forge with 3 grabbers that should receive coal and iron ore from a chest and output it into another chest. The 3 snatchers should not individually scan those chests themselves. Instead, have the Forge do it once. Boom, we just saved 2 cycles of computation. Another unnecessary work would be to update building references on the forge every frame instead of when needed, i.e. at place, destroy or rotation time. No need to scan a world position every frame to see if there's a chest there or not. Just do it once.

Third thing is to multithread the algorithm as much as possible and divide it in such a way to avoid passing in unnecessary data to the jobs. In the case of snatchers, do most of the logic on the buildings where the snatchers work at, not on the snatchers themselves.

[c]JobHandle jh1 = updateMovingParentPositionJob.ScheduleParallel(state.Dependency);[/c]

[c]jh1 = handleOutputOnlyBuildingsJob.ScheduleParallel(jh1);[/c]

[c]jh1 = handleInputOnlyBuildingsJob.ScheduleParallel(jh1);[/c]

[c]jh1 = handleBothInputAndOutputBuildingsJob.ScheduleParallel(jh1);[/c]

[c]jh1 = handleSnatcherStatesJob.ScheduleParallel(jh1);[/c]

[c]jh1 = handleSnatcherInteractionJob.Schedule(jh1);[/c]

[c]DoMainthreadOnlyOps(ref state, ref itemsToCreate, ref itemsToDestroy);[/c]

Almost everything should parallelized, except for the interaction between the entities themselves. That has to be singlethreaded because it would result in race conditions otherwise (what if snatcher A and B tried to snatch the same item from the same target at the same time? Crash!)

That's all for today. See you next time!