Repeated intro
What changed
0 fixes0 additions5 changes1 removal
changedWelcome back to another dev log for my indie factory automation game, Belts of Iron! Over the last week I've been testing the game in multiplayer while fixing a lot of bugs and improving network performance. It seems like the experimental branch is stable enough now to move to the main branch. I plan to release an updated demo version in the near future.
removedMultiplayer Networking OverviewMultiplayer games have two primary ways to sync data so all players see the same thing and know what other players are doing. First is a typical client-server architecture which has clients (players) sending requests to the server and the server sending (replicating) the state of the world to all clients. The requests from players include actions such as moving, placing a machine, changing a recipe, swinging their crowbar, etc. The server then verifies the request to make sure it's valid (usually by applying the action in its own locally running simulation) and then replicating any changed data back to clients. This would cause noticeable lag if the player had to wait for the server to respond, like loading a webpage. But clients often just assume the move will be valid and keep ticking forward. If the server sends something different back the client will need to correct it, possibly causing visible snapping movement or removing a building that shouldn't have been placed.
changedMultiplayer Networking OverviewFactory games are all about bottlenecks and this is no different. The bottleneck with this system is in sending data from the server to clients. If a client isn't doing anything they hardly send any data to the server so that's generally not a problem. However the server needs to constantly send data to all clients telling them what's happening. i.e. when items are extracted from mining drills, as they move along belts, which machines are crafting, when enemies spawn, when a player harvests a tree or rock, etc. If you guessed items moving along belts as requiring the most data, you were right!
changedSend less dataSo in order to replicate items on belts I previously serialized the entire array of items which contained the item ID and the space between each item. A big improvement in serializing this comes from knowing that most belts are going to contain the same item type, usually with equal spacing. With this knowledge I can pack the data by grouping consecutive items with the same ID and spacing into the same bytes with a simple count of repeating items. Because my game also randomizes the rotation of items I do need to serialize those in a separate array, but the data is less than 1 byte each so it's not a lot of data. Eventually I'd like to figure out a way to not have to replicate rotation at all since it doesn't matter for the simulation.
changedIn a future with lockstepThe second network architecture that many simulation games use is known as lockstep. With this architecture the server and clients all run their own simulation and "simply" synchronize the inputs. So that every "step" of the simulation clients have the same input data and execute the exact same resulting in identical output.
changedIn a future with lockstepThis would massively reduce the network traffic and improve server performance. The server would only need to replicate the client actions, which as we learned before is pretty minimal compared to what goes on in simulating a factory. However, because unreal engine is designed with a strong preference for the client-server architecture this lockstep method is much more complex to implement. It would involve rewriting a large chunk of code and undergoing months of work but could be worth it at some point in the future. For now, I plan to keep making optimizations to the current system as needed until it becomes too big a bottleneck of its own.
Belts of Iron changes
changedWelcome back to another dev log for my indie factory automation game, Belts of Iron! Over the last week I've been testing the game in multiplayer while fixing a lot of bugs and improving network performance. It seems like the experimental branch is stable enough now to move to the main branch. I plan to release an updated demo version in the near future.
removedMultiplayer games have two primary ways to sync data so all players see the same thing and know what other players are doing. First is a typical client-server architecture which has clients (players) sending requests to the server and the server sending (replicating) the state of the world to all clients. The requests from players include actions such as moving, placing a machine, changing a recipe, swinging their crowbar, etc. The server then verifies the request to make sure it's valid (usually by applying the action in its own locally running simulation) and then replicating any changed data back to clients. This would cause noticeable lag if the player had to wait for the server to respond, like loading a webpage. But clients often just assume the move will be valid and keep ticking forward. If the server sends something different back the client will need to correct it, possibly causing visible snapping movement or removing a building that shouldn't have been placed.
changedFactory games are all about bottlenecks and this is no different. The bottleneck with this system is in sending data from the server to clients. If a client isn't doing anything they hardly send any data to the server so that's generally not a problem. However the server needs to constantly send data to all clients telling them what's happening. i.e. when items are extracted from mining drills, as they move along belts, which machines are crafting, when enemies spawn, when a player harvests a tree or rock, etc. If you guessed items moving along belts as requiring the most data, you were right!
changedSo in order to replicate items on belts I previously serialized the entire array of items which contained the item ID and the space between each item. A big improvement in serializing this comes from knowing that most belts are going to contain the same item type, usually with equal spacing. With this knowledge I can pack the data by grouping consecutive items with the same ID and spacing into the same bytes with a simple count of repeating items. Because my game also randomizes the rotation of items I do need to serialize those in a separate array, but the data is less than 1 byte each so it's not a lot of data. Eventually I'd like to figure out a way to not have to replicate rotation at all since it doesn't matter for the simulation.
changedThe second network architecture that many simulation games use is known as lockstep. With this architecture the server and clients all run their own simulation and "simply" synchronize the inputs. So that every "step" of the simulation clients have the same input data and execute the exact same resulting in identical output.
Welcome back to another dev log for my indie factory automation game, Belts of Iron! Over the last week I've been testing the game in multiplayer while fixing a lot of bugs and improving network performance. It seems like the experimental branch is stable enough now to move to the main branch. I plan to release an updated demo version in the near future.
But first I want to talk about networking since I've been looking at it a lot lately.
Multiplayer Networking Overview
Multiplayer games have two primary ways to sync data so all players see the same thing and know what other players are doing. First is a typical client-server architecture which has clients (players) sending requests to the server and the server sending (replicating) the state of the world to all clients. The requests from players include actions such as moving, placing a machine, changing a recipe, swinging their crowbar, etc. The server then verifies the request to make sure it's valid (usually by applying the action in its own locally running simulation) and then replicating any changed data back to clients. This would cause noticeable lag if the player had to wait for the server to respond, like loading a webpage. But clients often just assume the move will be valid and keep ticking forward. If the server sends something different back the client will need to correct it, possibly causing visible snapping movement or removing a building that shouldn't have been placed.
Factory games are all about bottlenecks and this is no different. The bottleneck with this system is in sending data from the server to clients. If a client isn't doing anything they hardly send any data to the server so that's generally not a problem. However the server needs to constantly send data to all clients telling them what's happening. i.e. when items are extracted from mining drills, as they move along belts, which machines are crafting, when enemies spawn, when a player harvests a tree or rock, etc. If you guessed items moving along belts as requiring the most data, you were right!
Send less data
So in order to replicate items on belts I previously serialized the entire array of items which contained the item ID and the space between each item. A big improvement in serializing this comes from knowing that most belts are going to contain the same item type, usually with equal spacing. With this knowledge I can pack the data by grouping consecutive items with the same ID and spacing into the same bytes with a simple count of repeating items. Because my game also randomizes the rotation of items I do need to serialize those in a separate array, but the data is less than 1 byte each so it's not a lot of data. Eventually I'd like to figure out a way to not have to replicate rotation at all since it doesn't matter for the simulation.
In a future with lockstep
The second network architecture that many simulation games use is known as lockstep. With this architecture the server and clients all run their own simulation and "simply" synchronize the inputs. So that every "step" of the simulation clients have the same input data and execute the exact same resulting in identical output.
This would massively reduce the network traffic and improve server performance. The server would only need to replicate the client actions, which as we learned before is pretty minimal compared to what goes on in simulating a factory. However, because unreal engine is designed with a strong preference for the client-server architecture this lockstep method is much more complex to implement. It would involve rewriting a large chunk of code and undergoing months of work but could be worth it at some point in the future. For now, I plan to keep making optimizations to the current system as needed until it becomes too big a bottleneck of its own.
As always, thanks for reading and head over to discord to learn more
https://discord.gg/xmmPeZHj4T
