I implemented netcode for a canvas game I made a while ago over websockets. It used client-server with various RPCs from/to the clients and worked pretty well. I’m just starting on a kind of follow up of that game that will have more discrete movement so it was useful to read this overview. I’d maybe like to add that often games make a distinction between “authorized” actions players take (eg spell casts that damage other players), that often use the client-server RPC mode, and the “p2p rollback” actions that are less strict but generate more traffic like player movement. I believe World of Warcraft uses this model, and I also worked on a FPS that IIRC also worked this way: a gun fire was RPC over TCP; movement was done via UDP broadcast.
Just wanted to add that Unity can be a good candidate for deterministic netcode, but you'd have to look toward their new packages to implement it. Photon Quantum is a high quality solution for deterministic netcode as an example.
But in real game development, there's too few situations where lockstep fits, as it makes the game pretty slow to react to player's actions. And rollback brings so much complexity to the whole game that it significantly impacts development costs.
Determinism isn’t implemented (yet, maybe never) in DOTS NetCode. They have a lot bigger fish to fry right now.
I built a small game with Quantum a couple years back when I took a “sabbatical” (shipped an indie game didn’t know what to do next). I found it extremely limiting at the time, and despite determinism being the promise, I ran into constant desync issues.
Things have probably changed in the last few years, but Photon isn’t a large company and they have limited resources to improve it. YMMV
In particular, he talks a lot more about problems in floating point determinism, which was missing from the posted article IMO. It is possible to make it work without resorting to fixed point, however it is a big pain point.
A friend and I (mostly him, I just helped out a little) recently used netplayjs to create a p2p version of slime volleyball. It was surprisingly fun, not that hard, and this article is a pretty fair description of what to keep an eye out for. If you're looking to make an online p2p game, highly recommend this library, it's quite easy to work with.
I wrote all this stuff from scratch for a toy game engine I was making.
The 'desync' stuff was surprisingly easy to resolve. Instead of trying to make all game state deterministic (a near impossible task in some languages, where for example even basic math functions don't have fully defined rounding charactistics), I went for another approach...
Detect desyncs by periodically taking a ram snapshot of the whole game and calculating a CRC over it. It means your game state needs to be serializable, but you probably need that anyway so a new player can join the game mid game.
If the CRC doesn't match, do a majority vote to find out which player is wrong, then do the rsync algorithm to sync differences between the players. In nearly all cases there will only be a few bytes of differences.
There is no need for the game to halt or lag during any of this - the game can continue as normal. Players are likely seeing the exact same thing on the screen, because the differences between game states are probably an error of 0.00001 in some player position.
When the corrected state has arrived, replay frames to bring it up to the current timestamp and switch out states. Job done. User sees no glitchyness. Developers don't have to worry about tiny bits of nondeterministic behaviour in libraries out of their control. Everyone's happy.
Make sure it doesn't happen every frame though, or you'll waste a lot of CPU.
Depending on the game the error is often much larger. Especially when you have things like free player movement and obstructions. A little error can become a big error if one client sees a player move around a wall and the other see's them hit the wall and stop.
Wait, so you're saying that a majority of players can override any other player's entire RAM? That sounds like an amazing paradise for hackers and griefers :D
Still basically an infinitely larger attack surface than what OP describes which was fancy wording for some basic IPC primitives to synchronize some game state.
Well in the case of my game engine, all the game logic was untrusted anyway - in fact, in some games the actual game mechanic involved live-editing the code of the running game to try and give yourself an advantage - without ever causing a syntax error and while others are also live-editing the same code. Any keystroke causing a syntax error or infinite loop would be an insta-kick for the player who made that keystroke.
I mention this briefly, but with re-syncing it is possible to end up in a situation where you end up re-sending the game state on each check interval - more so with a full-state CRC since floating-point errors can make values binary different long before the difference would result in a visible desync. Floating point issues are thus generally approached by using fixed-point math instead.
Doing a CRC of full game state also poses some additional requirements - e.g. any pointers must be relative offsets and any dynamic allocation should be deterministic and within the same block. It does, however, also make it easier to implement serialization/deserialization, as mentioned by Jyaif in another comment here.
If you want to not hate your life building a real-time multiplayer game, fully embrace data oriented design.
I took the gamble and picked Unity DOTS and NetCode for our current MOBA-esque game (not other packages, we bolt regular Unity visuals on much like the V in MVVM). Outside the headaches of using preview tech, it has been amazing what we have been able to do compared to regular OOP hell.
By not having hidden game state lurking in classes, lambdas or coroutines, we have been able to hide much of the networking layer as an encapsulated set of glue systems that shuttle just enough data across the wire that the view-layer can present the game to the user. Movement and other prediction has been much easier too, because the game state is just data and not intertwined with behavior, I just run most of the same algorithms locally as I do on the server — AI, steering, etc.
This technique still requires a lot of foresight and challenging architecture decisions of course. I forced our core gameplay logic to not know anything about the visuals, which has made some things harder. However when it came time to make a headless build with no graphics, it was as simple as flipping a switch. No changes in our gameplay code.
I have a principal engineer from Sea of Thieves on my team, and he has also breathed huge sighs of relief working on this project. Unreal is the complete antithesis to this design pattern, and is full of hidden state, magic behavior, and heisenbugs.
I’d suggest searching for this question on reddit’s /r/gamedev to hear a lot of perspectives.
Mine? Don’t. By that I mean, and assuming you haven’t already, spend a few years making small games from start to finish. Pick games to copy, don’t even hold yourself to having a unique spin on things. The goal being to 1) practice for the real thing and 2) decide if you actually want to make games or just fantasize about doing so. The latter isn’t a judgement, I’ve almost left games twice because of the emotional burnout.
For a point of reference, I spent two years getting up at 6am twice a week with a friend to gamejam and practice. All in prep to go indie. This was even after 6 years making games and leading two.
The result? We almost failed completely. If I hadn’t decided to say fuck it and stick the game on PS4 for fun, we would have grossed a grand total of $100k for a year of work. The more depressing fact? $100k is considered a significant success for an indie game on Steam.
Happy to provide more anecdotal experience around this if you like.
Thanks for the grounding take; I'd appreciate hearing more about your experience if you wouldn't mind. Either here or offline if you'd rather not share publicly.
Take my cynical pov with a pinch of salt... I left (indie?) gamedev to get a normal job, and have been much better off both mentally and financially.
In addition to what hesdeadjim said, I'd also like to add don't blindly accept the common gamedev maxims. For example, "Make the game YOU want to play" is repeated as good advice, but I haven't found it to be very useful.
First off, if I'm working as the sole developer and have total control over design decisions, it's easier to follow that advice than if I'm working with multiple people who all play different genres of games. A lot of the time people will only use gamedev-jargon to back up their design decisions (immersion, positive feedback, vertical slices, etc), which are hard to argue about because they're so subjective and ill-defined.
Also, keep in mind how many times Level 1 gets tested. At a certain point, no matter how excited I am about the game, I'm going to get sick of playing it. From that point on, I'm no longer developing a game I want to play. I'm developing a game I should want to play but don't (I think this adds a lot to burnout). Maybe start the project as a game you want to play, but make peace with the fact that it's ultimately not for you to enjoy playing. It's for the players. You're very likely going to be sick of the game by the time it ships.
"Make it juicy" and "avoid premature optimizations" are more maxims with limited use, and are even contradictory in a lot of instances. For juice/polish: Focus on the core gameplay mechanics. Once those are set in stone, make them juicy if you want/have the time. A juicy boring game is still a boring game. For performance/optimization: If a core part of your game is having 10,000 enemies on the screen at once, you're going to have to make design decisions based on performance at the very beginning of development. Kinda related: I've shipped a game where at the beginning of development, I suggested using Multiple Render Targets (we were doing deferred rendering with the XNA Framework) to speed up the rendering (instead of rendering everything, switching render targets, rendering everything again, switching render targets, repeat...) but the lead dev took the "avoid premature optimizations" maxim to heart, and we ended up with a significantly hobbled final product on PS4s and lower-end PCs. I guess I could have gone into "crunch mode" and rewrote the entire rendering pipeline to make up for poor decisions at the start of the project, but I wasn't willing to do that.
It was my dream to make games from the age of probably 13 to ~28. I'm 35 now, and am not all that interested in games any more. I enjoy doing "gamedev" as a hobby, because I've realized my preferred method of "playing" is to program little toys I can tinker with for a few hours and then abandon. I also have MDD, so YMMV. Best of luck, though.
Just to add to the chorus of similar comments: I fantasized about becoming a game developer in school. I had no allusions, I knew what that took. I studied linear algebra, wrote toy games on my own, and even took up physics at university so that I could learn things like optics and motion simulation.
I got a job as a game developer, and I wrote a game engine. I was so burnt out and unhappy that I had to go on anti-depressants.
Years later, some of the most fun I had was optimising financial reporting databases. Why? I dunno, it was just more fun somehow. It wasn't my dream, and I felt like I was that kid that wanted to become an astronaut and ended up in middle management at some megacorp. But you know what? Optimising queries uses the same skills as optimising a game engine! It's fun, and you get paid for it. You get immediate feedback from users, instead of complaining from entitled brats on forums two years later...
> If you want to not hate your life building a real-time multiplayer game, fully embrace data oriented design.
I'd say if you want to not hate you life building any software, start with the data/schema/et. al. Before you open Visual Studio, open Excel and conference in your business stakeholders for a few design sessions.
We recently started down a "data-driven" path with our B2B application. We went from ~90% of the application being code-based to ~95% of the application being configuration-driven in a matter of months. We can bootstrap an entire customer's setup by doing some simple JSON copypasta now.
The best general model I have found so far would be to declare a global Domain type, and then put a List<T>/T of each constituent domain type within it. No nesting of complex types (reference types) should be permitted. All properties in the domain model types can map cleanly to/from SQL tables. The overall schema should be in 3NF/BCNF/DKNF. Relation types will be required. You can trivially serialize instances of the Domain type for whatever purpose. Each serialized copy of the Domain type can be thought of as a deterministic snapshot of all state, assuming you did everything correctly elsewhere.
Use of language features such as LINQ can make this approach almost fun to work with. A clean, well-managed schema is the most powerful tool in your box when dealing with complex problem domains.
Regarding rollback, there's an interesting technique that I stumbled upon:
Instead of serializing individual objects, you serialize all the memory relevant to the gameplay by using a custom memory allocator whenever the gameplay related code runs. This custom memory allocator is designed to allow taking fast memory snapshots, and supports restoring memory snapshots.
This allows rollbacking any code, including in my case rollbacking user-submitted Lua scripts.
I was a contributor to the old Playstation 1 title, 3D RTS game Warzone 2100, now FOSS.
I was well on the way to implement a "Drive Mode" when I came across several hurdles relating to the net code.
First I was on the path of implementing a pure FPS like game mode where any unit could be controlled, be it tanks, cyborgs, turrets, towers, bunkers, artillery, airplanes, boats, even walls! (Although they obviously couldnt move)
The first prototypes were awesome, really inspiring and the community gave lots of brainstorming.
We would be making battle royales, one on ones, elite single missions,
awesome multiplayer coops, you name it.
Then, I found out that the netcode only allows for move orders, go forward, turn left, speed 100, etc. Seemed manageable. So far so good.
But wait. What about aircraft? Ouch, thats important though. Now they just move with WSAD, and you just stop midair.
After that, I found out we couldnt simply click somewhere to shoot. Netcode only allowed to fire with intent to hit a specific unit. Hmmm ... Ok .. so turret cant rotate except when attacking.
And, dont click to shoot, but "click to attack".
I was initially really enthusiastic over reinstating this awesome game mode, but these restrictions totally killed it unfortunately.
I guess the bottom line is that yes robust deterministic netcode is good and all, but it kinda locks in your game to what it currently is designed to do and fundamental changes to that will sometimes be overwhelmingly difficult if not impossible.
It's not only about "whats is currently designed". If you choose lockstep, then after you get the player's input, you have to wait at least a couple of roundtrips before she's going to get a reaction, which is easily in 100-200ms. For RTS it's OK, but I don't think you could make a convincing action game with this limitation.
> This has to serialize/deserialize the entire game state (everything that affects gameplay) into some format that can be later read from - conventionally, binary serialization, but you can technically do whatever you want so long as it's fast enough (can execute in <10% of your game frame time).
This is only true if you use impure data structures. If you use persistent data structures you get this “for free” (the cost, of course, is worse asymptotics on mutable arrays and also potentially more GC pressure).
You just keep a copy of your old game states around as long as you want, which doesn’t cost anything except a few extra bytes that can’t be reclaimed by the GC for a bit.
Although I don't work in games anymore, I was recently thinking about peer-to-peer multicasting input events for prediction purposes, but using the client-server model for actual gameplay. That way you get low-latency movement prediction, with all of the perks of running a server (namely anti-cheat). When networking fast-paced games, proper prediction implementations can make or break the experience imo.
Edit: You can P2P the game state if you want, you just have to make sure it's usable for those with poorer internet connections (since you are sending/receiving much more data, position/velocity vectors etc. as opposed to just key presses, per number of players).
You phrase it as being the best of both worlds, but it kind of seems like the worst to me: you still need a central server without which the game stops, and which needs to be powerful enough to run the entire simulation, and on top of that every client also needs to run the full simulation (costs CPU), receive the entirety of inputs from everyone (costs bandwidth), and makes wallhacks possible since the whole state is known.
This seems very costly just to hopefully get faster prediction (for those clients that are close together, and whose NAT is not too broken for P2P to work). That information would also not be authoritative, since the simulation on the far-away central server is still the authority, so I'm not even sure how much it would actually help.
Haven't considered the wall hacks, that's a good point. Still though, I think you could mitigate this with a few different techniques.
The CPU cost is largely there anyways, as you are updating everything on the client from the server anyways. Any games that rely more heavily on physics will have more work to do to simulate yes, although basic gravity/velocity calculations with some linear interpolations thrown in is very lightweight, even when talking about 100 players. Bandwidth is quite small, how much data does it take to send a single key stroke? A number of bytes really, multiply by 100 players, multiply by say 2 or 3 keystrokes at the same time, multiply by 60 frames per second, and you still get a very reasonable (dare I say negligible?) amount of bandwidth.
The point is not for it to be authoritative whatsoever, the point is to not have to wait on two round trips from the server to get location data from your opponents, but try to do it in one round trip. If your connection to your opponent is lower latency/more stable if done through the server, then that should be the fallback of course. Movement prediction/smoothing happens even to your own personal character, even on a good, low ping connection to the server, the good games just know how to hide it effectively, but you often still get people complaining about hit registration. There is no perfect model outside of everyone having amazing internet connections.
43 comments
[ 4.7 ms ] story [ 105 ms ] threadYou have other articles discussing client/server netcode. The one from Quake 3 is a good start (and it is open source).
I implemented netcode for a canvas game I made a while ago over websockets. It used client-server with various RPCs from/to the clients and worked pretty well. I’m just starting on a kind of follow up of that game that will have more discrete movement so it was useful to read this overview. I’d maybe like to add that often games make a distinction between “authorized” actions players take (eg spell casts that damage other players), that often use the client-server RPC mode, and the “p2p rollback” actions that are less strict but generate more traffic like player movement. I believe World of Warcraft uses this model, and I also worked on a FPS that IIRC also worked this way: a gun fire was RPC over TCP; movement was done via UDP broadcast.
But in real game development, there's too few situations where lockstep fits, as it makes the game pretty slow to react to player's actions. And rollback brings so much complexity to the whole game that it significantly impacts development costs.
I built a small game with Quantum a couple years back when I took a “sabbatical” (shipped an indie game didn’t know what to do next). I found it extremely limiting at the time, and despite determinism being the promise, I ran into constant desync issues.
Things have probably changed in the last few years, but Photon isn’t a large company and they have limited resources to improve it. YMMV
In particular, he talks a lot more about problems in floating point determinism, which was missing from the posted article IMO. It is possible to make it work without resorting to fixed point, however it is a big pain point.
netplayjs: https://github.com/rameshvarun/netplayjs
The 'desync' stuff was surprisingly easy to resolve. Instead of trying to make all game state deterministic (a near impossible task in some languages, where for example even basic math functions don't have fully defined rounding charactistics), I went for another approach...
Detect desyncs by periodically taking a ram snapshot of the whole game and calculating a CRC over it. It means your game state needs to be serializable, but you probably need that anyway so a new player can join the game mid game.
If the CRC doesn't match, do a majority vote to find out which player is wrong, then do the rsync algorithm to sync differences between the players. In nearly all cases there will only be a few bytes of differences.
There is no need for the game to halt or lag during any of this - the game can continue as normal. Players are likely seeing the exact same thing on the screen, because the differences between game states are probably an error of 0.00001 in some player position.
When the corrected state has arrived, replay frames to bring it up to the current timestamp and switch out states. Job done. User sees no glitchyness. Developers don't have to worry about tiny bits of nondeterministic behaviour in libraries out of their control. Everyone's happy.
Make sure it doesn't happen every frame though, or you'll waste a lot of CPU.
Doing a CRC of full game state also poses some additional requirements - e.g. any pointers must be relative offsets and any dynamic allocation should be deterministic and within the same block. It does, however, also make it easier to implement serialization/deserialization, as mentioned by Jyaif in another comment here.
> 8 frames in 16ms
https://m.youtube.com/watch?v=7jb0FOcImdg
I took the gamble and picked Unity DOTS and NetCode for our current MOBA-esque game (not other packages, we bolt regular Unity visuals on much like the V in MVVM). Outside the headaches of using preview tech, it has been amazing what we have been able to do compared to regular OOP hell.
By not having hidden game state lurking in classes, lambdas or coroutines, we have been able to hide much of the networking layer as an encapsulated set of glue systems that shuttle just enough data across the wire that the view-layer can present the game to the user. Movement and other prediction has been much easier too, because the game state is just data and not intertwined with behavior, I just run most of the same algorithms locally as I do on the server — AI, steering, etc.
This technique still requires a lot of foresight and challenging architecture decisions of course. I forced our core gameplay logic to not know anything about the visuals, which has made some things harder. However when it came time to make a headless build with no graphics, it was as simple as flipping a switch. No changes in our gameplay code.
I have a principal engineer from Sea of Thieves on my team, and he has also breathed huge sighs of relief working on this project. Unreal is the complete antithesis to this design pattern, and is full of hidden state, magic behavior, and heisenbugs.
Mine? Don’t. By that I mean, and assuming you haven’t already, spend a few years making small games from start to finish. Pick games to copy, don’t even hold yourself to having a unique spin on things. The goal being to 1) practice for the real thing and 2) decide if you actually want to make games or just fantasize about doing so. The latter isn’t a judgement, I’ve almost left games twice because of the emotional burnout.
For a point of reference, I spent two years getting up at 6am twice a week with a friend to gamejam and practice. All in prep to go indie. This was even after 6 years making games and leading two.
The result? We almost failed completely. If I hadn’t decided to say fuck it and stick the game on PS4 for fun, we would have grossed a grand total of $100k for a year of work. The more depressing fact? $100k is considered a significant success for an indie game on Steam.
Happy to provide more anecdotal experience around this if you like.
In addition to what hesdeadjim said, I'd also like to add don't blindly accept the common gamedev maxims. For example, "Make the game YOU want to play" is repeated as good advice, but I haven't found it to be very useful.
First off, if I'm working as the sole developer and have total control over design decisions, it's easier to follow that advice than if I'm working with multiple people who all play different genres of games. A lot of the time people will only use gamedev-jargon to back up their design decisions (immersion, positive feedback, vertical slices, etc), which are hard to argue about because they're so subjective and ill-defined.
Also, keep in mind how many times Level 1 gets tested. At a certain point, no matter how excited I am about the game, I'm going to get sick of playing it. From that point on, I'm no longer developing a game I want to play. I'm developing a game I should want to play but don't (I think this adds a lot to burnout). Maybe start the project as a game you want to play, but make peace with the fact that it's ultimately not for you to enjoy playing. It's for the players. You're very likely going to be sick of the game by the time it ships.
"Make it juicy" and "avoid premature optimizations" are more maxims with limited use, and are even contradictory in a lot of instances. For juice/polish: Focus on the core gameplay mechanics. Once those are set in stone, make them juicy if you want/have the time. A juicy boring game is still a boring game. For performance/optimization: If a core part of your game is having 10,000 enemies on the screen at once, you're going to have to make design decisions based on performance at the very beginning of development. Kinda related: I've shipped a game where at the beginning of development, I suggested using Multiple Render Targets (we were doing deferred rendering with the XNA Framework) to speed up the rendering (instead of rendering everything, switching render targets, rendering everything again, switching render targets, repeat...) but the lead dev took the "avoid premature optimizations" maxim to heart, and we ended up with a significantly hobbled final product on PS4s and lower-end PCs. I guess I could have gone into "crunch mode" and rewrote the entire rendering pipeline to make up for poor decisions at the start of the project, but I wasn't willing to do that.
It was my dream to make games from the age of probably 13 to ~28. I'm 35 now, and am not all that interested in games any more. I enjoy doing "gamedev" as a hobby, because I've realized my preferred method of "playing" is to program little toys I can tinker with for a few hours and then abandon. I also have MDD, so YMMV. Best of luck, though.
I got a job as a game developer, and I wrote a game engine. I was so burnt out and unhappy that I had to go on anti-depressants.
Years later, some of the most fun I had was optimising financial reporting databases. Why? I dunno, it was just more fun somehow. It wasn't my dream, and I felt like I was that kid that wanted to become an astronaut and ended up in middle management at some megacorp. But you know what? Optimising queries uses the same skills as optimising a game engine! It's fun, and you get paid for it. You get immediate feedback from users, instead of complaining from entitled brats on forums two years later...
I'd say if you want to not hate you life building any software, start with the data/schema/et. al. Before you open Visual Studio, open Excel and conference in your business stakeholders for a few design sessions.
We recently started down a "data-driven" path with our B2B application. We went from ~90% of the application being code-based to ~95% of the application being configuration-driven in a matter of months. We can bootstrap an entire customer's setup by doing some simple JSON copypasta now.
The best general model I have found so far would be to declare a global Domain type, and then put a List<T>/T of each constituent domain type within it. No nesting of complex types (reference types) should be permitted. All properties in the domain model types can map cleanly to/from SQL tables. The overall schema should be in 3NF/BCNF/DKNF. Relation types will be required. You can trivially serialize instances of the Domain type for whatever purpose. Each serialized copy of the Domain type can be thought of as a deterministic snapshot of all state, assuming you did everything correctly elsewhere.
Use of language features such as LINQ can make this approach almost fun to work with. A clean, well-managed schema is the most powerful tool in your box when dealing with complex problem domains.
This read badly. Why is JSON in the picture?
In situations where we have "bad things" happen, we can simply dump the entire domain instance to disk and then reload it on a dev machine.
Instead of serializing individual objects, you serialize all the memory relevant to the gameplay by using a custom memory allocator whenever the gameplay related code runs. This custom memory allocator is designed to allow taking fast memory snapshots, and supports restoring memory snapshots.
This allows rollbacking any code, including in my case rollbacking user-submitted Lua scripts.
I wrote about it at https://www.jfgeyelin.com/2021/02/a-general-state-rollback-t...
(aside: code blocks in the post could use a fallback font in case Menlo isn't installed)
I was well on the way to implement a "Drive Mode" when I came across several hurdles relating to the net code.
First I was on the path of implementing a pure FPS like game mode where any unit could be controlled, be it tanks, cyborgs, turrets, towers, bunkers, artillery, airplanes, boats, even walls! (Although they obviously couldnt move)
The first prototypes were awesome, really inspiring and the community gave lots of brainstorming.
We would be making battle royales, one on ones, elite single missions, awesome multiplayer coops, you name it.
Then, I found out that the netcode only allows for move orders, go forward, turn left, speed 100, etc. Seemed manageable. So far so good.
But wait. What about aircraft? Ouch, thats important though. Now they just move with WSAD, and you just stop midair.
After that, I found out we couldnt simply click somewhere to shoot. Netcode only allowed to fire with intent to hit a specific unit. Hmmm ... Ok .. so turret cant rotate except when attacking.
And, dont click to shoot, but "click to attack".
I was initially really enthusiastic over reinstating this awesome game mode, but these restrictions totally killed it unfortunately.
I guess the bottom line is that yes robust deterministic netcode is good and all, but it kinda locks in your game to what it currently is designed to do and fundamental changes to that will sometimes be overwhelmingly difficult if not impossible.
PR on Github: https://github.com/Warzone2100/warzone2100/pull/829
My initial prototype, no relation to the netcode, point to shoot, camera follows turret: https://youtu.be/itsd29NebGE
My second prototype, no camera following aim, click to attack (iirc): https://youtu.be/ODBmOakkqWk
This is only true if you use impure data structures. If you use persistent data structures you get this “for free” (the cost, of course, is worse asymptotics on mutable arrays and also potentially more GC pressure).
You just keep a copy of your old game states around as long as you want, which doesn’t cost anything except a few extra bytes that can’t be reclaimed by the GC for a bit.
Edit: You can P2P the game state if you want, you just have to make sure it's usable for those with poorer internet connections (since you are sending/receiving much more data, position/velocity vectors etc. as opposed to just key presses, per number of players).
This seems very costly just to hopefully get faster prediction (for those clients that are close together, and whose NAT is not too broken for P2P to work). That information would also not be authoritative, since the simulation on the far-away central server is still the authority, so I'm not even sure how much it would actually help.
The CPU cost is largely there anyways, as you are updating everything on the client from the server anyways. Any games that rely more heavily on physics will have more work to do to simulate yes, although basic gravity/velocity calculations with some linear interpolations thrown in is very lightweight, even when talking about 100 players. Bandwidth is quite small, how much data does it take to send a single key stroke? A number of bytes really, multiply by 100 players, multiply by say 2 or 3 keystrokes at the same time, multiply by 60 frames per second, and you still get a very reasonable (dare I say negligible?) amount of bandwidth.
The point is not for it to be authoritative whatsoever, the point is to not have to wait on two round trips from the server to get location data from your opponents, but try to do it in one round trip. If your connection to your opponent is lower latency/more stable if done through the server, then that should be the fallback of course. Movement prediction/smoothing happens even to your own personal character, even on a good, low ping connection to the server, the good games just know how to hide it effectively, but you often still get people complaining about hit registration. There is no perfect model outside of everyone having amazing internet connections.