Are they running this game on a Raspberry Pi or what?
In what way, the nodes were reinforced correctly, you just hammered them too hard, a reinforced node still has finite resources you know
From the first post
āIn this instance the automated distribution placed one staging system with X47L-Q by chance and this caused some problems.ā
Anyone who was in system will also tell you the tidi was even worse after downtime, which is explained by this auto distribution. And eventually leading to node death.
You can point at the player base for āhammeringā the servers too hard but the post above shows that the dev team is still learning too.
Itās not as simple as ādonāt have big fights because servers will dieā. If CCP thought that they would say as much or very simply limit the number of pilots allowed in a system.
Yes, but it was still on a reinforced node, unless your staging system had 3000 people in it it likely didnāt require that much in the way of resources so this crash would have still happened
I mean they know that, we know that, everyone knows that, they wonāt cap the system because that means no more record breaking fights and because it would allow 1 side to potentially negate your ability to enter the system at all
There will always be an upper limit on the number of people that can be in a system at any one time because server resources are finite, you would have still killed a reinforced node even if it didnāt have 1 staging system on it aswell
2 Likes
Do you split hairās professionally, or is it just your hobby?
2 Likes
Let me clarify.
The problem that arose from the staging system being on the same node as X47 was resolved shortly after server downtime manually. X47 was the only system on the node for multiple hours before node death.
Those two problems are not related.
6 Likes
Well, if a system canāt even handle live data properly, I reckon the backup will be messy, too. In this case, restoring that set of data was the least bad result.
1 Like
CCP should do nothing.
The problem will go away on its own. 
People will do something else than take part in those big fights, eventually.
1 Like
For fleet fights the simple answer is that as the number of players n increases then the number of interactions increases O(nĀ²). Server performance roughly entails a linear increase in the number of interactions and therefore a O(ān) increase in players.
8 Likes
Sure buddy, its only been 20 years. Eventually Bloc warfare will stop being a thing.
I really appreciate CCP addressing this and offering their perspective and insight. Canāt say I necessarily agree with the interpretation of rollback vs. queue depth upon crash made records be weird. It is impossible to recover records that werenāt persisted, thatās understood. However Iād question the priorities here, seemingly status change when docking, taking damage, getting killed takes priority for database writes over position in system which doesnāt really seem to be persisted as long as thereās enough of the other requests clogging it up. This is especially weird since there must be a semblance of positioning on the server, since we saw people move and could anchor on them, but as far as the server was concerned that position was never persisted?
This is reminding me of some Ego Shooters with desync problems. Taking damage while not actually being in a place to take damage yet(or any more) becomes possible. So for example I might have been scooting away from grid for an hour and seemingly be way out of range, when as far as the server is concerned Iām still well within optimal of a given hostile pilot. This creates the possibility of some really weird situations that may look like rubberbanding, up to the extreme example of somebody leaving grid, tethering up and then suddenly exploding?
would it be helpful to not allow RF timers to be set within an hour of downtime , and use the random +/- timer to ensure that ?
it almost seems that players set that timer intentionally , using possible node crash to save their structures ā¦
1 Like
Apologies I missunderstood thanks for the clarification
New boundaries and unexpected technical challenges are always fun. Iām sure a huge battle around downtime exposed a number of things to seriously consider.
With the caveat of not knowing all the technical factors and challenges involved, I would like to recommend you consider the following as a potential part of a possible solution.
With the logins and loading of the objects in space taking up significant resources. Perhaps in another downtime situation, time dilatation could effectively become a time stop. Some timer based on the number of undocked players before downtime.
Allowing the players to at least login and allow the nodes to process object locations. 6,000 pilots * 0.15 seconds. A 15 minute timer before a massive brawl resumes would be preferable to watching the login spinner and attempting to login multiple times. And once the time stop ends, start accepting commands and ramp up the time dilation to the appropriate level.
Thatās my 2Ā¢. Hope it might be useful to consider. Good luck and great game! Thanks!
3 Likes
Sounds like a good way to deal with post-downtime log-ins to me!
Would be pretty impressive seeing all those ships and fleets pop in one by one, filling the grid while nothing else happens or is moving. Filling the overview as preparation for a huge fight to continue in 15 minutes once all log-in requests have been addressed.
I mean, we know what weāre up to. We signed up for a massive yet slow TIDI battle.
Waiting 15 minutes frozen until the majority of players has logged in is preferable over hurrying to log in yet waiting for 90+ minutes as the battle continues with some of the people who mamaged to log in first.
1 Like
Oh trust me he has gone pro with thatā¦
1 Like
every good hair stylist uses TRESemme Repair and Protect 7, or similar, to repair split ends
1 Like
This is already implemented. Just not widely used. When a node is shutting down normally, it do not crash the time flow abruptly, but instead gracefully pause everything, and could technically resume from the very same state after a restart. The main issue here is players, who may or may not connect to the node while it is starting. Then it has to either collapse disconnected ships or let them die instantly without control. With cluster downtime length going down to a few minutes, this is now possible to utilize nodeās ability to resume the fights.
In fact, it already works that way. The node is normally coming to a full stop before restart. The state is fully saved and technically could be restored.
Thanks for this detailed answer.
If I translate for my non developer corp mates (those eating Pepitos), if a server sustains correctly 3.000 players, the server would need to be something like 77 more powerful to sustain 6.000 players (77=ā6.000).
Also for the my Pepitosā eaters, having a linear increase with the number of interaction means the game is just coded in the most optimal way possible.
@CCP_Explorer I imagine itās much more complex than that. Do you have heuristics to reduce the number of interactions taken into for each object? (and give us hopes of bigger fights without superhamsters)
Is the only solution changing the game mechanics, like for entosis nodes, to spread us a bit everywhere, for big fights like this?
I think itās a little less extreme than that, but nonlinear nonetheless.
6000 players is a factor 2 times more players than 3000, but if I understand correctly it requires a server a factor 2^2 = 4 times more powerful.
A server 2 times more powerful than a server that can handle 3000 players would be able to handle 4243 players (= 3000 * sqrt(2))
1 Like
IMO this canāt be solved by stretching time, and then fail, but only by gradually reducing the accuracy of the simulation itself.
1 Like