Game engine migration

(The AntiChrist666 Diablo13) #1

I want to discuss here with you if you know the obstacles and the common boundaries about upgrading the game engine EVE online client.

I have seen other people mention that the eve online client has to be upgraded with a new engine, and got replied about the compliance, for now what I have seen the only obstacle is the amount of assets data which have to migrated.

I personally think isn’t that issue able to be solved with the recent developments in artificial intelligence, and I’m pointing here to Big Data processing, but I could be totally wrong about it and could zero to do in that matter…
Although I’m for sure at the moment it would be more easily to do than a few years back with current IT technologies in the field.

Also one of an idea to make things less of a burden on a game migration is to first process all the data (assets) of inactive accounts the one which are inactive for a verrry long time so that part could be done while the servers are online.
These accounts will also get a notification about to log in ingame or apply on the eve online website with account if they don’t want their account locked (prevent from playing) till the game is migrated the moment they are selected by the system for processing.
Although ccp have to hire some real scientists overseas to overwatch these icelandic technicals preventing them screwing things up in this like they always do :ccpguard: (still some people able to login after the system suppose to lock them from login in client messing up the process, or game migrated to empty space and not legit back up of the original one, ccp tech dudes has endless classics LUL)

But anyway, why I think it is necessarily is because of the TiDi followed by game breaking stuff happening, can’t lock, UI not responding, targets unlock, game break and game break everything literately breaks.
And I’m talking here about big battles, those are suppose to be the grail of EVE online, but when it’s there, everything breaks.
TiDi is a thing and than everything breaks, this is unacceptable.

Anyway if you can and able to educate me in how this works and how it does, what the known obstacles are, or if you have any knowledge about this topic please tell me. :+1:

(Nevyn Auscent) #2

You do know that the current game & ‘server’ engine are actually pretty unique and advanced right.
You aren’t going to magically get no bugs by outsourcing it to someone who has no experience with EVE, you are going to get far far more.

Not to mention that EVE’s engine constantly gets upgraded anyway.
Tidi is simply from putting 5,000 people on the same grid and letting them fight. Most MMO’s lag at 24 in the same space.

(The AntiChrist666 Diablo13) #3

You shouldn’t be trolling m8, incase you did not recieve it yet here is your badge reward :medal_military: ‘master troll’

Anyway thanks for your reply :smiley:

(MacGregor Orlenard) #4

That topic is some sort of troll. Why you want rewrite whole EVE to new engine or probable write new one that will be able maintain EVE universe.

You have 0 knowledge about how coding works and just complaining about “bugs” that are not even related to game engine.

Buy CCP super computer and pay electricity bills for next 20 years and then demand no lags in huge fleet fights

(The AntiChrist666 Diablo13) #5

Calm down brother, I’m just asking if people could share some knowledge about this topic with me.

You should invite more of your friends to here to put me in place in the computer world :blush:

(MacGregor Orlenard) #6

Simple example. In EVE you are flying with thousands other players and NPC all the time. Every action, yours or any other players must be calculated by CCP servers cluster. When you on a mission. Even change of speed can trigger NPC script that cluster must calculate. Imagine how much computing power demand such simple thing if you multiply it by all online mission runner.

When in fleet or with more players on grid all data and your actions and they effects also must be sent to that players almost in real time.

There are endless examples of how online gameplay works.

And assets you mentioned in first post is not so big deal to transfer. Just raw data: bitmaps, 3d objects, sounds etc. The biggest problem are all scripts and code that make work all together.

Ps. I don’t have friends, especially on EVE forum.

(Andrew Gernander) #7

Big battles and the difficulty of translating a database to a new engine are two totally different problems.

The big battles lag are a combination of the speed of the server having to calculate the position of tens of thousands of moving parts while receiving and responding to input chains from across the globe all moving at different speeds across the crazy quilt of the internet. The solution in the industry is shards or player caps which goes against the basic principle of Eve being a single shard. As most third party engines you can purchase with a big credit card employ the shard limit schema, they’re not the best idea for Eve. Even these third-party solutions lag if your connection isn’t fast enough or interrupted.

Database is its own beastie. Forget Eve is a game for a moment. Imagine Eve as the database for a huge mega-corporation with hundreds of thousands of employees, of which at any time about 30-40 thousand are making transactions in real time against the company inventories and ledgers.

These databases are usually tied to programs that make them usable to the employees, either an in-house solution or a package that is bought from a large vendor, that makes reports for the executives and employees to use to make business decisions. That’s your Eve Client and any API’s for the folks watching at home.

Do the engines and their reports need updated from time to time? Absolutely. Many business modules are programmed in COBOL or old version of Crystal reports, or some other incredibly archaic interface that looks like Windows 3.1 and need converted to something new so they can be used with the cloud. Bank regulations change, so the credit card module needs updated so it’s security compliant. This stuff happens on a regular basis. In eve you get UI changes, new ships, new items, whole bunch of things that are simply a client report pulled from the database, just in shiny object form.

Most of the time the customer will patch the existing software as the existing software is already optimized for that company’s data and this is the less expensive and less disruptive solution. Outages and data problems still happen, especially if a patch is rushed, but at a manageable rate that can normally be resolved from backup or a rollback. Eve is that company opting to patch modules as they go, as what they have at present fits their data like a glove. Sometimes they’ll make a new component that plugs in to the old data and sometimes they’ll adjust the tables so that it works better but this is a very much trial and error process involving time and repeated attempts to get right.

CCP gets a nod of respect from me for how long they’ve kept it running with the complexity involved even if it’s not perfect. They’ve sure done a better job than the consultants at my workplace keeping the database running consistently.

Now, sometimes it’s absolutely necessary to bite the bullet and switch to either a new database engine, or a full revision of the current database engine. Program is no longer supported or the programmer died of old age (this does happen), can’t get Windows 98 computers any more, any number of reasons. This is a costly process involving months of testing and debugging and if everything is not done and tested precisely will cause a huge amount of problems and incorrect data going forward. You wouldn’t want to lose your titan to a rounding error, would you?

Lots of companies cut corners and don’t do the proper testing time as the consultants who usually do this work get paid enough to drive Ferarris and Teslas and have two or three in the garage. When you’re dealing with in-house guys only to save costs you run in to the very human risk of “this is how we’ve always done it,” syndrome which is hard to break out of if the old way has usually worked.

Good luck if you’re going to a different engine type entirely for your data. I’ve had the “pleasure” of saving old tables and data schemas from outdated systems and trying to make them usable for something else. I marginally succeeded. No, I was not paid in ferarris. :stuck_out_tongue:

The tables are usually a mess, make no logical sense as the programmer notes are not available or no longer there, or simply won’t work with a new engine as part of the table was deprecated five updates ago. There is a staggering amount of forensics work involved in trying to take a massive database and figuring out how it all works together then translating it to work in a new system, even under ideal conditions.

AI and Neural Nets are not usable for this as in order to get the desired output you have to program set “values” in to the neural net and if you are not explicit with those values (such as, use proper grammar), the machine will get real “creative” with the results. View them more as incorrigible toddlers who will gladly wash the dog in pink dye as it was not told to them specifically that this is not allowed when you gave the command to “wash the dog.”

From CCP’s perspective, why risk destroying a mostly working product when you’ll have more consistent results and fewer major failures by massaging a system here and there and working with that smaller set of results? The structures transition from POS to ship-like things are a good example. New UI modules are made with new tables to work with new technology and while not perfect they are an advancement.

It doesn’t cost a programmer much to write a fiscal report off the existing tables with some creative math inside the report, certainly not as much as rebuilding a database from scratch chasing the optimization unicorn that transforms about once every six months in to a new beast. You’ll never catch that beast and you’ll die trying.

(The AntiChrist666 Diablo13) #8

It works in some exentend yes, but what is going on at the moment is the client not responding correctly, or 13 fps on a gtx980.
It’s not the time delay TiDi I have complaince about , in the end it could be something you have to work with duo to the limitations lets say in technology.
The thing what is now the client is let’s say breaking, erratic behaviour of the client, like I said the big star wars battles are the grail of EVE online, but when it’s there the client can’t handle it, you can’t send orders, you can’t click things in the UI, nothing responds, and after 1 minute or so It will perform a random action of all the 20 clicks you tried to perform, and that’s how it goes for 1 hour long, imagine that when you have to protect important strategic assets in space.
You should have been participating in big wars than really know what I’m talking about.
So should ccp limit the amount of pilots which can able to enter a system, probably no right?
What I’m saying is TiDi which is playing this game in slow motion like at 10% speed which is 10x slower followed by erratic behavior of the client which I just mention (if you participated in the last war you know 110% what I’m referring about)
This, this is unacceptable what I’m saying.

On the other/development side the pink dog is a funny example, but although I think getting the dog pink again can be prevented.
It’s a process which can take 100 years or 1 year.
What is important is to experiment, getting knowledge in the field, even if it’s bad or good, you always get know something.
And yes a lot of testing, for example when they released the playstation4 they start developing the playstation5, are they planning the production line to release it? no, just testing the waters in terms of development and tech etc etc you get what I mean.

On AI part I think it could be done, just tell it to learn doing that what you want

Like in this picture concept, for part I’ve seen is eve mechanics are based on equations, maybe I’m wrong here but I’m for sure all computer code is.
Maybe something like this hasn’t been done yet for sure, but if it hasn’t been done yet doesn’t mean it’s not possible, what if the creator of Minecraft thought like well this hasn’t been done yet, than why should I utilize this mechanic and create something simple, so I’m not doing it because it was not happened in the universe elsewhere to my knowledge.
Although I’m stating it’s not the same level of course haha, but it is the same level of taking opportunity, you can get basically anything done with artificial intelligence at this point, and every computer scientist knows this, the only reason things go at a slow phase is because people expecting others to do it or think it hasn’t been done elsewhere in the universe around them so they are stuck in their tunnel vision, but anyone in their ordinary room can make revolutionary breakthroughs behind their computer with AI at this point because it’s an unexplored field for what it is now.

But in the end migrating the game engine Is it possible? yes, do we know how to do it now, certainly not. :innocent:

If one of you forum trolls (who doesn’t even play the game) telling me again that I’m complaining about the TiDi here I’ll make you eat every chicken here around, I know you’re smart bro but don’t give your self away that easy.

(The AntiChrist666 Diablo13) #9

I’m not sure why you are telling me randomdly common stuff about how eve works but thanks for telling me transering assets is a not a big deal MacGregor Orlenard

(MacGregor Orlenard) #10

To show you that your problem is not related to client but network and whole internet. Also cluster limitations.

Ps. Role of graphic card in FPS number is much lower than you probable think.

(The AntiChrist666 Diablo13) #11

So what you are saying is, the reason the client is not responding propperly for example, not processing my actions, trying to lock a target and keeps being stuck at 100% (99%) which prevent to lock it on, refusing to select another target or showing a different info panel of the selected target, refusing to warp, refusing move into a direction, etc etc, and I can go on for a long time.
Which happens in fleet fights.
Also the client lags and stutters and fps at around 10 fps (literal client response stutter)

Is an issue limited by internet? and network related, So for example if you would upgrade the game engine of the current eve online engine with a modern one all this those same issues still exist are you saying?
So this is not able to be solved? ( you could be right if you’re saying that)

If you are again on about the TiDi and not about what I’m mentioned about above here I’ll make you eat a chicken.

(Andrew Gernander) #12

I play plenty and with noticeable lag. About 80% of the time on a 3G phone tether and two bars signal on a cheap low-end laptop that just recently replaced an 8-year-old one I got bargain basement from a literal crazy Russian. I honestly think the only difference in specs is the new one supports DirectX 11.

The other 20% of the time is spent on a much better desktop system on a cable modem and fairly stable wired router with minimal usage, good memory, with a decent video card (R9 200). My biggest source of lag on the desktop is my hard disk speed, which I plan to remedy with my Christmas bonus.

Is there a gameplay difference between the two? Absolutely. The client has to send and receive the same number of packets to the server to report my position, what I’m doing, and where I’m going whether I’m playing on a piano or a potato. It will delay response or rubber band waiting if my connection drops a few packets here and there. I lost a mach to a Redtail Shark due to a lag bubble during the last event while playing on the laptop. :joy: Client completely misreported my shield booster status then disconnected. I just laugh and move on.

On the laptop I patiently wait out the client and go about my business. Under normal load, the game is playable and I can usually succeed at what I’m doing.

It becomes a matter for me of shutting off the fancy, as well as some combat notifications, then waiting for the server and client to communicate with each other and do their magic. Hitting all the keys because nothing seems to happen is counterproductive because the two machines have to handshake across the internet and agree that what just happened actually happened. I also usually need to check my drivers are updated and run a defrag and optimize after an update as the files get scattered everywhere.

Visually the client just seems to have a mind of its own because it’s waiting and it will rubber-band to the proper position once it gets an update. It’s anticipating the next set of frames then adjusting when the report from the server corrects it. Pretty standard behavior for a network game these days with any sort of latency or large server load.

TiDi is the result of the developer’s decision to allow everyone to get a command in before the next server tick. It seems random from the client side, but it’s processing one command from the stack you sent, probably the first. If the client matched the server behavior exactly, you’d get the same results as one of the early Doom multiplayer clients where everything slowed to a stop, and maybe a frame would pass after about a minute once the guy on the 800 baud modem finally caught up.

CCP will not limit the number of players entering a system as it goes against the philosophy of single shard. If I am logged in, I can influence you, is the philosophy.

If I were to spitball a suggestion keeping those extremes in mind, perhaps on reinforced nodes the “please wait” indicator then the client locking the buttons while the first command entered processed and the next tick occurred would solve the discordant state between what the client was seeing and what was actually happening. Key Press - Client Acknowledges Command, locks input (and maybe visuals), handshake with server, send/receive current tick, unlock client and process next command. It would become EvE the tabletop game pretty quickly and cause other frustration though.

Finishing tiercide, so there’s far fewer modules and math to do, might help a little on the server end speed things up, plus some table optimization once the data’s cleaned up, but probably not enough to be noticeable as the servers still waiting on 1000+ clients in a system to shiptoast so it can blargh out the results.

Client side, make sure your drivers are up to date, your cache cleared, and run a defrag and optimize after each major update (I use Auslogics). I’ve discovered in my playing that the hard drive speed has much more to do with performance than the video card, which is just rendering the graphics provided by the local resources on the hard disk. It won’t solve the extreme load conditions of major wars, but It should help with the day-to-day.

(The AntiChrist666 Diablo13) #13

For me it’s like you activate TiDi, than atleast make it work propperly.
Like the last post of mine here just before this one of yours some explanation about a few things.
When I’m in fleets with couple of hundred people we are all told to be very precious with our clicks during the fights when things starts breaking.
The last big fight in perimeter where goons wanted to destroy the fortizar, the last moments there the client was breaking so bad none in fleet were able to issue orders with their ship + the usual stutter problems.
Can you imagine whole fleets being paralyzed like that in the heat of battle, luckily the automated drones will always keep swarming the structures.

Maybe I sound like a f*cking parrot, but this behavior is unacceptable for being accepted as factor to take in mind :joy:

Also thanks for the tips for optimization, although it’s a shame this got to be done to make it function more properly.

(Andrew Gernander) #14

Unfortunately the problem is mostly “physics says no,” if you want big battles and for everyone possible to contribute.

It’d be no different than if we had everyone LARPing Eve (Non-boffer) in the same room trying to shout out their attacks all at once with a single GM trying to hear them. We time freeze and go to turn order so everyone can participate.

Shutting off the combat notifications, lowering the client settings, and turning off auto-repeat on your modules should help you some, but if it’s lag city on an unenforced node, it’s lag city on an unenforced node. Not all large battles are predictable in advance.

I get that 99% lock but nothing locks bug quite a bit on the laptop and sometimes on my home if the router overheats (bought a new router due to that recently). Client happily goes about on its way until it gets a server update, then it rubber bands to the correct position. Nothing to do with fleets or tidi, just my connection being slow and cantankerous.

You put the server side under load though, be it fleet battles or someone evil DDoS’ing, or an undersea cable getting cut, or the hamsters going on strike, you get the exact same result.

So, winner winner chicken dinner? :slight_smile:

(Nevyn Auscent) #15

To answer about the client error issues, yes it’s the server/internet.
When you clock a module it does not activate on the client side. Your client sends the activation to the server and the server then has to send a message back to your client saying ‘you have activated a module’ before you actually get to lock it. Some stuff like facing & remaining lock time gets approximated by the client but doesn’t actually happen until the server confirms it has happened which is why lock time can get stuck.
Tidi is slowing the game down so the server has more time to process everything, but there is a limit to how much it can slow the game down, and beyond that the server ends up overloaded and dropping things. This will happen no matter what the engine is, it simply can’t calculate things that fast. Most MMO engines show this load of dropping skills at somewhere between 24-48 players. EVE can get up to 500 without Tidi on a reinforced now, and to the 3,000+ mark without reaching commands dropping when reinforced.

FPS is your side, and caused by trying to overlap too much information and work out what information actually gets shown. If 50 icons would render on the same part of the screen it has to work out what wins. If 5,000 icons have to render, it takes a lot longer. Engine ‘might’ slightly help with that but at the end of the day it’s pretty much to do with the handshake game, where 500 is not double the load of 250, but orders of magnitude more load due to all the possible combinations.

(Keno Skir) #16

It basically sounds like you need a new computer, not CCP.

(Nana Skalski) #17

I dont get it, how can you have 13 fps on gtx980. What processor you have? Maybe also check if someone dont mine ditcoins on your computer.

(Blade Darth) #18

The AntiChrist666 Diablo13

Putting the troll/ uninformed thread aside, a little birdy told me the icelandic overlords have some solutions in mind. One of them is “splitting the blob” via physical barriers like asteroids or space phenomena (or just game mechanics that favor smaller gangs) so we won’t get 1500 people on 1 grid.
ETA winter 2030.

(MacGregor Orlenard) #19

Because game drops FPS when has no information what to redener. Ale may be cpu overload when lot of data must be calculated in fleet battel. 80% of EVE performance is related to internet. Other 20% is simply what your hardware can redener.

@The_AntiChrist666_Diablo13 Can you tell us how you are connected to internet and what is performance of your connection?

You can use this website for tests:

Also by clicking ctrl + alt + shift + m, you open in game performance monitor. It can show you memory usage, how much data is sent by network etc. Maybe your PC is struggling with something and you get bad performance.

(The AntiChrist666 Diablo13) #20

I get that 99% lock but nothing locks bug quite a bit on the laptop and
sometimes on my home if the router overheats (bought a new router due to
that recently). Client happily goes about on its way until it gets a
server update, then it rubber bands to the correct position. Nothing to
do with fleets or tidi, just my connection being slow and cantankerous.

I don’t udnerstand this, are you telling everyone in my fleet is experiencing the same issue because of my slow internet?