Use the affiliation endpoint. It’s only cached for 1 hrs.
If the cunck fails, apply binary search to process the chunk.
The 1 day time was already fairly long. I agree 7 is unacceptable for most people who rely on automation to help manage access on many third party applications. I can understand that alot of uses hitting a particular endpoint could be a bit of a headache, but if its already cached for 1 day, running the one query a day for every use should not be that big of an issue. I would assume this is more to discourage the use of the ESI than to actually solve a load issue. To that end would it be possible to open up this and allow developers to sync their own copies once or twice a day to shift the burden to the third parties?
/corporations/{corporation_id}/members/
exists - why not use this one to regularly check for members in your own corp?
To retrieve the complete character information for every character one knows of regularly just to validate affiliation is definitely overkill and the wrong endpoint.
- Not everyone operates in a Corporation
- NPC Corporations Exist and are used regularly in FacWar
- Alliances are made up of many corporations
- This only returns Character IDs (meaning you need to use the affected get_character_id endpoint for more data
- This requires a Scope from a member of this corporation, becoming a headache for allies, NPC corps, or simply a ton of extra work and maintenance for your own corp, adding an IT advantage to groups that can manage this work.
- What if you dont know what corp someone is in. Since thats the point of this endpoint.
- What if you dont have a token for this corp, (say a victim or attacker on a killmail, someone who purchased an item from you, someone who sent you an evemail )
- This endpoint doesnt contain the same data as the affect endpoint
(this is partially a copy pasta from an equally uninformed question from reddit)
- The main argument brought up here was that it makes tooling for corporations more difficult - but this seems to be false if it is really about managing your own corp, and not to spy on unaffiliated players.
- sure NPC corps exist - but why would you want to track people in NPC corps anyway? this is exactly what was talked about on stream - that it is so easy to spy and get updates about people who do not concern you?
- well yeah - I am in one of the biggest alliances in the game. and that is why alliance endpoints exist. Get corps for alliance, get members for each corp. Overall a lot less endpoint calls to validate alliance membership of thousands of players compared to thousands of individual character requests.
- Querying the character endpoint once to retrieve the name and other static data if the ID is unknown, is a different thing than querying the endpoint every single time you want to get their affiliation.
- Managing your own corp should not be an issue with your own token … same goes for alliance tooling. You only require the token from a single director or similar per corp to use pretty much all the alliance/corp tooling… which you would require anyway for advanced alliance tooling, as all of that is behind authentication.
- Back to the previous topic, why would you want to know affiliation of someone NOT affiliated with your group? And if you do for whatever non-alliance/corp-management-reason - well then the tradeoff is now, that you cannot get it anywhere near real time.
- Same as before - why is the affiliation so important for that? Sure it was nice to have statistics on alliance level about combat outcomes etc… but why not query the
/characters/affiliation/endpoint instead with all the participants? Should give enough information to fill that gap (even though yes - it is too fragile and needs some fixing… but that is a different topic, as the current design explicitly breaks for projects scraping data, instead of doing targeted look ups).
I can totally relate to the issue CCP has, that one endpoint that gives mostly static or rarely changing data about any character in eve gets hammered with requests by so many tools for varying purposes, even though more specialized endpoints exist that are better suited for the probably intended purpose. I wouldn’t be surprised if there are more changes like this coming, that make it harder to aggregate information about anyone in eve in a timely fashion without their knowledge.
1 Like
You really try to make our lives difficult… Now a new potential recruit needs to wait up to 7 days before I can background check them before i can let them join corp…
Also, our communication software uses the auth to assign roles. People wont be able to join comms and when they leave (to for instance an enemy alliance), they can listen in on our fleet comms for an additional week?
Please revert it to something shorter than a full week refresh. The third party tools surrounding Eve Online are so much more important to us than you know.
Seriously CCP? You do know how important character information is for background checks and service access? Revert this immediately and learn to interact with your community before making decisions that impact them this much? Also please stabbing our community devs in the back with stunts like this…
When Premium ESI Service for money?
This is a seriously poor change CCP! Please revert this ASAP. We rely on this ESI to Auth access to our IT services which critically includes our wormhole mapping software, if it takes 7 days now to access this service its unusable!
How can we as 3rd party developers help you sort the underlying issue rather than being punished and rendered applications that becomes unreliable?
Changing the cache timer seems like a very reactive approach to the problem of “increase in number of projects scraping it”…
For one, you have the ESI hits remaining that we’re supposed to follow, which clearly is not accurate at all and cannot be relied upon, so we have to find workarounds. So I can assume this is a response to those sort of activities.
I understand that access to ESI is not a right, but a privilege, one which I’ve appreciated immensely over all the years since its creation as well as the API. I’d love to work with you to solve this rather than seeing my applications suffer from quite frankly, terrible changes like this.
But hey, maybe mine is one of the offending applications since I do have a large volume, as such I would love to try and understand what I (we) can do as 3rd party developers to ensure the ESI can be an enjoyable asset to everyone.
/c
5 Likes
This post from Chribba says it all, from one of the biggest developers, and provides some guidance on how to redesign the API based on main complaints from its users.
Problem: too many crappy requests, draining your resources.
Authenticate applications — keep the public ESI in its increasingly degraded state, but offer the “full” version behind an “applicationToken” or something…
This could be implemented by copy-pasta of current ESI at a different base URL, with just an extra mandatory header.
Application developers can migrate easily, and you can instead ban by application ID instead of IP.
Make X months CC-paid omega a requirement or something if you want. Nobody would care.
More importantly, should be relatively easy to implement on your side!
Problem: Nobody knows your “fair use policy”, with people getting banned for 30 days without notice.
Provide basic usage metrics to the application developers and an “impending ban unless you take action” warning.
You’re doing already doing the hard part — keeping stats. Just need to make the most basic ones available to the users. Or just the “action required” notification — we’d already appreciate it.
We’re all aware this is no longer an active project; but please do the minimum required for its maintenance!
sometimes you gotta make changes just to shake off all the old bad.
The game would be more fun if people had to trust one another instead of relying on scripts to make sure all the sheep are in the hurd.
Sorry for programmers; to average eve enjoyers I see this as a non issue. 7 more days for what? do a background check? damn maybe you should think twice about joining that corp anyways.
For RMTers/malfeasance I would imagine this is a boon. But the programmers and alliance leaders shouldn’t be tasked with keeping that in check anyways.
Lore wise the other did it
For people who think they can take big advantage of big data times up! Seems like to me the pros outweigh the cons. -hot take? from an electrician.
2 Likes
I’ve got 1200 nerds to manage here, can we please make this already 2nd job not be ■■■■■■■ atrocious to do.
33% price increase leads to 700% increase in authentication delays is not acceptable customer relationship management. @CCP_Zelus I want to add to the feedback vote that this is both technically technically stupid, and yet another slap to the faces of those who try to keep the boat afloat. I’ll send you my Nintendo 64 so you can upgrade the API server ffs
I suggest increasing cache to 2 days until a more professional approach is found.
Please let the community help out in this case if CCP does not have resources to allocate to this issue.
taken from reddit, an answer by CCP Swift:
https://www.reddit.com/r/Eve/comments/10duuu0/comment/j4pep8n
[…] The TL;DR for why these endpoints were taken offline is that there is a third party developer, or a series of third party devs, using AWS and requesting information from these endpoints so frequently that it puts the rest of the cluster at risk. These are on the scale of a 6,000% increase in hourly requests. […]
One way to combat this scraping of public data about everyone in eve is to make it impossible to scrape data about players without their consent (token) or any affiliation with you (corp member). Some endpoints for this already exist - but havent been utilized due to the ease of use of the public, authentication-less endpoints.
1 Like
Just because you are ignorant about why this is important does not mean it is not important.
I’m a Wormholer, in order to access our mapping tool so you can use the chain (there’s no in-game map) you need to verify your character is in corp by ESI. This is not a new thing it has existed since mapping tools have been invented. When someone joins our corp this means instead of taking 24 hours for a new corp member to see our mapper (what are we connected to, wheres the nearest highsec, where’s the PvP happening) it will now take 7 days. All this does is punish players and is entirely unacceptable.
1 Like
When someone joins our corp this means instead of taking 24 hours for a new corp member to see our mapper (what are we connected to, wheres the nearest highsec, where’s the PvP happening) it will now take 7 days. All this does is punish players and is entirely unacceptable.
/corporations/{corporation_id}/members/ exists, it has a one hour cache duration and lists you exactly this information. There is no need to use the public, non authenticated endpoint at all for this.
This post by @CCP_Swift should honestly be put into the first post of this topic so that people can better understand why this was necessary, and that CCP is, according to Swift, working on better solutions to this problem.
@CCP_Zelus
1 Like
Will tack it on!
4 Likes