I’ve recently created a new service that I think could be useful for the third-party developer community, and wanted to share it with you and ask for your feedback.
The service is an API for the SDE that provides the data via REST, which I believe could be a more convenient alternative to Steve’s SDE complements in SQL form (which are amazing!) in some instances.
For example, if you’re developing web-based tools that don’t really need/have a database, this API could simplify things a lot.
You can generate the data using the tool (tested using macOS/Linux; Windows not tested — use at your own risk!):
npx @jitaspace/sde-converter generate
I’ll open source the code soon, but will happily give access to it upon request.
Let me know what you think!
Some schema properties should be dictionaries, not objects with all available values enumerated: for example, the field corporationTrades in the NPCCorporation. This results in an overly complex schema that currently is about 10MB in size.
The UI has issues loading large responses (/inventory/items, inventory/names, /inventory/positions). Won’t fix for now, as the UI is not really the main way this data is intended to be consumed, nor am I able to offer pagination with the current format.
Generating all files and uploading them took me about 5 hours.
It should be feasible to have it update within 6 hours after the SDE changes to have it reflected in the API. This is not implemented yet, though! Updates are triggered manually, so expect 24h delays.
For your use-case I have some other tool planned in which I’d notify you (via webhooks) that the SDE changed, and give you a JSON with only the changes.
This could avoid the very unnecessary (IMHO) “ping API every X minutes and download everything” pull-based messaging the whole EVE 3rd party development currently relies on.
Sweet. This would be the single most & incredibly valuable reason for me to use the API. Instead of me attempting to manually trigger updates in a manner exactly as you described. Checking the page, downloading, unzipping, moving, generating updates, etc.
One additional concern is that I’m not (currently) behind cloudflare, so when I tried to use other SDE providers my traffic gets rejected. Not really sure why but probably because of bot related protections. I punted on figuring that out for now, but I would need to work through any similar issues before becoming an adopter.
So, since the checksum file is a really tiny file (32 bytes), you can just make periodical HTTP calls to its location, and parse the “Last-Modified” header from response. Probably most of HTTP-libraries from most of programming languages are able to get a header value from a server response, so it shouldn’t be a big challenge.