Importing data from ESI

Industry Gameplay Center Science & Industry
1

Hello there,

Quick question as i’m a little bit blocked trying to import data from the market.

I’m willing to import the sell order from Amarr to compare with my BPO’s what is worth producing or not …

If i take the data from this October 14 there are 145 pages of sell orders

My question is do i really have to make 145 queries like the following to get the data or is there a shorter way ?

https://esi.evetech.net/latest/markets/10000043/orders/?datasource=tranquility&order_type=sell&page=10

i was thinking to do 1 query per BPO i have but this will be more than 300 queries too …

Any idea to help maybe ?

2

I’d suggest looking at https://market.fuzzwork.co.uk/api/

It’s updated from ESI once every 30 minutes or so.

You’d be most interested in the percentile costs. They’re based of buying the lowest 5% of the market. (to reduce the higher and lower outliers.)

1 Like
3

The answer is yes, you need to have all pages.

The data in the pages are not sorted contractually, so one item may be spread all over the pages.

get first page, extract data, get all other pages, extract data, process
Use parallel calls to fetch all the other pages.

On the other hand, you can limit the market to one typeid
https://esi.evetech.net/latest/#!/Market/get_markets_region_id_orders
you will still need to fetch all the pages.

4

ok i was waiting this answer Anderson.

Is there a way to make excel ask from page 1 to X in 1 routine like in excel + VBA or so … i would be interested in learning that way like creating a function ( where i’m at level 0 atm )

Steve thank you for your reply but i did tried that option but those 5% volatility on each item make just bad results, i want to find another way to get the instant results when i ask them without that 5% insecurity on each item.

5

I don’t think it’s easily doable.

  • the information is stored in the headers so you need to use a library that allows to look at header.
  • you also should use cache to avoid requesting the same information several times. The cache should be managed by YOUR code (to avoid fetching the same data several times) and also you should handle the headers for cache managed by the server (to fetch the data once they are expired).
  • finally you need to iterate over the pages, and for each, translate the data in meaningful structure, then to merge all the structures retrieved, to finally sort the orders by your convenience (typically SO are sorted by price increasing, and BO are sorted by price decreasing).

When you are at this level of management, you may as well use a dedicated language to create the data (eg python), then import that data in excel. eg our python can create a csv format of itemname - BO - SO
Also I guess people already created libs to have this behaviour, typically to fetch all the orders for a list of item and aggregate them.

BTW note that in case you want several items, it’s better to just fetch all the market orders and then sort the orders by itemid, order type, price ; rather than to fetch each item individually.

6

VBA is very very bad at handling json, I’m afraid.

I mean, it’s possible, but I really don’t recommend it.

https://www.fuzzwork.co.uk/2014/07/26/excel-and-crest/ is for the older CREST, rather than ESI, but the basic premise holds.

Now, what may well work better is writing something in an alternate language to download the price data, and store it in a database, or output a CSV which you can load into your sheet.

My personal preference is python, but most modern languages will work well enough.

7

Try IPH.

http://eveiph.github.io/

8

Why would he try a random name that someone gave him without explanation ?
Also though it may be related, this is not an answer to his question.

1 Like
9

What he wants to do can all be done in IPH. If he wants to build his own thing he can. Just offering another option if he doesn’t want to spend weeks building something.

10

(Reinventing the wheel isn’t efficient. But it can be fun or interesting at times. Or good to do when the wheel doesn’t quite do what you want it to)

11

… and remember, citadel orders are visible only if they have range more that “station”
[so current API is half useless and sometimes it is deceitful]

12

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.