There should be a method to incrementally update market data via the API instead of pulling the entire data set every 5 minutes. Provide a complete dump every day, and then release an update every 5 minutes against the last 5 minutes: new, updated and removed orders only.
1 Like
I can see issues with that – like, if you do your first pull at midday, you’d need to pull the original dump plus every incremental dump.
Still, the idea of only pulling in changes is a solid one. How about a flag on each order that indicates whether it has changed from the previous five minutes? Or a “last changed” timestamp, so I can check to see if the order has changed since my last data pull?
you can’t make such a change ATM . This would need to have a sorted market, by order id, so that if two market data are the same semantically then the outut is also the same.
eg if you have two orders o with volume v and price p : [o1:v1:p1, o2:v2:p2]
This is semantically the same as [o2:v2:p2, o1:v1:p1] but the diff will be non-null. so if later monolith fetches the orders and returns the second, then the path get will be changed even though the data is semantically the same.
In order to ensure a semantic- aware diff, you need to have the list of orders sorted by order id, so to change the whole path.
This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.