first I’d like to thank CCP for the new and wonderful ESI. I still remember my first steps with the EVE database dumps, which we had to download and which some of us first had to convert from a MS SQL Server data dumps into MySQL data dumps before we were happy, only to find ourselves with missing or outdated data every now and then. So now I’m excited to use the new ESI and it’s been working nicely for me so far. Thank you!
Still, there are always a few questions open and the most easiest to answer should be this:
1.) Why don’t I see Citadel prices when I query for market prices with /markets/<region_id>/orders/… ? - I’m assuming I will have to authorize my access first, but I’m wondering why this is and why the returned data doesn’t include these public orders as seen by any player in-game?
2.) This is a technical question and about the use of the Naggle algorithm. As of curl 7.50.2 the option TCP_NODELAY is set by default, which turns off the Naggle algorithm. This may seem like a good thing to do in general, but I was wondering in the case of ESI and when making multiple requests simultaneously if it might not be better to keep using the Naggle algorithm, because all requests go to the same server. Meaning, I’d rather be sending multiple requests in a few, large packets than to be picking away at CCP’s server with many, tiny packets.
3.) Does CCP’s ESI server support TCP Fast Open ? Curl supports it, but it doesn’t enable it by default and I would like to use it, as do most browsers these days, too.
4.) Somewhat similar to the above question is the use of SSL/TLS False Start. Curl supports it, but it has to be set as an option. Does the ESI server allow for this?
5.) I’ve seen in a few people’s code that they’ve disabled SSL verification or offer it as an option. Is this a wise thing to do or am I inviting trouble in?
Thanks in advance for anyone answering my questions or just sharing their insights.
It doesn’t answer my questions, but I did try HTTP/2 with curl, and it only made things slower for me. I didn’t benchmark too much as I don’t want to stress ESI for no good reason, but enabling HTTP/2 added 50% more time to all of my test queries.
I’m using nghttp2-1.31 together with curl-7.58.0 and php-7.2.3. Enabled with:
does it directly backfire on me. It’s not that it didn’t change anything or only immeasurably, but that it quite noticeably slows things down that I’ve stopped using it.
Something is likely very wrong if it slowed down. To get all market data you will need to do hundreds of requests. One call per structure plus one per region. With http2 you can launch them concurrently across one connection saving much overhead.
I believe I’ve figured it out. There doesn’t seem to be anything wrong, but it appears to have something to do with how CURL handles multiple requests. It seems to be an optimization within CURL itself.
CURL uses pipe-lining for HTTP/1.1 just as it does for HTTP/2, but it also opens multiple TCP/IP connections to the same host. It then doesn’t strictly pipe-line the data, but tries to use multiple connections first. This causes the requests when using HTTP/1.1 to multiplex similar to HTTP/2 does over a single connection, only does the multiplexing take place not within one single TCP/IP connection, but it takes place on the IP level in form of the multiple TCP/IP connections.
As a result are multiple requests with CURL over HTTP/1.1 about as fast as if these had been send over an HTTP/2 connection. And because HTTP/2 does a bit more than HTTP/1.1 is it adding a bit of overhead to the data, and so can lead to slower response times.
This does not mean HTTP/2 is always slower. It just isn’t always faster. As soon as the number of requests goes above the number of open TCP/IP connections does the advantage of HTTP/2 show.