I wish to create a simple web application that I can put on colo hardware that pulls information directly from ESI.
I would like to start out with just being able to type in an item ID or name and get all buy/sell orders.
The only problem is, I have minimal python/dev skills! I would love to learn how to do all of this, but I need some guidance as to where to get started.
I know how ESI works, you just call from a URL and it returns data (in .json form, I think?). I am struggling to understand how to get from data raw data to a useful web application. I apologize for my lack of knowledge.
I know quite a bit of HTML and some PHP, and know my way around a lamp stack. I just need someone to guide the way
Basically, you query an URL, parse the data it gave you, and that’s it
In a more precise way, considering what you want to do (the basic way):
you have a form asking for an item ID (for the ease of explaining, i’ll take 34 for tritanium) and region ID (10000002 for The Forge)
submit form with these data
the backend (php, python, whatever) will take these IDs, put them in the market URL, which will then become https://esi.tech.ccp.is/latest/markets/10000002/orders/?datasource=tranquility&order_type=all&type_id=34
the backend call that URL, and get a json array in response of your call.
it then loop over this data to render the page
the user get the current order data for tritanium in The Forge
In python, using requests lib for example, it’ll be something like that for step 4-5
import requests
import json
response = requests.get(`https://esi.tech.ccp.is/latest/markets/10000002/orders/?datasource=tranquility&order_type=all&type_id=34')
data = json.loads(response.content)
for row in data:
# print whatever you want from the order data
That’s pretty much it. If the user have only the name of regions / items, you’ll have to get the corresponding id’s (either using SDE or ESI endpoints for this).
If you are going to use python, you can either use the examples from here (eve dev blog) or you can use EsiPy which will make your use of ESI easier (imho btw). You can find an example with EsiPy using in the doc I linked.
Edit: As for the web part, for python you have several choices. The more common are Flask and Django (each with their pro/con).
For PHP, if you go for it, it’s the same: you will just have a php page that call ESI (using PHP Curl), parsing the response and giving it back to the user.
If you have some python skills, I suggest you pick a framework to run the actual webpage (django or flask, just to name a few) and do the tutorial. Django actually has a pretty good one: https://docs.djangoproject.com/en/1.11/intro/tutorial01/
Thanks! I think I am going to go with a simple PHP page for now and just parse the response data.
I like how Fuzzmarket works, especially on the front end how he separated logic pages from the display pages. I will probably expand on that for the front-end, save myself some time.
Another question, what packages will I need specifically on an Ubuntu build to run a python/php combo? I was think build-essential and your typical lamp stack and FTP (trying to keep it light).
Again, thanks for tolerating my baby questions.
Edit: I would appreciate the dependencies for FuzzMarket, as it looks like I am going to end up running a very similary structure
If you want python, you’ll need “at least” (imho) python, python-dev, virtualenv packages. You’ll also need a web application server for web python app (like uwsgi, gunicorn…). For python dependencies, within a virtualenv you can just use pip
For php, if you have lamp, just also install composer for dependency management
Okay, to be honest I have no earthly idea what I am doing (Domain sysadmin here, never really learned any of this stuff)
Lets start with the script he uses to aggregate the data from ESI.
from sqlalchemy import create_engine, Column, MetaData, Table, Index
from sqlalchemy import Integer, String, Text, Float, Boolean, BigInteger, Numeric, SmallInteger, DateTime
import time
import requests
from requests_futures.sessions import FuturesSession
import requests_futures
from concurrent.futures import as_completed
import datetime
import csv
import time
import sys
import re
import pandas
import numpy
import redis
import json
import os
import shutil
import base64
What is happening here? I get that python is grabbing something, but what is it grabbing, why is it grabbing it, and where is it grabbing it from? That would go a long way to help me understand the basics of how this works.
I assume that this script is heavily dependent on whatever is contained in the stuff that python is grabbing?
According to what you pasts, this only includes libs (unless you forgot to paste something :p)
From what I see here, it requires a few external libs (pandas, numpy, redis, shutil, requests, sqlalchemy).
import <module> and from <package> import <module> are just ways to include libs within a python module (a module is a “file”, a package is a collection of module (a folder containing python files)
Tbh, if you want to learn python try to find / read / follow some tutorials or courses. It’s not hardest language to learn. A few links:
Despite using composer to ensure I have all the dependencies for FuzzMarket, it still simply will not work. Either FuzzySteve left something out or I left something out, but this should be fairly simple yet it refuses to cooperate.
