A simple yet efficient way to extract different types of market data with stock APIs
Let’s face it, finding accurate and reliable stock data at an affordable cost is hard (merely impossible). The only way to access such data is to buy them directly from the exchanges for a hefty amount of money. But recently, I discovered a financial data API provider that offers all sorts of market information ranging from historical data to economic data at a low cost. It’s none other than EOD Historical Data. In this article, we will first have a brief look at who they actually are then we’ll dive deep into the APIs they offer to extract some useful information using Python.
Note: This article is intensely based on the APIs provided by EOD Historical Data. So to follow up with the upcoming content without any distress, make sure that you have an account for yourself as it enables you to access your private API key (essential for extracting data using APIs). You can create an account using the link here. Also, not all APIs mentioned in this article are offered for free but a subscription is needed. View the pricing of all subscriptions here and choose what best fits you.
A bit of history and stats
EOD Historical Data is an organization dedicated to providing a broad range of financial data APIs with precision. To me personally, their numbers are pretty impressive and convinced me to avail of their services. They have 30 years of financial data with a grand total of more than 760 million data points which consists of 70+ stock exchanges around the world, 150000+ tickers, 20000+ ETFs, 600+ indices, and 1100+ forex pairs. Now, let’s move on to the technical part of this article where we will utilize a bunch of APIs using Python and hopefully extract some interesting data.
Extracting Financial data using Python
As I said before, EOD Historical Data offers its users a plethora of APIs. So, we will break down this technical part into different categories to make sure nothing is left out. This section of the article is categorized into the following topics:
1. Fundamental Information
2. Historical Data
3. Real-Time Market Data
4. Technical Indicators
5. Intraday Data
Each category comprises the code to extract the respective data using a set of APIs. Before directly proceeding with the categories, we should first import all the required packages into our python environment. In this article, there are three important packages involved which are Pandas for dealing with data, Requests to make API calls, and Datetime for date manipulations. To import these packages, follow the code below:
import pandas as pd
import requests
from datetime import datetime
We have now successfully imported the packages into our environment. Also, make sure that you have installed the packages before importing them. Without wasting any time, let’s jump into the first category where will extract the fundamental information of stocks.
Fundamental Information
When compared to the other APIs out there, EOD Historical Data’s API for Fundamental Information offers more comprehensive data ranging from general details about the stock to earnings and financial reports. This makes it easy for researchers since they don’t have to rely upon various APIs for different types of information. Now, we can easily interact with the fundamental information API in Python and extract the data using the following code:
def extract_fundamentals(symbol):
api_key = 'YOUR API KEY'
url = f'https://eodhistoricaldata.com/api/fundamentals/{symbol}?api_token={api_key}'
data = requests.get(url).json()
return data
tsla_fundamentals = extract_fundamentals('TSLA')
In the above code, we are defining a function named extract_fundamentals which takes the symbol of the stock as the parameter. Inside the function, we are first storing the API key followed by the URL in their respective variables. Then, using the get function provided by the Requests package, we are calling the API and stored the data. Finally, we are returning the extracted data and called the created function to access Tesla’s fundamental stock information. The output is not represented given its massive size. We have successfully completed the first category of this section. Time to proceed to the next one!
Historical Data
The most common yet useful data used among various researchers for different types of purposes is the end-of-day or the so-called historical data of stock. The historical data provided by EOD Historical Data can be classified into two types which are CSV format, and JSON format.
1. CSV format
It is known that the response we get from an API is in JSON format. Now, this can be a bit harder for beginners because to work with the extracted data it is essential to go through data preprocessing and manipulation steps. But with the option to opt for a different response format, which is the CSV format, the process is much simplified. Here is the python code to extract a stock’s historical data in a CSV format:
def extract_historical_csvdata(symbol, start_date, end_date):
api_key = 'YOUR API KEY'
url = f'https://eodhistoricaldata.com/api/eod/{symbol}?from={start_date}&to={end_date}&api_token={api_key}'
df = pd.read_csv(url).set_index('Date')
return df
aapl = extract_historical_csvdata('AAPL', '2021-01-01', '2021-10-01')[:-1]
aapl.tail()
The structure of the code is very similar to the code we saw in the previous category but takes some extra parameters and the API URL changes a little. The output of the above code is a Pandas dataframe which contains the historical data of Apple from the beginning of 2021 to October 10 of the same year. The dataframe would look like this:
2. JSON format
To people who love to build huge applications, data in the form of JSON is essential. Though the default output format of EOD Historical Data’s end-of-day API is CSV, it doesn't take much to convert it to JSON. All we need to do is just add another special parameter fmt=json to the query and everything else will be taken care of! The final Python code would look like this:
def extract_historical_jsondata(symbol, start_date, end_date):
api_key = 'YOUR API KEY'
url = f'https://eodhistoricaldata.com/api/eod/{symbol}?api_token={api_key}&from={start_date}&to={end_date}&fmt=json'
raw_df = requests.get(url).json()
df = pd.DataFrame(raw_df).set_index('date')
return df
msft = extract_historical_jsondata('MSFT', '2021-01-01', '2021-10-01')
msft.tail()
The above code is almost similar to the code for the CSV format but has a special parameter fmt=json at the end of the API URL. The output is not represented as it is identical to the previous one. For all the forthcoming categories, both CSV and JSON formats are available but in this article, we will be focusing only on the JSON format. Let’s now hop on to the next category!
Real-Time Market Data
Stock traders are heavily relied upon real-time market data to proceed with their actions, either buying or selling. Accessing such crucial data is made easy with EOD Historical Data’s yet another efficient API. The code to extract the real-time market data using Python would look like this:
def extract_realtime(symbol):
api_key = 'YOUR API KEY'
url = f'https://eodhistoricaldata.com/api/real-time/{symbol}?api_token={api_key}&fmt=json'
data = requests.get(url).json()
data['timestamp'] = datetime.fromtimestamp(data['timestamp'])
del data['gmtoffset']
return data
googl_realtime = extract_realtime('GOOGL')
for i in googl_realtime.keys():
print(f'{i}: {googl_realtime[i]},')
Once again, the structure and idea of the code are more or less similar to the code we saw before in the previous categories but with a change in the API URL. Along with extracting the data, we are also doing some data manipulations inside the function just to make sure the end result is clear enough to be understood by anyone. The output of the above code is a series of values which are the real-time information of a particular stock and it looks like this:
code: GOOGL.US,
timestamp: 2021-10-05 01:30:00,
open: 2719.21,
high: 2719.21,
low: 2621.27,
close: 2673.1899,
volume: 2371139,
previousClose: 2730.8601,
change: -57.6702,
change_p: -2.1118
It is important to note that the data has a slight delay of 15–20 minutes from the market. That’s all about this category. Let’s proceed with the next category of this section where we will extract technical indicators of any given stock.
Technical Indicators
Technical indicators are the new trend around the market research arena given its efficiency and ease of learning. Like how feasibly we were able to extract data in the previous topics, extracting technical indicators is no such exception. Follow the code below to extract the Simple Moving Average (SMA) of any given stock using Python:
def extract_technical(symbol, function, period, start_date, end_date):
api_key = 'YOUR API KEY'
url = f'https://eodhistoricaldata.com/api/technical/{symbol}?api_token={api_key}&from={start_date}&to={end_date}&fmt=json&function={function}&period={period}'
raw_df = requests.get(url).json()
df = pd.DataFrame(raw_df)
return df
aapl_sma10 = extract_technical('AAPL', 'sma', 10, '2020-01-01', '2021-10-01').set_index('date').rename(columns = {'sma':'sma10'})
aapl_sma14 = extract_technical('AAPL', 'sma', 14, '2020-01-01', '2021-10-01').set_index('date').rename(columns = {'sma':'sma14'})
aapl_sma21 = extract_technical('AAPL', 'sma', 21, '2020-01-01', '2021-10-01').set_index('date').rename(columns = {'sma':'sma21'})
aapl_sma50 = extract_technical('AAPL', 'sma', 50, '2020-01-01', '2021-10-01').set_index('date').rename(columns = {'sma':'sma50'})
aapl_sma100 = extract_technical('AAPL', 'sma', 100, '2020-01-01', '2021-10-01').set_index('date').rename(columns = {'sma':'sma100'})
aapl_sma200 = extract_technical('AAPL', 'sma', 200, '2020-01-01', '2021-10-01').set_index('date').rename(columns = {'sma':'sma200'})
aapl_sma = pd.concat([aapl_sma10, aapl_sma14, aapl_sma21, aapl_sma50, aapl_sma100, aapl_sma200], axis = 1)
aapl_sma.tail()
In the above code, we are defining a function named extract_technical which takes a stock’s symbol, the statistics we want to extract, the lookback period, and the start and end date of the dataframe. Inside the function, like how we did before, defined two variables to store the API key and the URL, and using the get function we are extracting the data, converting it to a Pandas dataframe, and finally, returned the final result. The following lines of the code carry less importance than the ones we just discussed. The idea behind the rest of the code is to use the created function to extract Apple’s SMA with various lookback periods. The final dataframe or the output looks like this:
EOD Historical Data’s API for technical indicators is not just limited to SMA but comprises tons and tons of them ranging from basic to advanced indicators. The APIs are also designed in such a fashion where we could customize a lot of parameters to attain the desired indicator feasibly and seamlessly. With that being said, let’s move to the next and the final category!
Intraday Data
Like real-time data, intraday data also plays a vital role in a day trader’s life. It’s as simple as it is to extract the intraday data of any particular using EOD Historical Data’s API. The python code to extract the data would look like this:
def extract_intraday(symbol):
api_key = 'YOUR API KEY'
url = f'https://eodhistoricaldata.com/api/intraday/{symbol}?api_token={api_key}&interval=5m&fmt=json'
raw_df = requests.get(url).json()
df = pd.DataFrame(raw_df).iloc[:,2:].set_index('datetime')
return df
twtr_intraday = extract_intraday('TWTR')
twtr_intraday.tail()
The above code just like the previously discussed ones starts with defining a function named extract_intraday which takes the stock’s symbol as the parameter. Inside the function, we are doing all similar activities like storing the API key and the URL, making API calls, extracting the data and converting it to a Pandas dataframe, and finally, calling the function to access the intraday data of a particular stock. The end result is a dataframe that looks like this:
One important thing to be noted is that the days between the start and end date should not exceed 100 days. Also, this API can be customized further by changing the interval time between the data points from 5 minutes (default) to 1 minute. That’s everything you need to know about this particular API!
Final Thoughts!
So far, we have discovered five important APIs which are offered by EOD Historical Data and there are a lot more to be discovered. Personally, I feel there are three important advantages of using EOD Historical Data’s financial APIs.
The first is, the data is accurate than anything I’ve ever seen. Throughout my life, I’m confronted with tons of tons of APIs to extract market data, and honestly, I cannot find any other resources other than EOD Historical Data with such precision and accuracy.
Secondly, the ease of using the APIs is great. Usually, the JSON output we get from calling an API is clumsy and consumes some time to get cleaned. I had frustration with some of the APIs I’ve used before as it takes a millennium to make them usable but EOD Historical Data’s APIs’ JSON output is far cleaner which requires less to zero data preprocessing or cleaning processes. Not to mention, they also offer an option to opt for CSV form of output which really comes into handy.
Finally, great documentation for all APIs. It’s important to have great documentation to easily follow up with the APIs being provided. Having great services go obsolete if there is no well-directed documentation for them. EOD Historical Data strongly follows this principle and maintains one of the best documentation for each and every API which includes clean code blocks with live examples and outputs.
Overall, EOD Historical Data is a great resource to start with if you’re a beginner, and also a wise option to opt-in for advanced users. Hope you learned something new and useful from this article. If you want to support me and my work, you can do that by availing of an EOD Historical Data subscription plan through the link here (which helps me generating some commission). Thank you very much for reading this article.
Good One Nikhil