Fetch Forex API With Python and Pandas

In this tutorial, I would like to narrate a simple REST API implementation for Python users to obtain live and historical forex data. We will try to understand consuming streaming and historical forex data using Python and Pandas.

A caution before we begin, you require at least an intermediate level of understanding of Python. At least the basics through watching those tutorial videos, like “Learn Python in under 10 minutes.” Again, developers experienced in other programming languages and aspirants willing to use TraderMade Forex REST API would also find this tutorial helpful.

So let us start.

Initially, you should sign-up for our API by clicking ‘Join API for Free.’ When you get your API Key, please note it in a secure place. The API Key and Python environment is what you need to practically implement this tutorial.

To further simplify, I would suggest people without a Python set-up go to Google Colab and launch a Jupyter Notebook. Readers having Python installed with them can run these programs locally.

You require installing some libraries. People using Colab do not need to install it. They can import the libraries. Beginners in Python or people willing to quickly test a code can use Collaboratory from Google, as it is appropriate for this purpose.

Calling real-time forex rates using live endpoint

import requests
  from pprint import PrettyPrinter
  pp = PrettyPrinter()

People using Google Colab can copy-paste the above code into a cell on the Google Colab Notebook. Then, you would need to hit ‘Shift+Enter’ or click the ‘Play’ button.

The cell runs; a new one is available to write the next bit of code. Let us understand the meaning of the following code before you run it. We are calling the URL, which is the endpoint. The currency appears as a string separated by commas; the API key is the unique key you received after signing up.

Remember you imported a requests library in the last cell? You will need that to call the Forex REST API. The ‘get’ function in the request library takes in a URL and a JSON parameter. Here, it is the “querystring.” We will receive a ‘response’ from the API. Then, we print the response using PrettyPrinter for a better appearance.

url = "https://marketdata.tradermade.com/api/v1/live"

  currency = "USDJPY,GBPUSD,UK100"
  api_key = "api_key"
  querystring = {"currency":currency,"api_key":api_key}

  response = requests.get(url, params=querystring)

  pp.pprint(response.json())

Thus, we have received the live rates for USDJPY, GBPUSD and FTSE100 in less than 6 lines of code. However, for analysing this data or visualising the rates better, it would be essential to arrange them in a tabular format, like DataFrame in Pandas.

At this point, I would like to introduce pandas, a vast library built for data analysis. Pandas bring a great deal; of ease to essential functions like iterating, arranging, amending, and calculating data.

Yet, for this tutorial, let us consider obtaining, saving, and bringing the data in the tabular format from the API. Let us proceed.

 import pandas as pd
  df = pd.DataFrame(response.json()["quotes"])
  df

Pandas can obtain data directly from the API. However, for better understanding, we are using requests in this tutorial. If the data you seek is not the exact data frame, it is better to use the request.

Please pass on the quotes received in your response.json into a pandas function. The pandas function converts the data into a data frame, and there it is! We get the data tabulated in just one line of code.

You can see that the pandas data frame is easy to understand. However, we are not yet done. We will further adjust it to make it simpler and more readable. You can see Nan in the instrument column. Ultimately, ask, bid, and mid is not appropriate. Similarly, we do not have a timestamp. As the next step, we will fix these.

 # if all instruments are currencies just write a # in front of line 1 and take # off line 2 in the below code. If all instruments are CFDs just write # in line 1 and 2.
  import numpy as np
  df["instrument"] = np.where(df["base_currency"].isnull(),df["instrument"],df["base_currency"]+df["quote_currency"])
  #df["instrument"] = df["base_currency"]+df["quote_currency"]
  df["timestamp"] = response.json()["timestamp"]
  df["timestamp"] = pd.to_datetime(df["timestamp"], unit="s")
  df

At this point, we will import another library, NumPy, which goes well with pandas. Then, we will alter the instrument column using the “np.where” function.

This function takes in (condition, is true, is false). Then, it creates a new column named ‘Timestamp’ and notes the timestamp received by us through API response.json. Eventually, it converts the timestamp to a readable format.

Now, you can realise why pandas are preferred; and Python is efficient. Notably, the entire code executes surprisingly faster. Here is another way:

It is possible to save your work using a single command shown here and share it via a CSV file.

df.to_csv('live_rates.csv')

Please check our REST API data documentation page for an extensive overview of our endpoints.

Using Historical Endpoints to Call Historical Forex Data

Daily Historical Data

The daily historical endpoint is quite identical to the live endpoint. In place of a timestamp, the response JSON has a date. Another difference is you get Open, High, Low, and Close prices instead of the Bid, Mid, Ask, and Quotes. As we have already defined the currency and the API Key, we need to provide the date for which we need the data.

url = "https://marketdata.tradermade.com/api/v1/historical"
  date = "2021-03-15"
  querystring = {"currency":currency,"date":date, "api_key":api_key}
  response = requests.get(url, params=querystring)
  df = pd.DataFrame.from_dict(response.json()["quotes"], orient='columns', dtype=None, columns=None)
  df["instrument"] = np.where(df["base_currency"].isnull(),df["instrument"],df["base_currency"]+df["quote_currency"])
  df["date"] = response.json()["date"]
  df["date"] = pd.to_datetime(df["date"])
  df

You realise that you can bring the data in tabular format with ease. Similarly, with the help of pandas, you can arrange the data as you need. You can also loop through the requests and get data for the desired dates.

You can leverage pandas in numerous ways. It is possible to call only one rate per currency for the other two data historical endpoints: minute_historical and hour-historical. Thus, it is essential to loop through the data.

Minute and Hour Historical

As the calls for both endpoints are similar, let us use one example for a minute historical:

fx = ["EURUSD", "USDJPY"]
  dates = ["2021-03-15-13:00"]
  array = []
  url = "https://marketdata.tradermade.com/api/v1/minute_historical"
  for i in fx:
      for date in dates: 
          querystring = {"currency":i,"date_time":date, "api_key":api_key}
          response = requests.get(url, params=querystring)
          array.append(response.json())
  df = pd.DataFrame(array)
  df

You must have seen by now that you can easily ask for various dates and currencies. It takes a little effort to pre-set some conditions and create a system to track rates and highlight some alerts based on them. We will not dig deeper into this as it is out of the scope of this tutorial. I suggest you explore pandas, a comprehensive library for data analysis.

Using Timeseries Endpoint to Call Timeseries Forex Data

Usually, while requesting historical endpoints, the requests are for a price at a particular time. Let us consider timeseries. It is utilised for charts, trend analysis, and obtaining data in nuggets. We can perform this task using pandas and would not need the requests library. We can accelerate the process of getting data.

url = "https://marketdata.tradermade.com/api/v1/timeseries?"
  currency="USDJPY"
  start_date="2021-03-01"
  end_date="2021-03-22"
  format="split"
  interval="daily"

  df =pd.read_json('https://marketdata.tradermade.com/api/v1/timeseries?currency='
                +currency+'&api_key='+api_key+'&start_date='+start_date+'&end_date='
                +end_date+'&format='+format+'&interval='+interval)
  df = pd.DataFrame(df.quotes['data'], columns=df.quotes['columns'])
  df.tail()

Pandas can use a function like read_json or read_csv to request data from an API. The forex API timeseries endpoint takes in a parameter named ‘format.’ It is simple to convert that parameter into a pandas data frame when it is set to “split.”

Then, it is practical to read the columns passing through the API. We can use the command df.tail() and print the last 5 lines of data we received. You can use the command df.head() to see the first 5 data points.

You get the data properly formatted and can easily play with it. You can also make an identical request to obtain hourly and granular data. Make some minor changes. You need to change the start_date and end_date to “YYYY-mm-dd-HH-MM.” Similarly, change the interval to hourly. Take a look at the below requests:

You can do a lot of things with the above data frame. You can create an additional column for range, % change, volatility, and rolling correlation. However, these aspects are out of the scope of this article. Kindly look at our visualisation article, which covers basic commands in pandas for volatility and visualisation.

Using Tick Sample Endpoint to Call Tick Forex Historical Rates

Historical tick data is essential for in-depth data analysis, including trading algorithms and machine learning. Finally, we will cover this critical aspect. Notably, tick data has a multidimensional use. It is practically possible to get our data in a pandas data frame in just one line of code from the REST API.

A gentle reminder before we run our code: For free users, tick data for only the last four days (excluding today) is available. Similarly, you cannot request data for more than 30 minutes in a free plan. Thus, each call will take 20 API requests. Therefore, it is advisable to save the file locally using df.to_csv(“example.csv”).

Another reminder for Google Colab users: Our start date and end date format has a gap for the tick data endpoint. You need to fill that gap with “%20.” I found it in the Google Colaboratory notebook. It is not required to do it in a Jupyter notebook or Python script.

currency="EURUSD"
# start date is 2021-03-22 14:00 but %20 is added to the gap as url dont like empty spaces (this is added automatically but Google Colab doesnt seem to like this)
  start_date="2021-03-22%2014:00"
  end_date="2021-03-22%2014:30"
  format="csv"

  url = "https://marketdata.tradermade.com/api/v1/tick_historical_sample/"+currency+"/"+start_date+"/"+end_date+"?api_key="+api_key+"&format="+format

  df =pd.read_csv(url)

  df.tail()

You have obtained almost 2000 tick historical data points for the currency pair EURUSD. Visit the TraderMade Github page to get the Jupyter notebook file.

I hope this tutorial will help beginners and seasoned programmers obtain data from the Forex REST API using Python and pandas. Your suggestions to improve this tutorial and any future tutorial you would like to see are most welcome.

TraderMade provides reliable and accurate Forex data via Forex API. You can sign up for a free API key and start exploring real-time and historical data at your fingertips.

Please go through the originally published tutorial on the TraderMade website: Forex Rates API with Python & Pandas

If you are a Python enthusiast, please refer to the following tutorials to enhance your coding and data retrieval skills:

1. Get Forex & CFD Data with Python
2. Data Visualization Python
3. Stream Data Using Python with 8 Lines of Code