Visualizing Cryptocurrency Data With AWS Serverless Technology

We have about 20,000 cryptocurrencies, with over 9,000 of them actively trading. Similarly, we have over 4,000 exchanges, with over 500 of them being active. The CoinMarketCap website has all of this information. However, I wanted to be able to analyse cryptocurrency data beyond just how many cryptocurrencies are active. To accomplish it, I ended up leveraging AWS Serverless services.

This blog post describes how I used AWS serverless services to visualize cryptocurrency data using AWS Quicksight. So, if someone wants to do something similar, they know where to start. To achieve my goal, I used the following AWS services:

• Lambda Functions

• S3

• AWS EventBridge

• Step Function

• QuickSight

Lambda Functions

In this use scenario, I have three lambda functions. Each lambda function runs a given function and saves the result to an S3 bucket. One of the lambda functions is shown below, which uses the Coinmarketcap map API and saves the response to an S3 bucket.

from os import path
import json, boto3, sys, uuid

# from botocore.vendored import requests

from requests import Request, Session
from requests.exceptions import ConnectionError, Timeout, TooManyRedirects

s3_client = boto3.client('s3')

def lambda_handler(event, context):
    url = "https://pro-api.coinmarketcap.com/v1/cryptocurrency/map"
    headers = {
        'Accepts': 'application/json',
        'X-CMC_PRO_API_KEY': 'xxxxxxxx',
    }

    session = Session()
    session.headers.update(headers)

    try:
        response = session.get(url)
        data=json.loads(response.text)
        json_body=json.dumps(data['data'])
        AWS_BUCKET_NAME='coinmarketcap-data'
        S3 = boto3.resource("s3")
        bucket_name = S3.Bucket(AWS_BUCKET_NAME)
        bucket_name.put_object(
            ContentType='application/json',
            Key="map-file.json",
            Body=json_body,
            )

        return {'statusCode': 200,'body': json.dumps('file is created')}
    except (ConnectionError, Timeout, TooManyRedirects) as e:
        print(e)

S3

Lambda functions write their data to an S3 bucket, data in S3 is then later consumed by Quicksight for visualization.

Here is an extract from the map data file stored in S3 bucket.

AWS Step Function

The step function is used to connect Lambda functions into a serverless workflow.

AWS EventBridge

For this specific use case, the EventBridge is used to schedule to run the Step function every 24 hours.

AWS QuickSight

Finally AWS QuickSight is used to load the data from S3 bucket and visualize it as shown below.

The whole AWS serverless architecture for this use case looks like this:

Thanks for reading!

If you enjoyed this article feel free to share it on social media 🙂

Say Hello on: Linkedin | Twitter | Polywork

Blogging: Dev | Hashnode

Github: hseera