Overview of My Submission

As we all know, bitcoin works like a ledger whose updates or insertions are approved between the participating nodes in the block chain (how it works). This allows us to see the chain of blocks in real time and analyze this chain backwards.

In this application, only the forward block chain will be analyzed. This decision was made because the software is conceived as a toy application to exemplify the use of a technology (redis stack) and not as a final product.

The application will analyze the blockchain, the interactions between wallets and the price of bitcoin in real time. This will be done getting data of the following APIs:

• Kraken
• Blockchain

We will provide a tiny SPA to see the processed information but is not the main purpose of the project.

Submission Category:

Microservice Mavens

Video Explainer of My Project

Language Used

Python, Javascript.

Link to Code

Additional Resources / Info

Architecture

Data ingest

Here we will comment on the functions of the services that ingest data and those entered in our system.

Market

Checks the kraken.com API periodically updates the price of btc in euros and dollars.
Update that price in the redis cache and post an event to let you know that price has been updated.

Outputs: Redis commands

• Redis cache set market:cache:USD, market:cache:EUR.
• Redis publish market:update

SET market:cache:USD {...}
PUBLISH market:update {....}

We can check cache value stored with redis insight:

Chain

Connects to the blochain.com websocket to retrieve bitcoin transactions

At this point a technical decision was made, being an example of a toy, we filter transactions less than one bitcoin so as not to have storage problems in the development of the app.

For each transaction, it calculates its value in eur/usd (consulting the cache) and publishes an event with the transaction information and the value in FIAT currency.

Outputs: Redis commands

• Redis publish transactions:new

PUBLISH transactions:new "{...}" 

Data processing

These nodes listen for events from the ingest nodes and process the information that comes to them. Each node is responsible for specific processing and exposing the data through an API.

Graph (Graph redis)

The responsibility of this node is to see the transactions between wallets, for this we use the redis graph database, so for each transaction we will create a node (if it does not exist) for the sender and receiver of the transaction, we will also create an edge with the transaction value.

Inputs: Redis commands

• Redis suscribe transactions:new

Outputs: Redis commands

• Redis graph wallets:graph

# Search for node

GRAPH.QUERY wallets:graph "CYPHER address=\"bc1q7cyrfmck2ffu2ud3rn5l5a8yv6f0chkp0zpemf\" MATCH (source:wallet {address:$address}) return source" 

# Create node
GRAPH.QUERY wallets:graph "CREATE (asotepwxth:wallet{address:\"bc1qrjkw3d0pfuklk8vd45xr78xfgdf0w396k3wavu\"})"
# Create edge
GRAPH.QUERY wallets:graph "CYPHER source=\"bc1q7cyrfmck2ffu2ud3rn5l5a8yv6f0chkp0zpemf\" destination=\"bc1qrjkw3d0pfuklk8vd45xr78xfgdf0w396k3wavu\" hash=\"a530a08e3baa1085b8fb07875467c95a6c51437ebe097f0022b2dd074033bd4f\" amount=8.92932302 MATCH (source:wallet {address:$source}) MATCH (destination:wallet {address:$destination}) CREATE (source)-[t:transaction {amount:$amount, hash:$hash}]->(destination)" "

With the following query we can see all nodes on the redis graph database.

TopK (TopK redis)

The responsibility of this node is to have a list with the top of the wallets that appear the most to see the wallets with the most activity (both sending and receiving btc), which should give us a list with wallets from SCAMs, exchanges... etc.

*Inputs: * Redis commands

• Redis suscribe transactions:new

*Outputs: * Redis commands

• Redis topK add topk:wallets:senders
• Redis topK add topk:wallets:receivers
• Redis publish topk:updated

TOPK.ADD topk:wallets:senders "bc1q7cyrfmck2ffu2ud3rn5l5a8yv6f0chkp0zpemf"

PUBLISH topk:updated "{\"senders\": [\"1GQdrgqAbkeEPUef1UpiTc4X...}"

Stats (Time series redis)

This as a responsibility to maintain the real-time statistics of the btc.
It will give us the real-time evolution of:

• The volume of transactions per minute (blockhain.com).
• The volume of btc per minute (blockchain.com).
• The quote of the btc (with the info of kraken).

Inputs: Redis commands

• Redis suscribe transactions:new
• Redis suscribe market:update

Outputs: Redis commands

• Redis timeseries add transactions:series:btc (btc per minute)
• Redis timeseries add transactions:series:tx (transactions per minute)
• Redis timeseries add btc:[USD|EUR]:bars:[close|high|low|open] (Stock)

TS.ADD transactions:series:btc "1661074336000" "8.92932302"
TS.ADD transactions:series:tx "1661074336000" "1"

TS.ADD btc:usd:bars:close "1661074356000" "21424.3"

Stream

This node will listen for redis pub/sub posts and relay them over a websocket to the presentation layer of our software.

*Inputs: * Redis commands

• Redis suscribe transactions:new
• Redis suscribe market:update

Frontend

SPA

React spa which will query an nginx that balances requests depending on the URL.

NGINX

API that exposes the application, redirect requests to each ingest node depending on the urls and the root address of the SPA.

Application overview

Dashboard

This pages provides a information of:

• Btc quote (provided by redis cache)
• Graphs of BTC/transaction volume per minute (provided by redis time series)
• Real-time transactions (provided by redis pub/sub)

As an alternate view, dashboard could show incoming BTC transactions as a graph, the result would be something like this:

Top wallets

• Wallets with more transactions (Provided by redis TopK)

Wallet detail

• Graph with btc transactions (provided by redis graph)
• information about the wallet (provided by blockchain api and cached)
• Reports to the wallet (future work iframe consult an api and redis search)

Stock

• Time series info from stock data provided by kraken and hosted in time series redis.

References

Redis

• Redis stack stocks repo
• Redis - Timeseries
• Redis - TopK
• Redis - Pub/Sub
• Redis - Graph

Backend

• Nginx reverse proxy
• Nginx reverse proxy websockets
• Fast API
• Redis python

Frontend

• React
• React - router
• React - redux
• Apache echarts

Collaborators

• mandrewcito

Future work

• Perform wallet info obtanied from bitcoin abuse with Redis search

• Check out Redis OM, client libraries for working with Redis as a multi-model database.
• Use RedisInsight to visualize your data in Redis.
• Sign up for a free Redis database.