Anyone who has planned to purchase tokens or other digital assets on the blockchain has encountered the difficulty of researching such assets. Blockchains are poorly adapted for collecting analytics - nodes and lightnodes provide information only on a specific block of the network, so you have to use blockchain indexers services or API of services running on the blockchain.
As decentralized exchanges strive to have a presence on platforms like coingecko, they create standardized APIs, often making them open source. Such APIs allow you to get a lot of information related to token trading.
Therefore, in this article we will put together a dashboard for tokens on the TON blockchain and see how easy it is to get data from DEX, but in the end we will talk about the problems of this method of data collection. I hope such a simple tutorial will make the steps in the TON blockchain clear and enjoyable.
Before we get into data collection, a few disclaimers:
the code in the tutorial is as simple as possible so that it can be read diagonally
tokens are the standard for fungible tokens on TON
Helper functions
Before calling the API handle, let's write helper functions that will help us with the parameters for the called APIs, namely, determine the dates and put them in the format we need:
def now_utc():
return datetime.datetime.utcnow().strftime('%Y-%m-%dT%H:%M:%S')
def thirty_days_utc():
return (datetime.datetime.utcnow() - datetime.timedelta(days=30)).strftime('%Y-%m-%dT%H:%M:%S')
def seven_days_utc():
return (datetime.datetime.utcnow() - datetime.timedelta(days=7)).strftime('%Y-%m-%dT%H:%M:%S')
def day_utc():
return (datetime.datetime.utcnow() - datetime.timedelta(days=1)).strftime('%Y-%m-%dT%H:%M:%S')
now_utc()
Let’s call API GET https://api.ston.fi/v1/stats/pool
payload = {'since': day_utc(), 'until': now_utc()}
r = requests.get('https://api.ston.fi/v1/stats/pool', params=payload)
r.json()['stats'][0]
Let's get various pool parameters:
Now let's try to collect the information we need
Take fields
For our dashboard we will need:
Token name and symbol
Latest Jetton price for the selected period
Volume in TON
Swap pool link
Let's select this data using our API:
def take_pool_stats():
temp_arr=[]
payload = {'since': day_utc(), 'until': now_utc()}
# stonfi pools stats
r = requests.get('https://api.ston.fi/v1/stats/pool', params=payload)
resp = r.json()['stats']
for jetton in resp:
temp_dict = {'coin': jetton['base_name'],'pair': jetton['base_symbol']+'/'+jetton['quote_symbol'],'url': jetton['url'],'price_ton': jetton['last_price'],'volume_ton': jetton['quote_volume']}
temp_arr.append(temp_dict)
return temp_arr
take_pool_stats()
We will see these jsons:
{'coin': 'Chow Chow',
'pair': 'CHOW/TON',
'url': 'https://app.ston.fi/swap?ft=EQBtWFPgVknfzu6xaVcRBbNP3h_6UJ_xe29sVkiFTyPiv2bq&tt=EQCM3B12QK1e4yZSf8GtBRT0aLMNyEsBc_DhVfRRtOEffLez',
'price_ton': '0.010646867',
'volume_ton': '0.000000000'},
{'coin': 'Tonald Duck',
'pair': 'TDUCK/TON',
'url': 'https://app.ston.fi/swap?ft=EQBUGgcu-h4SqMh5hrentq4vE59tBRRfrYE3H_0s6D_1Xzsa&tt=EQCM3B12QK1e4yZSf8GtBRT0aLMNyEsBc_DhVfRRtOEffLez',
'price_ton': '0.000000363',
'volume_ton': '0.000000000'},
Don’t be surprised that the volume is zero - anyone can create a pool, so not all of them are in great demand.
Step back
It is inconvenient to look at prices in TON, so to represent prices in dollars, we will get the current value of TON/USD. To do this, we will use tonapi.io for our task, free limits are quite enough:
def take_ton_price():
resp = requests.get('https://tonapi.io/v2/rates?tokens=ton¤cies=usd')
# Ex: {"rates":{"TON":{"prices":{"USD":2.1215},"diff_24h":{"USD":"+0.85%"},"diff_7d":{"USD":"-6.04%"},"diff_30d":{"USD":"+2.75%"}}}}
return resp.json()["rates"]["TON"]["prices"]["USD"]
take_ton_price()
Let's remove the unnecessary
We use the resulting exchange rate to recalculate volume and price. As mentioned above, we give away all pools to the API, this means that unclaimed pools whose volume is zero can get there. As well as pools between tokens, which are not interesting to us, since they do not display the price and trading volume relative to TON or the dollar. Respectively:
choose pools with TON
we will remove pools in which the last price is zero
we will remove pools where the trading volume for a period is less than $1000
And immediately sort by volume:
def take_pool_stats():
temp_arr=[]
payload = {'since': day_utc(), 'until': now_utc()}
# stonfi pools stats
r = requests.get('https://api.ston.fi/v1/stats/pool', params=payload)
resp = r.json()['stats']
# ton_usd
ton_usd = take_ton_price()
for jetton in resp:
temp_dict = {'coin': jetton['base_name'],'pair': jetton['base_symbol']+'/'+jetton['quote_symbol'],'url': jetton['url'],'price_ton': jetton['last_price'],'volume_ton': jetton['quote_volume'], 'price_usd': round(float(jetton['last_price'])*ton_usd,2),'volume_usd': round(float(jetton['quote_volume'])*ton_usd,2)}
if(jetton['quote_symbol']=='TON' and int(float(jetton['last_price']) != 0) and int(float(jetton['quote_volume']) != 0) and int(temp_dict['volume_usd']) > 1000):
temp_arr.append(temp_dict)
return sorted(temp_arr, key=lambda d: d['volume_usd'],reverse=True)
# Coin - base_name
# Pair - base_symbol/quote_symbol url
# Price - last_price * TON in USD price
take_pool_stats()
As result we get json like this:
{'coin': 'STON',
'pair': 'STON/TON',
'url': 'https://app.ston.fi/swap?ft=EQA2kCVNwVsil2EM2mB0SkXytxCqQjS4mttjDpnXmwG9T6bO&tt=EQCM3B12QK1e4yZSf8GtBRT0aLMNyEsBc_DhVfRRtOEffLez',
'price_ton': '2.220362684',
'volume_ton': '74651.187439449',
'price_usd': 5.89,
'volume_usd': 197926.43},
Percentage of total
It is convenient to consider the volume from the total volume on the market, so let’s calculate the amount of volume and add a percentage. (In the code you can see that I left as many as 10 digits after the decimal point, this is due to the fact that TON recently took an initiative to add liquidity to the pools, due to which the volumes were greatly “blurred”. In a normal situation, 4 characters would be enough)
.
# Percentage from total
def take_pool_stats(payloa):
temp_arr=[]
# stonfi pools stats
r = requests.get('https://api.ston.fi/v1/stats/pool', params=payload)
resp = r.json()['stats']
# ton_usd
ton_usd = take_ton_price()
# for volume percentage
all_volume = sum(float(item['quote_volume']) for item in resp)
for jetton in resp:
temp_dict = {'coin': jetton['base_name'],'pair': jetton['base_symbol']+'/'+jetton['quote_symbol'],'url': jetton['url'],'price_ton': jetton['last_price'],'volume_ton': jetton['quote_volume'], 'price_usd': round(float(jetton['last_price'])*ton_usd,4),'volume_usd': round(float(jetton['quote_volume'])*ton_usd,2),"volume_perc": round((float(jetton['quote_volume'])/all_volume)*100,10)}
if(jetton['quote_symbol']=='TON' and int(float(jetton['last_price']) != 0) and int(float(jetton['quote_volume']) != 0) and int(temp_dict['volume_usd']) > 1000):
temp_arr.append(temp_dict)
return sorted(temp_arr, key=lambda d: d['volume_usd'],reverse=True)
payload = {'since': day_utc(), 'until': now_utc()}
take_pool_stats(payload)
Let's put the resulting list in a dataframe of the Pandas library for ease of display and get:
Using such a simple example, you can test your hypotheses related to tokens, for example, by looking at the dynamics over several days or by considering different periods. But almost any study leads to the need for detailed information on pools. After all, it is perfectly clear that you can increase the volume from a couple of wallets, thus accelerating the token. To do this, you need to collect information on operations.
Collecting information on operations
The StoneFi API has a separate method that returns operations. Conveniently, for each operation the pool in which the operation was performed is registered. We will use this to count the number of swaps by pool and see how many unique wallets are among these operations.
First, let’s get the operations for the period:
def take_operations(payload):
r = requests.get('https://api.ston.fi/v1/stats/operations', params=payload)
return r.json()["operations"]
Operations in pools are different, there are swaps, when users exchange tokens and TON, there is the addition of liquidity to pools, possible operations may differ from exchange to exchange. For our simple example, we will only take swaps.
Let’s assemble a function that, for each pool we select, will return the number of swaps and the number of unique wallets that made these swaps.
def count_pool_operations(operations,addr_str):
pool_oper_list = list(filter(lambda person: person['operation']['pool_address'] == addr_str, operations))
unique_counts = collections.Counter(e['operation']['destination_wallet_address'] for e in pool_oper_list )
return len(pool_oper_list),len(set(unique_counts))
Let's add this information to the dashboard:
def take_pool_stats(payload):
temp_arr=[]
# stonfi pools stats
r = requests.get('https://api.ston.fi/v1/stats/pool', params=payload)
resp = r.json()['stats']
# ton_usd
ton_usd = take_ton_price()
# for volume percentage
all_volume = sum(float(item['quote_volume']) for item in resp)
# take operations
payload['op_type'] = 'Swap'
operations = take_operations(payload)
for jetton in resp:
temp_dict = {'pool_address': jetton['pool_address'],'coin': jetton['base_name'],'pair': jetton['base_symbol']+'/'+jetton['quote_symbol'],'url': jetton['url'],'price_ton': jetton['last_price'],'volume_ton': jetton['quote_volume'], 'price_usd': round(float(jetton['last_price'])*ton_usd,4),'volume_usd': round(float(jetton['quote_volume'])*ton_usd,2),"volume_perc": round((float(jetton['quote_volume'])/all_volume)*100,2)}
if(jetton['quote_symbol']=='TON' and int(float(jetton['last_price']) != 0) and int(float(jetton['quote_volume']) != 0) and int(temp_dict['volume_usd']) > 1000):
temp_pool_count = count_pool_operations(operations,temp_dict['pool_address'])
temp_dict["count_swaps"] = temp_pool_count[0]
temp_dict["count_unique"] = temp_pool_count[1]
temp_arr.append(temp_dict)
return sorted(temp_arr, key=lambda d: d['volume_usd'],reverse=True)
payload = {'since': day_utc(), 'until': now_utc()}
df = pd.DataFrame(take_pool_stats(payload))
Result:
A spoon of tar
The example we collected above shows how easy it is to collect data on pools and operations, but there are also problems.
Since the API’s task is to provide statistical data, based on the price of tokens we get the last price for the period, which is not very correct from an analytical point of view. Therefore, if the price of a token for a period is important to us, it will be important to calculate the time-weighted price, which will lead to an increase in the number of API requests.
The second problem with such data collection is the time it takes to complete one request; the APIs of decentralized exchanges are not adapted for deep queries, so if you are interested in some kind of live analytics, then the only way out is to collect data directly from smart contracts - calling get methods .
Conclusion
I like the TON blockchain for its technical elegance; at least it’s not another copy of Ethereum, which is being overclocked with the help of a lot of capital without looking back, and in general why the user needs it. If you want to learn more about the TON blockchain, I have open source lessons that will teach you how to create full-fledged applications on TON.
https://github.com/romanovichim/TonFunClessons_Eng
I post new tutorials and data analytics here:
Top comments (0)