This article is a combination of two things that I love to do. The first one is what I started doing when I was in Cuba. I used to scrape the world's websites, mostly in Python. Using libraries like Beautiful Soup and Requests, I used to fetch a lot of data. I really love to do that, it's been a while since I haven't done it, so it is time to practice that again. On the other hand, recently I'm starting to learn more about blockchain. Now that I'm unemployed I have free time to do it, which is awesome. Specifically I've been focusing on the TON(Telegram Open Network) blockchain.
Today I'm going to share with you a way to scrape Fragment Marketplace, to look for the cheapest available Anonymous Numbers and to buy them, at least some of them. From the scraping we are going to get information about the numbers, and we are going to buy them interacting with the smart contract associated with this NFT. The source of this smart contract is public and can be found here. Let's go then.
Source code
The source code for this script can be found in Gealber/blog_recipes/fragment_buyer
Goal
My goals with the script
- Write a script that allows me to buy Anonymous Numbers at the cheapest price available.
Requirements
This will only require Golang.
Flow
The flow will be simple
Code
Let's see first how we can retrieve the phone numbers from Fragment Marketplace. For that we are going to use go-colly.
File: utils/fragment.go
package utils
import (
"fmt"
"strconv"
"strings"
"github.com/gocolly/colly"
)
func ScrapeFragmentMarket() ([]AnonNumbers, error) {
numbers := make([]AnonNumbers, 0)
c := colly.NewCollector()
c.OnHTML("tr td.thin-last-col a", func(e *colly.HTMLElement) {
name := strings.Trim(e.Attr("href"), "/number/")
valueStr := e.ChildText("div.table-cell-value.tm-value.icon-before.icon-ton")
value, err := strconv.Atoi(valueStr)
if err != nil {
return
}
numbers = append(numbers, AnonNumbers{Name: name, Value: value})
})
c.OnRequest(func(r *colly.Request) {
fmt.Println("Visiting", r.URL)
})
err := c.Visit("https://fragment.com/numbers?sort=price_asc&filter=sale")
if err != nil {
return nil, err
}
return numbers, nil
}
Now that I have the numbers or token name, I will need to know its address in the blockchain somehow. Now is when it comes into play our knowledge of the Telemint smart contract. Reading this smart contract you can notice that there's a method available for requesting the address of a given NFT item in this contract. The method is called get_nft_address_by_index, this method is a getter method so can be requested off-chain, without cost. I will need to compute the item_index, unfortunately we don't have a getter method for that. However, reading the contract you can notice that this item_index is computed as the hash of the token_name and that we do have it, so it's an easy piece.
File: utils/address.go
package utils
import (
"crypto/sha256"
"math/big"
// other imports as well …
)
// … other piece of code goes here
func itemIndex(tokenName string) *big.Int {
h := sha256.New()
h.Write([]byte(tokenName))
hash := h.Sum(nil)
index := new(big.Int)
index.SetBytes(hash)
return index
}
Fetching the address can be done in this way
package utils
import (
"context"
"crypto/sha256"
"fmt"
"math/big"
"time"
"github.com/xssnick/tonutils-go/address"
"github.com/xssnick/tonutils-go/ton"
)
const (
MainnetConfig = "https://ton-blockchain.github.io/global.config.json"
MaxRetries = 3
Timeout = 50 * time.Millisecond
)
var (
AnonymouAddress = address.MustParseAddr("EQAOQdwdw8kGftJCSFgOErM1mBjYPe4DBPq8-AhF6vr9si5N")
)
func GetAnonymousNumberAddress(ctx context.Context, api *ton.APIClient, tokenName string) (string, error) {
index := itemIndex(tokenName)
nftAddr, err := numberAddress(ctx, api, index)
if err != nil {
return "", err
}
return nftAddr, nil
}
func itemIndex(tokenName string) *big.Int {
h := sha256.New()
h.Write([]byte(tokenName))
hash := h.Sum(nil)
index := new(big.Int)
index.SetBytes(hash)
return index
}
func numberAddress(ctx context.Context, api *ton.APIClient, itemIndex *big.Int) (string, error) {
tries := 0
START:
block, err := api.CurrentMasterchainInfo(ctx)
if err != nil {
return "", fmt.Errorf("CurrentMasterchainInfo::%v\n", err)
}
tries++
res, err := api.RunGetMethod(ctx, block, AnonymouAddress, "get_nft_address_by_index", itemIndex)
if err != nil {
if lsErr, ok := err.(ton.LSError); ok && lsErr.Code == 651 && tries <= MaxRetries {
fmt.Println("Sleeping ", Timeout)
time.Sleep(Timeout)
goto START
}
return "", fmt.Errorf("RunGetMethod::%v\n", err)
}
cs := res.MustSlice(0)
addr := cs.MustLoadAddr()
return addr.String(), nil
}
Given that tonutils-go implements connection pool, it can be the chance that some block is not yet applied on one of the nodes, so it can lead to errors. In order to handle that I set a max retries policy of three times. A better approach would be to stick your connection to one node, but that requires you to have a ton node running, that's too expensive for us.
note: I'm poor feel free to donate to this address if you are going to use this script
EQC9n6aFb2oxQPMTPrHOnZDFcvvC2YLYIgBUms2yAB_LcAtv
Stop asking for money bro, no one will donate a shit…
Ok, to summarize we now have the item address in the blockchain, so we can now interact with the smart contract to buy this NFT.
package utils
import (
"context"
"fmt"
"log"
"strconv"
"time"
"github.com/xssnick/tonutils-go/address"
"github.com/xssnick/tonutils-go/tlb"
"github.com/xssnick/tonutils-go/ton"
"github.com/xssnick/tonutils-go/ton/wallet"
"github.com/xssnick/tonutils-go/tvm/cell"
)
const (
// another declaration goes here…
nanoTons int64 = 1e9
)
func buyNumber(
ctx context.Context,
w *wallet.Wallet,
tokenAddress string,
value int,
) error {
nftAddr, err := address.ParseAddr(tokenAddress)
if err != nil {
return err
}
transfer := wallet.SimpleMessage(nftAddr, tlb.MustFromTON(strconv.Itoa(value)), nil)
return w.Send(ctx, transfer, true)
}
// code related to auction of numbers goes here…
func BuyNumbers(
ctx context.Context,
w *wallet.Wallet,
api *ton.APIClient,
numbers []AnonNumbers,
) error {
// check available balance
block, err := api.CurrentMasterchainInfo(ctx)
if err != nil {
return fmt.Errorf("CurrentMasterchainInfo::%v\n", err)
}
balanceCoins, err := w.GetBalance(ctx, block)
if err != nil {
return fmt.Errorf("GetBalance::%v+\n", err)
}
balanceStr := balanceCoins.TON()
balance, err := strconv.ParseFloat(balanceStr, 64)
if err != nil {
return err
}
// check if we have enough balance
if enoughBudget(numbers, int(balance)) {
for _, number := range numbers {
if !number.OnSell {
err = buyNumber(ctx, w, number.Address, number.Value)
if err != nil {
return err
}
// add this number into db maybe
log.Printf("Number %s bought\n", number.Name)
}
}
}
return nil
}
// more code go here…
Almost done, in order to see the whole code I recommend you to clone this repo. As always I won't put the whole code here, because it doesn't make sense, as a developer you should get used to reading other people's code. So feel free to dig into that repository.
Conclusion
The combination of these two tools can be quite amazing. Try it yourself, feel free to reach me out on any doubts. Bye 👋.
Top comments (0)