Paradigm: How to use on-chain data for strategy research?
Original title: "Using On-Chain Data for Policy Research: Part 1"
Original author: Brendan Malone, Paradigm
Original compilation: Kxp, BlockBeats
I. Introduction
Crypto policies rarely use real and detailed data when formulating, mainly for the following three reasons:
< /p>
1. Most of the relevant policies in the field of emerging technologies are still at the level of theory and qualitative analysis, and data are rarely used in the early stages.
2. Although the data on the chain is all open and transparent, people need to complete a lot of work in a short period of time to access the data (that is, extract the original data), even for Crypto native practitioners.
3. Blockchain "forensics" companies and data vendors have a small number of data products, but none of them are flexible/customizable nor can Meeting the needs of economic/financial researchers.
Many modern economics and finance researchers miss the opportunity to apply tools to Crypto data analysis Chance. By design, Crypto can provide granular data to anyone, but most of the policy still relies on external pre-aggregated time series data sources such as CoinMarketCap instead of obtaining data directly from the data source. What about?
Just as policymakers can look up the balance sheets of every major U.S. bank and see, second by second, They can look at Stablecoin issuance across the Ethereum ecosystem as easily as they can observe changes in consumer deposits, but most policy documents analyzing Stablecoins take an analytical approach that explores hypothetical events.
I will explain the following points in this article, I hope you can use Policy researchers on chain data are helpful:
· How to obtain chain data
· The structure of data on the chain
·< /b> Several basic tools for extracting and using data on the chain
In the following In this article, I will explore how the data collected here can be used to judge the trend of the Crypto market. In the meantime, I'll post free-to-use data and code at the end. By illuminating how data can be queried in the blockchain, I hope to show you the new ways that Crypto's openness can open up for data decision-making.
If you work for a regulatory agency or research institution and encounter Difficulties, welcome to get in touch with us to share your ideas and learn how Paradigm can help you.
II. On-chain data acquisition method
In general, data collection efforts should focus on a blockchain (Ethereum) and a subset of specific projects, which are mainly denominated in USD Denominated, fiat-backed stablecoins including USDC, Tether, Binance USD, Pax Dollar, and Gemini Dollar. This approach is broadly applicable to on-chain data, even if you want to create a different data set.
Block explorers such as Etherscan are very useful for viewing transaction snapshots and collecting information, but in my experience they are less useful for generating large datasets. When collecting and processing raw data, you basically have two options: (1) run a full node locally, or (2) query a database that has raw data written directly from the chain. The first method has relatively high requirements for professional skills and computing resources, while the second method only requires basic SQL and Python skills, so we will use the second method here.
Dune and Google Cloud Platform (GCP) BigQuery have the latest on-chain data, which All are stored in the database in the form of tables, which can be queried by researchers using SQL commands. Dune also provides a free database, but it is slower and has some limitations, but it is very suitable for A/B testing data queries, and it also helps you familiarize yourself with the operating mode of the database, especially for those who are not good at using SQL A person who queries an associated database. Although BigQuery is more flexible and faster, Google charges for computing resources, so its price is relatively expensive. When I first process data, I test queries in Dune before feeding them into GCP to save money. For the most part, this works pretty well (it's also worth noting that Dune has at least 100 times as many Crypto tables as GCP, including some user-generated, pre-cleaned, high-quality Crypto tables. Tables. In contrast, the data in GCP is mostly raw blocks or transactions. Dune also has some very handy built-in data visualization tools that are worth buying separately.)
III. On-chain data structure
To answer this question, you first need to understand the purpose of your processing data. For this test case, I decided to build a large time-series dataset for the main fiat-backed Stablecoins and observe some specific behaviors: minting (i.e. issuing Stablecoins), burning (i.e. decommissioning Stablecoins) and transfers. I chose to conduct my research this way because policymakers and academics are currently focusing most on fiat-backed stablecoins, so the data could be quite useful in the short term.
Several major stablecoins denominated in US dollars have adopted the ERC-20Token standard. As the name suggests, ERC-20 is a standardized way to create tokens using smart contracts on Ethereum. If you think of the blockchain as a giant decentralized Excel sheet, then a smart contract is like an Excel function. After input parameters in the function, it will use its built-in logic to produce a specific output result (for example, the MAX function is used to output the maximum value among the input parameters).
We can locate smart contracts using their Ethereum addresses, which are blocks Unique identifier in the chain data structure:
· USDC
Similar to API, smart contract It is also a program that can be reused. Every time a smart contract receives an interaction command, a record of that interaction is generated and recorded on the blockchain by the Ethereum protocol in the form of a log, and these logs constitute a reliable source of information about the smart contract's activities.
When a smart contract executes a specific function, such as burning an ERC-20 Stablecoin to remove it from Removed from circulation, the function and its parameters will be recorded on the blockchain as a transaction log.
In the transaction below, Circle, the issuer of the USDC Stablecoin, burned $1056.92 worth of USDC .
If you switch to the "log" tab, you can view the transaction event log, the corresponding field is
· Address: The contract address of the smart contract. The contract address of USDC Stablecoin is 0xa0b86991c6218b36c1d19d4a2e9eb0ce3606eb48.
· Name: the function executed by the smart contract, and the parameters in the function. Here, the smart contract is calling the burn function, which receives parameters specifying where the burned coins are sent (such as a burn pool, which must be an Ethereum address) and the amount of burned coins (the amount must be an unsigned integer less than 256 bits ).
Etherscan's output also shows the subject and data fields containing the Most of the relevant information that needs to be parsed when trading.
· Topic0 is the hash of the function signature. Essentially, it passes the function and its arguments through a one-way algorithm to get a unique function hash. Ethereum uses the Keccak-256 hash function, and when you enter a function signature through the Keccak-256 algorithm, it will always produce the same hash, so anytime that hash appears in the logs, you can be sure is calling the same function.
· Topic1 is an index parameter of the burn function. Here, Topic1 is the address where burned tokens are sent to. (Note: If the burn function has more parameters, these will appear as additional topics)
· here The data field indicates the number of tokens burned.
Now that we have understood the basic structure of data on the chain, we can start to learn from Dune and GCP extracted data.
IV. Basic tools for extracting and processing on-chain data
< br>
As mentioned, in this example I chose to pull on-chain data from an existing database rather than accessing the Ethereum network Active nodes on . For ease of understanding, I extracted a large number of raw data tables from GCP using SQL and then cleaned them in Python using the pandas library.
When we pull the table from GCP, we will use BigQuery, which stores a lot of Ethereum's The data table, as shown in the left column of the figure below. When you click on a table, the corresponding database schema appears, like the ethereum.logs table in the image below. At the same time, the addresses, data and topics involved will be recorded in the log data.
The query statement in the figure below will be used to extract all records in the log table that involve interactions with USDC, Tether USD, Binance USD, Pax Dollar or Gemini Dollar contracts. In addition to the information in ethereum.logs, some additional information is useful, so I also merged the data in the ethereum.block table, which covers gas fees and other information.
The resulting table can be read directly by Python and subdivided into the following fields with the help of a pandas data frame:
· log_index
· transaction_hash
· transaction_index
· data
· topics
· block_timestamp
· block_number
· block_hash
· number
· miner
· size
· gas_limit
· gas_used
· base_fee_per_gas
V. Conclusion
This article uses the log data of Ethereum, and the same method can also be used to access various data on the chain. Python and SQL are tools familiar to most economists and policymakers, and they can make a big difference. Compared with traditional finance, Crypto is more transparent. This allows researchers to use real-time data to shed light on how the financial system works and contain possible risks in a timely manner.
In the next article, I will focus on analyzing the minting and burning of Stablecoin supported by legal currency, and prepare the corresponding data set in advance. In the third article, I will show you in the form of diagrams the problems that the refined on-chain data can be used for research.
Original link
Welcome to join the official BlockBeats community:
Telegram Subscription Group: https://t.me/theblockbeats
Telegram Discussion Group: https://t.me/BlockBeats_App
Official Twitter Account: https://twitter.com/BlockBeatsAsia
Related Articles
