How AI Can Help You Avoid Overpaying for Bitcoin Transactions

:::info Authors:

(1) Limeng Zhang, Swinburne University of Technology, Melbourne, Australia (limengzhang@swin.edu.au);

(2) Rui Zhou Swinburne, University of Technology, Melbourne, Australia (rzhou@swin.edu.au);

(3) Qing Liu, Data61, CSIRO, Hobart, Australia (q.liu@data61.csiro.au);

(4) Chengfei Liu, Swinburne University of Technology, Melbourne, Australia (cliu@swin.edu.au);

(5) M.Ali Babar, The University of Adelaide, Adelaide, Australia (ali.babar@adelaide.edu.au).

:::

Table of Links

Abstract and 1. Introduction

1. Preliminaries
2. Problem definition
3. BtcFlow
4. Bitcoin Core (BCore)
5. Mempool state and linear perceptron machine learning (MSLP)
6. Fee estimation based on neural network (FENN)
7. Experiments
8. Conclusion, Acknowledgements, and References

\ Abstract In the Bitcoin system, transaction fees serve as an incentive for blockchain confirmations. In general, a transaction with a higher fee is likely to be included in the next block mined, whereas a transaction with a smaller fee or no fee may be delayed or never processed at all. However, the transaction fee needs to be specified when submitting a transaction and almost cannot be altered thereafter. Hence it is indispensable to help a client set a reasonable fee, as a higher fee incurs overspending, and a lower fee could delay the confirmation. In this work, we focus on estimating the transaction fee for a new transaction to help with its confirmation within a given expected time. We identify two major drawbacks in the existing works. First, the current industry products are built on explicit analytical models, ignoring the complex interactions of different factors which could be better captured by machine learning based methods; Second, all of the existing works utilize limited knowledge for the estimation which hinders the potential of further improving the estimation quality. As a result, we propose a framework FENN, which aims to integrate the knowledge from a wide range of sources, including the transaction itself, unconfirmed transactions in the mempool and the blockchain confirmation environment, into a neural network model in order to estimate a proper transaction fee. Finally, we conduct experiments on real blockchain datasets to demonstrate the effectiveness and efficiency of our proposed framework over the state-of-the-art works evaluated by MAPE and RMSE. Each variation model in our framework can finish training within one block interval, which shows the potential of our framework to process the real-time transaction updates in the Bitcoin blockchain.

1 Introduction

Nowadays cryptocurrencies have become a buzzword in both industry and academia [23, 34, 27], and the market cap of Bitcoin, one of the most popular cryptocurrencies, reached a record high of over 1,200 billion USD in November 2021[1]. Blockchain provides the techniques underpinning for managing cryptocurrencies through a distributed ledger. Compared to the traditional currency systems [17, 4], the blockchain distributed ledger has three novel characteristics: decentralization, transparency and immutability. Currently, the majority of blockchain research focuses on the technologies that underpinning the deployment of blockchain systems, such as consensus protocol [9, 14, 21, 40, 32], query evaluation [39, 42, 35, 36], scalability enhancement [37, 28, 5], system engineering [7, 6, 33, 25], etc. However, one component is also indispensable to have a cryptocurrency system to be widely acceptable or even more prominent. That is the usability of the system-affiliated software, eg., client-side software.

\ In this paper, we focus on one important usability-related issue, transaction fee estimation in the Bitcoin blockchain system. The transaction fee serves as an incentive to complete the transfer of Bitcoin assets (recorded in a transaction) on the blockchain [8, 19, 20, 10]. Generally, a transaction with a higher fee is likely to be processed faster, whereas a transaction with a lower fee or no fee may be delayed for long or may never be processed. However, a user is required to specify a transaction fee when the user submits the transaction. If the specified fee is too high, the user will suffer from over-spending; if the specified fee is too low, the transaction may not be confirmed in time. Unfortunately, a user usually has no idea about how to properly preset a fee. We strongly believe the problem needs an in-depth study. A helpful fee estimation software could attract a large number of users and win a substantial portion of the market.

\ Most of the existing fee estimation tools were developed as built-in components of Bitcoin software by industry developers, such as BtcFlow[2], Bitcoin Core[3], statoshi[4], buybitcoinworldwide[5], bitcoinfees[6], etc. All of these strategies are based on the assumption that transactions with greater transaction fees are superior during the confirmation process. BtcFlow returns the minimal transaction fee for transactions in which the arrival rate is equal to or less than the confirmation rate. Bitcoin Core[3] delivers a weighted transaction fee based on the transaction fees of blockchain transactions with the same confirmation time. This result also applies to Statoshi[4] and buybitcoinworldwide[5]. Bitcoinfees[6] analyses the confirmation possibility of different transaction fees through an estimation API, which is not open sourced. As a result, it only works on real time online data and could not be evaluated and compared with other methods on collected datasets.

\ Although the existing industry tools have provided preliminary solutions to the fee estimation problem, there are opportunities lying in the following three aspects. (1) Insufficient disclose of the industry methods to academic community: Industry methods are not systematically documented and published academically (even for those open source tools), so researchers are not much aware of them, not to mention thinking of improving the existing methods. (2) Inferior estimation accuracy: Most of the existing methods are built on explicit analytical models, which means they consider transaction confirmation is driven by fixed mechanisms, eg., higher fee strictly affirms earlier confirmation in BtcFlow[2] and Bitcoin Core[3], blocks are generated following a specific Poisson distribution in BtcFlow[2], historical confirmation time determines future confirmation time in Bitcoin Core[3]. The existing methods ignore the complex interactions of different factors which could be better captured by machine learning based methods. (3) Limited knowledge is utilized: Partly caused by the intrinsic characteristics of the analytical models, it is difficult to integrate a diverse range of knowledge sources to do the estimation, eg., BtcFlow[2] mainly makes use of the information of newly submitted transactions in mempool, Bitcoin Core mainly considers historical transaction fees recorded in the blockchain. Surprisingly, the details of a given transaction itself are not sufficiently considered, eg., transaction inputs and outputs are two essential factors contributing to the complexity of transaction verification [2]. Other factors include block capacity (as it controls the upper bound of the transaction volume in each block), the blockchain network (such as computational power) evolving mining efficiency (which in return affects the speed of transaction confirmation), etc.

\ Other than the industry products, AI-Shehabi [1] proposed a neural network framework to model the interactions of transaction fee and transaction confirmation time. By assuming that the unconfirmed transactions in the mempool comprise the future block sequence in the blockchain, it organises these transactions, according to their transaction fees, into a virtual sequence of blocks. It then uses neural networks to extract patterns among the transaction fee, the virtual confirmation time (virtual block position), and its actual confirmation time, which are then used for fee prediction.

\ To further investigate the transaction fee estimation problem, we construct a new framework integrating more details in transactions and blocks. In our framework, we take into account, the features of the transaction itself, unconfirmed transactions in the mempool and the blockchain confirmation environment, such as the mining rate, block capacity, etc. We tackle this problem by leveraging the power of neural networks. Specifically, we make the following contributions

\ – We organize and systematically document the principle and the detailed estimation procedure of existing transaction fee estimation works from industry tools. Meanwhile, we analyse the limitations of these works. Finally, we implemented them to support offline evaluation.

\ – We propose a neural network framework to capture the complex interactions of different factors related to transaction confirmation. Meanwhile, compared to information used in the existing works, we integrate the knowledge from a more wide range of sources, including the transaction itself, unconfirmed transactions in the mempool and the blockchain confirmation environment, in order to estimate a proper transaction fee.

\ – We conduct extensive experiments on the real-world datasets to study the performance of fee estimation algorithms to demonstrate the efficiency and effectiveness of our proposed framework.

\ The rest of this paper is structured as follows: We introduce blockchain background information in Section 2 and define the target research problem in Section 3. Related works on transaction fee estimation are studied in Section 4-6, where we assess the benefits and limitations of the existing fee estimation methods. In Section 7, we present the proposed transaction fee estimation framework. In the following Section 8, we conduct experiments to evaluate the performance of different fee estimation methods as well as our proposed solution. Finally, we conclude our work in Section 9.

\

:::info This paper is available on arxiv under CC0 1.0 UNIVERSAL license.

:::

[1] https://www.slickcharts.com/currency

\ [2] https://bitcoiner.live/

\ [3] https://bitcoin.org/en/bitcoin-core/

\ [4] https://statoshi.info/dashboard/db/fee-estimates/

\ [5] https://www.buybitcoinworldwide.com/fee-calculator/

\ [6] https://bitcoinfees.earn.com/