Characterization and Benchmarking of Current Blockchains

Motivation

After the introduction of Bitcoin in 2008, the popularity of cryptocurrencies has risen, and with the presentation of Ethereum in the whitepaper [0] and the start of the platform in 2015, the world started to realize the potentials of distributed ledger technologies for purposes beside the cryptocurrency market and payment systems. By now, an enormous number of blockchain variants exists, e.g. Hyperledger Fabric, IOTA, Ripple, and Zcash. These blockchains describe permissioned or permissionless blockchains, address Byzantine faults using different approaches, and all differ in their programming logic, rules, protocols, and algorithms.

This variety makes it difficult to evaluate which platform is the most suitable for a given use case. Today, these platforms have a lot of restrictions and vulnerabilities, such as the transaction throughput (i.e. maximum number of concurrent transactions), confirmation time, energy consumption, and security concerns. They are therefore not all equally suitable for every production ready purpose. It is therefore necessary to have an exhaustive set of criteria with which the blockchain variants can be classified, and a common evaluation benchmark of blockchain platforms is needed.

Task Description

The goal of this thesis is to evaluate and compare several current blockchain platforms, for example Ethereum, Hyperledger Fabric, IOTA, Bitcoin, Ripple, VeChain, Monero, and Zcash, by setting up a network for each of them and simulating real world throughput. Common criteria for the evaluation of blockchain platforms should be identified and a benchmark using these criteria should be developed. This should continue and improve on previous work in this area, such as "Blockbench" [1], which describes a framework for analyzing private blockchains. Particularly, the thesis consists of the following tasks:

1. Foundations of around ten current blockchains, e.g. the previously mentioned ones, should be investigated and a list of criteria for comparison should be compiled. These criteria should include among others:
   • Design and architecture
   • Employed consensus protocols, e.g. PoW and BFT
   • Energy consumption, transaction throughput, and confirmation time
   • Support of smart contracts
2. A comparative evaluation of these blockchain variants according to the set of criteria should be created. Factors that are affecting the performance, scalability, and security of the platforms are to be identified and evaluated.
3. An evaluation testbed for at least three chosen platforms should be set up by deploying test networks to enable comparative evaluations.
4. A common benchmark allowing a comparative evaluation of different blockchain platforms should be developed, according to the previously identified variables and influencing factors.
5. The platforms should then be evaluated:
   • Performance evaluations regarding the maximum transaction throughput and latency should be included.
   • Optional: A workload generator, e.g. for simulating the behavior of different client machines, should be implemented.
   • A critical comparison of the platforms should be conducted.
6. Optional: Also include smart contracts and chaincode in the evaluation, especially with regard to security aspects and availability.

References