Benchmarking
Benchmarking provides insights into Phink’s performance in real-world scenarios, in order to vizualise its efficiency
and
fuzzing ink! smart contracts. Below are the benchmark results for various smart contracts, detailing
coverage, speed, corpus size, and the potential usage of generate-seed. Each contract were fuzzed for maximum a day.
Statistics (especially average speed) are given for one core only. The coverage percent is calculated using the
number of
lines covered divided the number of reachable lines, as a percentage.
Benchmarks
| Contract name | Coverage percent | Average speed (execs/sec) | AFL++ corpus size | Using Phink seed generation |
|---|---|---|---|---|
| abax_governor | 48% | 1500 (early phase) / 100 (late phase) | 1639 | NO (no tests available) |
| multisig | 91% | 113 (late phase) | 1524 | YES (without E2E) |
- Github for
abax_governor: https://github.com/AbaxFinance/dao-contracts/tree/main/src/contracts/abax_governor - Github for
multisig: https://github.com/use-ink/ink-examples/blob/main/multisig/lib.rs
Explanation of terms
-
Coverage: Represents the percentage of the code that have been executed during the fuzzing campaign. Higher coverage indicates more thorough testing (the higher the better).
-
Average Speed (for 1 core): The number of executions per second that the fuzzer can handle on a single CPU core. This metric helps gauge Phink’s performance efficiency. As a reminder of execution contains multiple calls, up to
max_messages_per_exec. -
AFL++ Corpus Size: The size of the corpus generated by AFL++ during fuzzing. A larger corpus generally implies a diverse set of inputs to test the contract.
-
generate-seed usage: Indicates whether
generate-seedwas used to seed the initial tests. This depends if the contract include tests within the contract.
Environment details
- CPU: AMD EPYC 7282 16-Cores
- Operating System: Linux 5.4.0-189-generic #209-Ubuntu x86_64
- Phink Version: 0.1.4
Contributing to the benchmarks
We encourage contributions to our benchmarks! If you have a contract you would like to see benchmarked, please submit a pull request to our repository.