Ethereum is an open-source platform based on blockchain technology. Unlike other cryptocurrencies such as Bitcoin, Ethereum not only facilitates digital currency transactions but also enables developers to build and deploy smart contracts. Smart contracts are programs that run on the blockchain, automatically executing predetermined conditions and operations. Ethereum’s smart contracts utilize the Turing-complete programming language Solidity, allowing developers to write complex logic and algorithms. This makes Ethereum more than just a digital currency; it is also a powerful decentralized computing platform.

One of Ethereum’s core components is the Ethereum Virtual Machine (EVM) [16], a stack-based virtual machine used to execute code for Ethereum smart contracts. The EVM uses an intermediate language called Ethereum bytecode to represent smart contract code. After developers write smart contract code in Solidity, it is compiled into Ethereum bytecode and deployed onto the Ethereum network.

The execution process of the EVM is stack-based. Each smart contract has its own storage and stack spaces. When executing a smart contract, the EVM sequentially reads the code line by line and places it into the stack. Instructions can include mathematical operations, conditional statements, jumps, storage, and loading operations. While executing instructions, the EVM manipulates the stack based on the opcode and operands of each instruction. For example, when executing an addition instruction, the EVM pops two operands from the stack, adds them together, and pushes the result back onto the stack.

The execution result of a smart contract is recorded on the Ethereum blockchain. Once a smart contract completes execution, it can change its own state and send messages to other smart contracts or call methods of other smart contracts. The execution result of a smart contract can also be recorded as part of a transaction on the Ethereum blockchain.

The emergence of smart contracts in the field of blockchain has garnered widespread attention and discussion. Traditional centralized institutions encounter several issues in areas such as financial transactions and contract execution, including high fees, lack of transparency, and security concerns. However, the decentralized nature of smart contracts built on blockchain offers a novel solution to these problems. Decentralization allows anyone to participate in the operation of smart contracts without relying on the trust of intermediaries. This has brought about more efficient, transparent, and secure solutions in various fields such as finance [17,18], supply chain management [19], Internet of Things [20–23],construction [24], and energy [25].

Smart contracts have introduced numerous innovations and conveniences across various industries within the realm of blockchain technology. By enabling automated contract execution, they eliminate intermediaries and the issues of distrust commonly associated with traditional contracts. However, the security of smart contracts has always been a pressing concern. Security vulnerabilities primarily refer to logical flaws that arise during the design process of smart contracts. Once deployed on the blockchain, these logical flaws can easily lead to security issues, such as financial losses for users. Moreover, since smart contracts operate on a blockchain, once deployed, they cannot be altered, making security issues potentially consequential.

Currently, the most widely used programming language for smart contracts on the Ethereum platform, the largest smart contract operating platform, is Solidity. Solidity is based on a Turing-complete language [26], which offers flexibility and powerful features that allow developers to implement complex logic and business operations. However, this flexibility also provides opportunities for attackers to discover and exploit vulnerabilities.

Researchers have proposed classifications of vulnerabilities for Ethereum smart contracts. Li et al. [27] conducted a survey and analysis of six real attacks targeting blockchains. Nicola et al. [28] were the first to categorize smart contract vulnerabilities into three classes and studied twelve vulnerabilities related to smart contracts. Zhu et al. [29] identified forty different vulnerabilities. As shown in Table 1, we summarize the current common smart contract vulnerabilities.

Currently, the mainstream techniques for detecting vulnerabilities in smart contracts can be categorized into four types: feature matching, fuzz testing, symbolic execution, and taint analysis.

Feature matching is a static analysis-based technique for detecting vulnerabilities in smart contracts. It abstracts the features of malicious code [30,31] and detects smart contract source code or bytecode using similarity comparing to features, which are representated by intermediate layer, such as an Abstract Syntax Tree (AST) [32] or Control Flow Graph (CFG) [33].

Fuzz testing [34] is a unique vulnerability detection technique. It provides smart contracts with a large amount of random or exceptional data to trigger potential abnormal states and observe the results of contract execution to discover possible vulnerabilities. The core of fuzz testing lies in the generation and evaluation of test cases. Based on the current execution results and coverage information, it selects optimal test cases as inputs for the next round, thereby improving detection efficiency and quality.

Symbolic execution [35] is a program analysis technique that employs abstract symbols to represent program variables and simulates the execution of program instructions while collecting path constraints for each branch encountered. When encountering program branches, symbolic execution explores each branch path. Once a path is executed, the path constraints generated during symbolic execution are handed over to a satisfiability modulo theories (SMT) solver for resolution. If the path constraints can be solved by the SMT solver, concrete values that can execute the corresponding path are obtained. If the solution fails, it indicates that the path is unreachable. However, due to the need for symbolic execution to explore two program paths at each conditional branch, the number of program paths to be explored grows exponentially as the program size increases. This leads to the problem of path explosion, which is an inherent limitation of symbolic execution.

Taint analysis [36] is used to detect whether sensitive data or operations in smart contracts are influenced by external input or untrusted data, leading to security vulnerabilities. This technique tracks the generation, propagation, and elimination of tainted information in the contract to determine if it interferes with critical operations.

In recent years, deep learning-based techniques for smart contract vulnerability detection have received widespread attention and research. These techniques leverage deep neural networks to learn the semantic and structural features of smart contracts, enabling automatic identification and classification of vulnerability types. Compared to traditional approaches, this technology does not rely on manually defined vulnerability rules or patterns. It can handle large-scale smart contract data, adapt to different smart contract languages and platforms, and detect multiple types of vulnerabilities. Currently, deep learning-based detection techniques can be categorized into three main types: text processing, static analysis, and image recognition. Text processing-based deep learning methods employ natural language processing (NLP) techniques to extract semantic features from smart contracts, such as word embedding, sentence embedding, and attention mechanisms. Static analysis-based deep learning approaches utilize static analysis techniques to extract structural information from smart contracts, such as control flow graphs and abstract syntax trees. Image recognition-based deep learning methods convert smart contracts into images and employ convolutional neural networks (CNN) or other image processing models for feature extraction and classification to determine the presence and type of vulnerabilities. However, it is regrettable that most current deep learning-based methods suffer from usability challenges and lack corresponding installation and usage instructions. Therefore, they are beyond the scope of this paper’s discussion.

We searched for relevant works in Google Scholar and Web of Science using the keywords ‘smart contract tools’ and ‘smart contract detection’. In the first stage, 781 articles were found.

In the second stage, we selected papers related to our work by defining a set of inclusion and exclusion rules, which are shown in Table 2. A total of 112 articles that meet the criteria were selected through reviewing abstracts and main findings from 781 literature pieces.

In the third stage, we conducted detailed research and selected 38 open-source articles that provided installation methods out of the 112 articles about tools to be included in our research. The above work was independently completed by three individuals.

This section examines working principle and the methods used to identify targets in the 38 articles.

Vulpedia [37] is a static analysis tool developed using Python in 2022. Vulpedia identifies vulnerabilities using vulnerability signatures. It clusters contracts with the same vulnerabilities using abstract syntax trees and summarizes signatures through program dependency graphs. To build a feature library, it extracts vulnerability signatures from vulnerable contracts of the same type and benign signatures from benign contracts. If a vulnerable signature is matched and no benign signature is found, the contract is considered vulnerable.

SpCon [38] is a dynamic analysis tool that was released on 22 January, 2022. The tool utilises the role-based access control (RBAC) model to divide contract users into different roles, each with different permissions. Lower roles are unable to read or write data owned by higher roles, which necessitates user mining and identity determination. Existing role mining techniques often assume that user permission assignments are fully observed, but this is not the case in smart contracts. Therefore, this tool considers users with the same behavior to have the same role and uses the average frequency vector (AFV) for quantitative capture. The AFV measures how often users with a particular role execute different permissions and serves as the signature of the role. The similarity in AFV between different roles is low. Any discrepancy found between what is allowed and what is actually found indicates a potential permission flaw.

Published in 2022, xFuzz [39] is a vulnerability analysis tool that utilises machine learning and fuzz testing methods. The tool focuses on vulnerabilities that are most likely to cause cross-contract problems. As multiple contracts interacting increases the search space, machine learning methods are used to guide the fuzz testing of smart contracts. The contract vulnerability is transformed into a vector using Word2Vec and combined with the contract’s static characteristics. This combined information is then used as input for the machine learning model to train it. The trained model is then used to determine the potential location of the vulnerability in the contract. This approach reduces the search space and improves efficiency.

ESBMC [40] is a context-bounded model checker based on SMT that was published in 2022. The tool initially receives the source code of the smart contract and analyses it through lexical and syntactic analysis. It then converts the smart contract sol into an abstract syntax tree (AST) in JSON format. The resulting AST is then transformed into the intermediate representation (IR) of ESBMC, which is used to generate the symbol table. The table is then converted into a GOTO program and delivered to the symbolic execution engine (SymEx) for processing to generate its static single assignment (SSA) form. This form is then used to generate verification conditions. Finally, create logistic equations to represent constraints (C) and properties (P) from the SSA form. When the SMT solver finds a solution that satisfies the equation, a vulnerability is considered to exist.

In 2022, eTainter [41] was published. It takes the bytecode of smart contracts as input and builds a control flow graph (CFG). By identifying the bytecode, it extracts the EVM instructions for reading user data and can identify gas-related vulnerable EVM instructions. CFG paths are then extracted and analyzed for contamination.

Sailfish [42] is a tool that identifies inconsistent state errors in smart contracts. It was released in 2022 and consists of two parts: the EXPLORE phase and the REFINE phase. During the EXPLORE phase, the contract is converted into a storage dependency graph (SDG), which is then queried to match vulnerable subgraphs. During the REFINE phase, the tool will utilise symbolic execution to analyse contracts that match vulnerable subgraphs and use pre-stored variable constraints as a prerequisite for symbolic execution.

RA [43] was released in 2021 and is used to detect reentrant vulnerabilities. It combines symbolic execution and SMT solvers to simulate and verify reentrant vulnerabilities.

Eth2Vec [44] is a tool that automatically extracts features for each contract using machine learning methods and neural networks for natural language processing. The tool will be released in 2022. Its purpose is to detect vulnerabilities in contracts by comparing the similarity of the extracted features to previously learned code.

EtherSolve [45] released in 2021 proposes a symbolic execution algorithm based on the Ethereum operand stack. The bytecode is converted to CFG by parsing the jump operation in the contract, and reentrancy vulnerabilities are detected through CFG.

Gasgauge [46] was published in 2021. The tool divides its detection process into three stages: detection, identification, and correction. In the detection phase, static analysis is used to identify all loops in the contract. Then, fuzz testing methods are used to generate inputs that exhaust contract gas. Finally, vulnerabilities are reported to users through the results of the running verification and static analysis phases.

Echidna [47] is a smart contract vulnerability detection fuzz testing tool published in 2020. It provides a complete Ethereum smart contract fuzz testing framework, which can analyze and simulate the execution of smart contract source code and generate compliance with contract call specifications. The tool uses random transaction data to fuzz test the contract and introduces coverage information to detect the execution efficiency of fuzz testing.

WANA [48] was published in 2020. It uses the Wasm symbolic execution engine to parse, load, and initialize the Wasm bytecode. It checks unsatisfied paths by calling the Z3 constraint solver and collects execution information for vulnerability analysis. WANA performs vulnerability analysis based on the collected execution information during symbolic execution.

SmartEmbed [49], published in 2020, is an effective tool for checking bugs in smart contracts. It allows for easy development of error checking rules and addition of new errors. The similarity checker is the core component of SmartEmbed, which takes in the error embedding matrix, code embedding matrix, and embedding vector as input, and outputs error reports.

VERISMART [50] is a fully automatic tool based on static and dynamic program analysis technology, implemented in OCaml and published in 2020. This tool analyzes the basic path of a smart contract. Verify the generation conditions and detect vulnerabilities by collecting unknown paths.

sFuzz [51] was published in 2020 and aims to improve test case coverage by adopting a feedback adaptive fuzzing strategy, while learning from the traditional fuzz testing tool AFL. The tool is composed of three main modules: runner, libfuzzer, and liboracles. The runner module creates a private test network for deploying test contracts and executing transactions. The libfuzzer component generates test cases, while the liboracles component detects the execution of test cases and checks for vulnerabilities.

EthBMC [52] is a dynamic symbolic execution-based bounded model checker published in 2020. It can automatically generate specific inputs to simplify further analysis of EVM bytecode. EthBMC addresses identified issues by more precisely reasoning about the internals of the EVM. The contract is encoded with constraints and then solved using an SMT solver. EthBMC is effective in solving the parity error vulnerability.

The EVMFuzzer [53] is a fuzz testing tool designed to detect abnormalities in the Ethereum Virtual Machine (EVM). The tool was published in 2019. It uses smart contracts as input, generates random mutation data, and sends it to the target EVM and baseline EVM. The output results and status changes of each EVM are then compared. Any differences or abnormalities found are reported as suspicious cases.

SolidityCheck [54] was published in 2019. This is a static open-source tool written in C++ that uses regular expressions and program instrumentation to address vulnerabilities in smart contracts [55]. The tool’s detection process is divided into four main stages: 1. Formatting the smart contract code; 2. Filtering keywords to extract problematic statements; 3. Prevention and detection; 4. Generating detection reports and prevention contracts. SolidityCheck primarily identifies reentrancy vulnerabilities and integer overflows with exceptional accuracy.

Published in 2019, Vultron [56] is a dynamic smart contract fuzz testing framework developed using JavaScript. It can detect irregular transactions caused by various types of adversarial exploits.

VeriSolid [57] is an open-source web-based verification framework developed in 2019. It is built on webGME and FSolidM and allows for collaborative development of Ethereum contracts with built-in version control.

SIF [58] is a static vulnerability analysis and detection tool that is open source. It was developed in C++ language and was invented in 2019. This tool is a comprehensive tool for analyzing, querying, detecting, and generating code for Solidity contracts. Additionally, the tool incorporates a module called Assertion Analyzer, which identifies Integer Overflow vulnerabilities based on the analysis.

FEther [59] is an open-source static smart contract analysis and detection tool developed using Coq. Published in 2019, this tool is designed to provide objective evaluations of contract code. It takes Solidity as input and analyzes and detects contract code through symbolic execution and logical constraint solving.

Published in 2018, Vandal [60] is a static and open-source tool for detecting smart contract vulnerabilities. The tool decompiles the bytecode of the Ethereum Virtual Machine (EVM) into an intermediate representation (IR) and builds the control flow graph (CFG) of the program. The Vandal tool converts the IR and CFG into logical relationships, which are then stored in a knowledge base. The tool uses the Soufflé language to write security analysis specifications, which are inputted into the Soufflé engine along with the knowledge base for query and reasoning. Finally, the tool generates security analysis specifications based on the output results of the Soufflé engine. The report is then produced.

Securify [61] is a static and open-source smart contract vulnerability detection tool that was published in 2018. It uses symbolic execution to analyze smart contracts and extract data flow, control flow, function calls, and other information from the code. The tool then checks for violations of the preset pattern.

EthIR [62] is a static and open-source smart contract vulnerability detection tool published in 2018. Its main contribution is converting Ethereum bytecode into a rule-based representation (RBR), which allows for easy application of advanced analysis methods to infer the nature of EVM code. To begin, generate a control flow graph (CFG) from Ethereum bytecode using OYENTE. Next, extract the basic blocks and jump targets from the CFG using a decompiler. Then, extract the rules and operand stacks from the basic blocks using an interpreter. Finally, convert the rules and operand stacks to RBR using a reconstructor.

MAIAN [10] was published in 2018. It uses symbolic execution and Hybrid Depth First Search (Hybrid Depth First Search) technologies to analyze possible state transitions and abnormal behaviors that may occur during multiple calls of smart contracts. The process mainly consists of four steps: preprocessing, symbolic execution, recording paths and conditions that may trigger the vulnerability using the Z3 theorem prover, and hybrid depth-first search. Use a heuristic algorithm to sort the paths obtained by symbolic execution, and select the path most likely to trigger the vulnerability for further analysis; Verification, use specific input and transaction sequences to verify the results obtained by symbolic execution and hybrid depth-first search vulnerability path and report the discovered vulnerabilities.

Published in 2017, Porosity [63] is a static detection tool that converts EVM bytecode into an intermediate representation (IR). It then analyses basic blocks and boundaries based on jump instructions and function calls. Porosity connects basic blocks into CFGs and identifies loops, conditional branches, and function entries. Finally, it converts the CFG into a Solidity syntax contract and outputs it to standard output or a file.

FSolidM [64] is a tool for detecting static smart contract vulnerabilities that was published in 2017. It is based on the Finite State Machine (FSM) model, which abstracts smart contract behavior into a set of states and transitions. The tool includes a graphical editor that enables developers to create and modify FSMs using drag-and-drop. Additionally, FSolidM provides a code generator that can convert FSMs into Solidity language smart contracts and deploy them to the Ethereum network.

KEVM [65] was created in 2016 as a tool for detecting static smart contract vulnerabilities. It uses the K framework to define the syntax and semantics of the EVM’s bytecode stack language, allowing it to execute and simulate the behavior of the EVM. KEVM can simulate all instructions and abnormal behaviors of the EVM, and its correctness and performance can be verified through the official Ethereum test suite.

SmartCheck [30], a static open-source smart contract vulnerability detection tool developed in Java, was published in 2018. SmartCheck conducts lexical and semantic analysis on Solidity source code and utilizes ANTLR [66] and a custom Solidity language to generate an XML parse tree as an intermediate representation. Users can detect vulnerability patterns by employing XPath queries on the intermediate representation. SmartCheck offers significant improvements compared to existing alternatives, as it has been extensively tested on real-world contracts and successfully identified code issues in the majority of them.

Slither [31], published in 2019, is an open-source static analysis framework for smart contract vulnerabilities. This tool serves four main purposes: (1) automatic detection of vulnerabilities, (2) automatic detection of code optimization opportunities, (3) enhancing user understanding of contracts, and (4) assisting with code reviews. Its working principle involves transforming Solidity smart contracts into an intermediate representation called SlithIR. SlithIR utilizes Static Single Assignment (SSA) form and a simplified instruction set to streamline the implementation of analysis while retaining semantic information. Slither allows the application of common program analysis techniques such as data flow and taint tracking to detect vulnerabilities. The tool is capable of detecting Reentrancy, Integer Overflow, Unchecked Low Level Calls, Tx.origin, Time Manipulation, Delegatecall, and Access Control.

Mythril [67], published in 2017, is a static and open-source smart contract vulnerability detection tool. It utilizes symbolic execution techniques to simulate the execution process of a program. Instead of using concrete input values, it uses symbolic variables to explore all possible execution paths of the program. In addition, Mythril incorporates techniques such as taint analysis and control flow graphs to enhance the efficiency and accuracy of the analysis.

Oyente [68], a tool that emerged in 2016, utilizes symbolic execution to analyze smart contract bytecode. It consists of four key components: CFG Builder, Explorer, Core Analysis, and Validator. Oyente operates at the bytecode level and dynamically explores the program’s control flow graph during symbolic execution. It detects contract vulnerabilities by considering path constraints, variable origins, and other relevant information.

Solmet [69], published in 2018, is built upon the existing ANTLR4 grammar to parse Solidity language. Although it does not directly detect vulnerabilities, Solmet analyzes Solidity smart contracts and calculates relevant metrics.

Solhint [70], introduced in 2017, is a static analysis tool for smart contract vulnerability detection. It begins by converting the source code of a smart contract into an abstract syntax tree (AST) using a parser. Then, it traverses the AST using a visitor while loading user-specified configurations, which include enabled or disabled rules and plugins. Solhint checks each AST node based on the rules in the configuration file and collects detected issues. Finally, it formats the problems in different output formats, such as console or file.

ContractFuzzer [71], published in 2018, employs fuzz testing techniques to identify security vulnerabilities in Ethereum smart contracts. It operates in four steps: firstly, it generates fuzzy inputs based on the smart contract’s ABI specification; then, it uses predefined test cases to detect common vulnerability types; next, it utilizes an EVM instrumenter to record the runtime behavior of the smart contract; finally, it employs a log analyzer to analyze the recorded logs and report any discovered vulnerabilities.

Osiris [72], which was published in 2018, is a static and open-source framework for smart contracts vulnerability detection and analysis. It primarily utilizes a combination of symbolic execution and taint analysis to detect vulnerabilities. The tool converts the bytecode of smart contracts into an intermediate representation (IR) and performs static analysis to identify instructions and variables that may lead to integer-related vulnerabilities. Then, Osiris utilizes taint analysis to mark inputs, outputs, and state variables related to integer-related vulnerabilities, creating a taint graph that represents their data flow dependencies. Subsequently, the tool employs symbolic execution to traverse all paths in the taint graph and generate corresponding path constraints and objective functions. Finally, Osiris uses a constraint solver, such as Z3, to solve the path constraints and objective functions, resulting in a set of input values satisfying the conditions, i.e., exploits. If no satisfying input values are found, it indicates that the path is secure or unreachable.

HoneyBadger [73], published in 2019, consists of three components: symbolic analysis, cash flow analysis, and honeypot analysis. The symbolic analysis component constructs a control flow graph (CFG) and executes its different paths symbolically. The results of symbolic analysis are then propagated to the cash flow analysis and honeypot analysis components. The cash flow analysis section utilizes the results from symbolic analysis to detect whether a contract can receive and transfer funds. Lastly, the honeypot analysis component aims to detect different honeypot techniques studied in this article by combining heuristic and symbolic analysis results. All components employ the z3 SMT solver to check for constraint satisfaction.

This section presents a summary of current open source tools. Table 3 presents specific information, such as the detection method, project address, update time, installation method, and ease of use. Ease of use is divided into five levels: 5★ indicates that it is extremely easy to install and does not require the installation of any environment; 4★ indicates that it is easy to install and only requires the installation of one environment dependency; 3★ indicates that it is easy to install but requires the installation of two environment dependency; 2★ indicates that requires the installation of three environment dependency; 1★ indicates that more than three environment dependencies.

Table 4 displays the vulnerabilities that can be detected by the tools mentioned in this paper. It is important to note that some tools only convert the code into an easy-to-analyze form without directly instrumenting the smart contract. Although this method of converting code may aid in further analysis and identification of potential vulnerabilities, it still necessitates manual review and verification. Therefore, the security of smart contracts still requires a combination of expertise and experience when using these tools.

In this section, we will introduce the operating environment of the framework. For the overall framework operation, we utilize Python as the primary building tool to implement the user interface and interaction logic. Python is a versatile programming language with powerful cross-platform capabilities, which allowing it to run on different operating systems.

In the vulnerability detection section, we employ VMware to run the detection tools. Specifically, we choose VMware as the platform to execute the detection tools. VMware is a virtualization software that enables the creation of multiple virtual machines on a single host, each running different operating systems. This allows for the simulation of various environments to accommodate the use of different detection tools.

Regarding the storage of user information and running results, we utilize MySQL as the database system. MySQL is an open-source relational database management system that is suitable for small to medium-sized applications. It is user-friendly, providing sufficient performance and reliability to meet most typical data storage and querying needs.

The design philosophy of our framework is based on converting functional requirements analysis into a modular code structure. This design allows us to achieve better modularity and maintainability. Each functionality is encapsulated as an independent module and interacts with other modules through well-defined interfaces. This design enables us to conveniently expand and modify functionalities while reducing code coupling.

Our framework consists of four parts: the display module, the preprocessing module, the database module, and the virtualization validation module. The web interaction module serves as the user interface of the system, providing users with convenient features such as uploading contract files, viewing detection results, and accessing historical information. The preprocessing module performs initial processing on the uploaded contracts to improve the efficiency and accuracy of subsequent detection. The database module is an essential component of the smart contract vulnerability detection system. It primarily handles the storage and management of contract detection-related data, including preprocessed contracts, detection results, and the order of contract detection. The preprocessed contract results are stored in the database in the form of file addresses, facilitating subsequent testing and analysis. The virtualization testing module is the core part of the smart contract vulnerability detection system. It is responsible for comprehensive testing of smart contracts and feeding the results back to the database module for further analysis and processing.

Through the combination of these four modules, our framework is capable of achieving comprehensive detection and analysis of smart contracts. Moreover, this modular design allows for convenient expansion and modification of functionalities, enhancing the system’s maintainability. Additionally, the interaction between modules through well-defined interfaces reduces code coupling. This design makes our framework more flexible and user-friendly.

In this section, we will briefly describe the implementation details of the module based on the schematic diagram.

The operational logic of these interconnected modules is illustrated in Fig. 1. The system comprises four key modules: the Display Module, the Preprocessing Module, the Database Module, and the Validation Module. The Display Module functions as the user interface, allowing users to input contract codes. The Preprocessing Module processes these input codes, preparing them for use by subsequent modules. The Database Module acts as an intermediary for storage and transmission, storing the preprocessed codes and transferring the data to the Validation Module in the order received. The Validation Module verifies the received data and sends the validation results back to the Database Module. Finally, the Database Module presents these validation outcomes to the user, enabling them to review and understand the system’s evaluations.

By employing such a modular operational logic, users can conveniently input contract codes and obtain corresponding validation results. This modular design enables each functional module of the system to work independently, facilitating ease of maintenance and expansion.

To ensure the integrity of the study and ensure the scalability of the detection framework, we further propose the corresponding network structure and hardware planning. The purpose of these supplementary works is to address the limitations of our method in specific tasks and provide more efficient and accurate solutions.

As shown in Fig. 6, the web server provides services through a unique domain name. The user’s access request first reaches the Display and Preprocessing Module Server, where the router evenly distributes the requests to the servers and performs authentication services. At the same time, this server can preprocess the contracts after user submission. This architecture ensures that the system maintains good performance and availability even under high concurrent requests.

The Database Server is responsible for receiving smart contracts and storing data locally to ensure the security of user information. Additionally, this server can transmit the contracts to the virtualized detection module in the public cloud via the VPN Server through a dedicated link. The public cloud offers almost unlimited scalability. When demand increases, it is easy to add more virtual machines. Similarly, when demand decreases, it is convenient to reduce the number of virtual machines. This flexibility allows for dynamic adjustment of resources based on demand without the need for expensive hardware investments. Furthermore, deploying virtual machines in the public cloud usually takes only a few minutes, while deploying new physical servers locally may take several days or even weeks. This enables the framework to quickly scale with new tools.

Additionally, the Supervision Server allows for obtaining information from each server, including performance logs and detection logs, for analysis.

a. As shown in Figs. 7 and 8, the framework offers user registration and login functionality, enabling users to create accounts and access the framework by logging in.

b. As shown in Figs. 9 and 10, the framework possesses the capability of unified detection and tool-specific detection to meet user requirements. Users can perform unified detection operations through the framework or choose to use different tools for detection. This flexibility allows users to select the most suitable detection method based on their own needs.

c. As shown in Fig. 11, the framework offers a functionality to query the history records, allowing users to conveniently retrieve their previous operation records. Users can trace their historical queries on the framework through this feature, facilitating reference, review, or analysis purposes.

d. As shown in Fig. 12, the framework utilizes backend virtualization technology to establish a detection server, which provides a detection interface to the frontend. Users can interact with the backend virtualized detection server through the frontend page, submitting data for detection and retrieving the corresponding detection results.

In recent years, the Ethereum community has developed numerous tools for analyzing vulnerable smart contracts. However, there is a lack of standardized datasets available. To gather sufficient experimental data, we utilized the dataset called SB Curated, manually constructed by Durieux et al. [81]. This dataset consists of 112 smart contracts, categorized into the five most common vulnerabilities: access control, denial of service, reentrancy, time manipulation, and unchecked low-level calls.

We conducted the tests on a computer equipped with an Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50 GHz (48 CPUs) and 128 GB of memory. We chose ESXI 7.0 as the host system and used Ubuntu 20.04 as the virtual machine system.

For the batch testing script files, we used Python 3.8.10 as the programming language. Additionally, unless otherwise specified by the tools, the environment consisted of Docker version 20.10.16, npm version 6.14.4, and Java version 11.0.20.1.

We selected nine tools with different features and functionalities to cover various contract testing requirements. To ensure the accuracy of the test results and eliminate any interference from unexpected situations, we performed five tests on each contract and excluded the highest and lowest values to eliminate possible outliers. In total, we tested over 25,000 contracts. The entire testing process was time-consuming, totaling 1796.47 h. Among them, the main runtime was occupied by the Mythril tool for approximately 1700 h. To meet the running requirements of Mythril, we employed multiple servers and virtual machines for parallel execution.

In the experimental section, we conducted subgroup analysis to divide the study sample into different subgroups and compared the differences among them to draw conclusions.

To evaluate the operational efficiency of the tool, we conducted tests on the tool’s runtime. We standardized the configuration to 8 threads (4 cores, 2 threads) to simulate common setups. During the testing process, we recorded the runtime of each tool and performed a statistical analysis of the results. By comparing the runtime of different tools, we were able to assess their efficiency in handling contracts.

As shown in Fig. 13, differences in runtime between various smart contract detection tools are evident. Notably, Solmet is the fastest tool, completing the detection in just 28.53393111 s. This tool primarily focuses on extracting relevant metrics from contracts, contributing to its faster processing speed.

Next in line are Slither and Solhint, with runtimes of 60.2818644 and 77.7760623 s, respectively. Both of these tools employ feature matching techniques, which involve analyzing the relationships between feature codes, thereby requiring less time. In contrast, ContractFuzzer and Honeybadger have relatively longer runtimes, clocking in at 1134.240254 and 3048.585183 s, respectively. The slowest tool is Mythril, which takes 254086.36 s. We can observe that tools utilizing symbolic execution and fuzz testing for detection operate at a slower pace compared to those employing feature matching. This can be attributed to Slither and Solhint directly analyzing Solidity code.

To further evaluate the operational efficiency of the smart contract detection tool and enhance its performance in real-world applications, we conducted multi-core optimization testing. These tests measured the recommended number of cores utilized by different tools and provided crucial indicators for optimizing resource utilization in subsequent frameworks.

During the testing process, we modified the configuration settings to various core counts and ran the tools for performance testing, while recording the execution time. We adjusted the test configuration to 1 thread, 2 threads, 4 threads, 8 threads, and 16 threads, respectively, and noted the execution time for each tool under each configuration. Subsequently, we compared the execution times to assess the impact of multi-core optimization on tool performance.

Fig. 14 shows that, in most cases, the number of threads does not affect the running speed of the tools. This is because the execution process of these tools primarily depends on other factors rather than CPU processing power, such as the speed of input data and the bottleneck of memory reading. Therefore, the performance of some tools remains relatively stable, regardless of whether they are executed in a single-threaded or multi-threaded environment.

When studying the tools SmartCheck, Solmet, and Solhint, which utilize feature matching, we discovered their sensitivity to the number of threads through Fig. 15. Under configurations with a high number of threads, these tools exhibited only a slight reduction in running time, indicating marginal effects. However, as the number of threads decreased, the performance of these tools noticeably declined. Particularly, the transition from 2 threads to a single thread resulted in a significant increase in running time, suggesting a high level of support for multi-threaded parallel computing by these tools.

Furthermore, during our multi-threading optimization testing, we also observed some anomalies. Sometimes, increasing the number of threads did not necessarily result in a significant reduction in run time. As in Fig. 16, the slither tool could even lead to a slight increase. This may be attributed to competition among threads in a multi-core system, causing an uneven distribution of resources. In particular, the slither tool demonstrated noticeable advantages in a single-threaded environment. This probably because it can more efficiently utilize the computational resources of a single core. However, in a multi-threaded environment, its performance decreases due to the presence of resource competition.

To assess the memory usage of the system, we conducted a series of memory occupancy tests. These tests aimed to measure the amount of memory the system uses under different load conditions and provide us with crucial indicators to evaluate the system’s resource utilization.

During the memory occupancy tests, we first ensured that the system was in a stable state and recorded the initial memory usage. Then, we employed various tools to test different types of contracts, such as withdrawal contracts, game contracts, gambling contracts, and more, to simulate diverse usage scenarios. Throughout the execution process of each test scenario, we continuously monitored the system’s memory usage and recorded the memory occupancy at each time point.

To accurately measure memory occupancy, we utilized the psutil [82] module, a professional Python library for monitoring system resource usage. Throughout the testing process, we consistently used 16 GB of memory for the tests and employed the psutil module to record the memory occupancy of each contract at different time intervals.

After analyzing the experimental results in Fig. 17, we observed that Mythril and Honey Badger consume a significant amount of memory during operation, reaching 13.76 and 3.952 GB, respectively. In contrast, Solhint, Solmet, and SmartCheck exhibit noticeably lower memory usage rates, with only 82, 98, and 393 MB, respectively. Therefore, it can be concluded that symbol execution tools with deep path coverage require a higher amount of memory. Conversely, tools based on feature matching require lower memory resources.

It should be noted that certain tools exhibit a high sensitivity of memory usage to the number of cores. As shown in Fig. 18, Solmet and SmartCheck showed a memory usage increase of up to double when running in multi-threaded mode compared to single-threaded mode. This indicates that increasing the number of cores significantly increases the memory requirements of these tools. This phenomenon could be attributed to the data interaction and synchronization operations involved in parallel processing. It also explains why these tools show a significant improvement in detection speed with higher thread counts.

In this section, we conducted a practical evaluation of the vulnerability detection capability of the aforementioned tool and provided corresponding vulnerability detection results. However, honeybadger, which can only detect honeypot contracts, and Solmet, which does not test the contracts themselves, were excluded from the evaluation.

Fig. 19 illustrates the specific vulnerability detection capabilities of each tool on SB Curated. Among them, Slither, SmartCheck, and Solhint exhibit higher vulnerability coverage compared to other tools. Slither and Solhint can cover all Time Manipulation vulnerabilities, while SmartCheck can cover all Access Control and Unchecked Low-level Calls vulnerabilities. For Reentrancy, Slither tool demonstrates the best coverage. On the contrary, Mythril and SmartCheck have lower coverage for Time Manipulation, and Oyente exhibits poor coverage for Unchecked Low-level Calls.