TEE architecture and design have been explained in various documentation [8–10]. A common TEE design contains a normal world (or non-secure, also known as Rich Execution Environment in some documents) and an isolated environment called a secure world. Each of the normal and secure worlds has its operating system and kernel. An application that runs inside a secure world is referred to as a Trusted Application (TA). On the other hand, an application running in the regular operating system is called an Untrusted Application (UA).

The TEE drivers are designed as Linux kernel drivers located under/dev/, and the drivers are named differently between TEE platforms. For Qualcomm’s Secure Execution Environment (QSEE), the TEE driver is named qseecom [11]. Trustonic is another vendor of TrustZone that provided TEE drivers under the name mobicore [12]. Other projects such as Portable Trusted Execution Environment (OpTEE), OpenTee, and Intel Software Guard Extensions (SGX) also provided their own Linux drivers [9,13–15]. Trusted execution environments, such as TrustZone and Intel SGX, could gain control of hardware resources and isolate those from the normal world. Because of that design, TEEs are able to prevent attacks from the normal world to the processes that interact with the isolated resources.

In this section, this work provides an explanation of security layers in Android where trusted execution regions are placed at the highest security level. Fig. 1 illustrates the four main security layers in Android.

The order of Security Levels (SL) is increasing from the highest layer to the lowest layer (Secure mode). At the application level, protections are made through Android components such as IPC binder and Dalvik/ART runtime. The malicious apps can also be filtered out by GMS.

The next level of security in an Android device is provided from the Operating system (OS) level with Discretionary Access Control (MAC). Based on the document about Linux Security Framework (LSF) written by Wright and others [16], the LSM hook is a module that provides MAC along with Discretionary Access Control (DAC). In normal Linux, there are many options for MAC, such as AppArmor (for Debian distribution), SELinux (for Redhat distribution), and more. For Android, the MAC option is a modified version of SELinux, which is called as SEAndroid. From Android 10, the control group (Cgroup) is also a part of the security layer for controlling the resources of a specific process or thread.

The Hypervisor (HYP) mode is the next privileged level in ARM architecture which is higher than kernel privilege at the OS level. Hyp mode is used for hosting the hypervisor code in a secure CPU state [17]. However, the HYP mode still has a lower security level than the TEE design since it only supports CPU protection. On the other hand, TEE could support various resource protection, including memory, GPU, and other security elements.

Hardware-based TEEs, as illustrated in Fig. 2a, have been invented by various CPU vendors and deployed in mobile devices for several years. TrustZone, a System-on-Chip (SoC) security solution for securing applications and data, was introduced in 2004 and officially released by ARM in 2005 [18]. In 2016, Intel introduced another hardware-based TEE solution in the name of Intel SGX [15]. As for the open-source community, Open OpTEE is a project developed and supported by Linaro Limited [14]. The project is commonly known since it supports various development boards as well as Quick Emulation (QEMU) environment. Andix OS is another open-source TEE project that mainly serves research purposes on TrustZone [19].

Software-based solutions is more attentive to solve limitations of hardware-based ones. Firstly, it is cheaper to have software-based solutions deployed because it doesn’t depend on specified hardware requirements such as ARM TrustZone, Intel SGX. Secondly, deploying software-based one is easier to add new function when non-OEM developer need to add special security functions to their application.

Software solutions for deploying TEE are mostly built on top of virtualization environments, as shown in Fig. 2b. A virtualizing OS is run by the host system to provide trusted applications for securely computing and processing sensitive data. Previous researchs on developing software-based TEE environment could be categorized as follows.

Intra-privilege isolation: known as same-privilege isolation, where the virtualized TEE is located in same security level to the rich world application. Not many studies focus on this approach due to less security provided. Related techniques proposed to enhance isolation for virtualization in ARM platforms [20,21].

Inter-privilege isolation: there are couple of techniques proposed to deploy isolation environment using ARM platform [22,23]. These techniques leverage the CPU separation of ARM HYP mode to provide isolation. Li et al. [24] proposed a smaller trusted computing base (TCB) size than using KVM in term of line-of-code with the goal of protecting integrity and confidentiality for virtual machines running in cloud platform. Open-TEE which provides a software-based TEE environment followed by the Global Platform Standard [9]. In CCS ’19, Zhao et al. introduced SecTEE [25], their Software-based architecture, which was built on top of OP-TEE. SecTEE could resist side-channel attacks self-sufficiently with its software-based enclave architecture. Another approach presented by Lee et al. is SofTEE [26]. Their solution provides a memory isolation space by separating the kernel into the normal and secure worlds, which is inspired by the ARM TrustZone.

In short, software-based TEE environments have been developed for different purposes using various approaches. However, these works didn’t show the way to adjust between the security and performance of virtualized TEE solutions. Furthermore, virtualized TEE solutions are mostly implemented for cloud computing but are limited to Android platforms.

On the one hand, Hardware-based solutions rely on chipset technological support such as TrustZone, SGX, etc., which might lead to the difficulty for OEM developers to provide cross-platform trusted services. On the other hand, current software-based approaches, to our best knowledge, have not supported Android mobile devices, the most popular in the mobile market [27]. Motivated by the existing problems, this study introduces a software-based TEE model called Virtualized TEE (or VTEE).

This section introduces VTEE model with an analysis of the effectiveness of the model for the existing problems of hardware-based TEEs. Fig. 3a illustrates how VTEE can be a part of the Android system as well as the design of the VTEE broker model. The proposed model adds a TEE broker service to the normal world that contains the following components:

VTEE Library is an Android library (.jar) file for providing an API interface to the VTEE service. This library file is a part of the VTEE broker model.

The function of this proposed model can be summarized as follows. Firstly, untrusted applications in normal-world requests for secure computation by calling API from the VTEE library. The API helps the application establish, transfer and receive responses from VTEE Service App. Then, the VTEE Service App collects requests and forwards data to VTEE Sandboxes securely. VTEE Sandboxes deploy TA inside an isolated environment, ensured by HYP mode, following the request data from UA. Finally, TEE functions are safely computed in side TAs, and the result will be sent back to UAs via the broker model.

The application itself does not have enough and should not have the higher privilege to escape its sandbox. Therefore, a library that provides sound interfaces that acts as a bridge between the application and the VTEE service app. There are two main challenges of deploying VTEE lib: (1) Authentication & authorization; (2) Data transferring.

For authentication & authorization, it is easier whether the private services models are taken place. However, when public services are chosen as the default provider, there is a need for a lightweight CA model, whereas the UA is required a token to access its data or functions provided by the VTEE sandbox. That also ensures that an adversary cannot access other UA data inside the VTEE sandbox.

VTEE service app is a system-privileged application that starts along with the Android OS. VTEE service app binds with the UA using VTEE lib. VTEE service app has full access to the communication channel with the VTEE Sandbox. VTEE service app has a filtering system that ensures UA can get enough required information without breaking the sandbox (e.g., prevent privilege escalation attack). Note that the VTEE service app does not have any encryption module or hold the encryption key. Therefore, the VTEE service app only forwards the data to the VTEE sandbox without access to the clear-text data.

VTEE sandbox plays the core role in the VTEE Broker Model. It provides TEE services whenever receiving a request from VTEE service. The sandbox contains a request handler (inside), customization of the OPEN-TEE project for providing TEE services, and a lightweight CA system (optional) for authentication while the public service model is set to default. VTEE Sandbox is designed to run at the most privileged level, which can be in HYP mode (L2) or OS (kernel) level (L1). The design depends on the hardware, which PL2 could be considered when hardware-supported virtualization is enabled [28]. In this work, the side-channel attack is out-of-scope. Therefore, the VTEE sandbox is intended to be safe in the highest privilege level.

In the proposed model, there are two ways to provide VTEE services: Public service and private service. Fig. 4 illustrates the difference between public and private services. The Public service takes responsibility for providing security services for every application in the system. The Public service should always be on and available for any request forwarded by the VTEE service. The purpose of public service is to reduce the latency which is caused by the initialization of the VTEE sandbox (provided in the Experience part). However, the Public service requires more resources (which is one main concern of the mobile environment) while its VTEE sandbox constantly manipulates a lot of memory disk space during the phone working. Also, security design for Public service is more complicated. In the Public service model, a VTEE sandbox is deployed at the Android booting time and is available for serving until Android OS is turned off.

The Private service takes responsibility for providing security services for the only corresponding application. Therefore, a VTEE sandbox is only deployed whenever an application requests a set of security services (e.g., in a TEE session). In this way, the Private service requires less security design and fewer resources manipulation. However, the latency from the session opening to the first security service response is much longer due to the VTEE sandbox initialization progress.

The execution time of each algorithm was measured by the total costs of application requests, application queries, channel communication, and TEE calculation. There are various ways to create a communication channel between VTEE service and VTEE sandboxes, such as 9p (Plan 9 Filesystem Protocol) [30] or shared memory. 9p acts as virtio for file transferring enabled by default in lkvm. 9p allows us to share files between a host and guest VMs. This approach is a passive way to trigger TEE functions and often causes high delay. On the other hand, shared memory is an active way to handle request data and trigger the TEE function. Thus this method can be considered one of the low-delay solutions. Fig. 6a illustrates the performance between running the Open-TEE project [9] in a normal environment and inside the VTEE sandbox. The sandbox causes at most 30% overhead and only a few percent of overhead in some algorithms, such as AES (Advanced Encryption Standard) or SHA (Secure Hashing Algorithm). It is reasonable since the VM is used HYP mode, a hardware-assist virtualization technology, to support CPU virtualization.

As introduced in Section 3 there are two services available in our model, which are public and private VTEE services. This experiment measured the execution time between two services. Fig. 6b shows that there is a difference in available time between public and private services. The research has shown that the Open-TEE client will have to wait from 75 to 90 s for the TEEC_OpenSession call to work in the virtualized environment. This incident still occurred in public service, and the public TEE service was available at the time of application request because it started at the booting time, which already consumed 90 s of delay time. We also confirmed that the TEEC_OpenSession has not occurred in a normal Linux system.

This experiment explored the feasibility of the deployment of VTEE in commercial products, which consider the replacement of hardware security solutions due to the incompatibility and the scalability limitation. A popular hardware-based TEE Operation system (TEE-OS) is OpTEE [14].

Two hikey960s were deployed with the same Android OS version (Android 9.0). The data flow of the test, from the android application to the Trusted application, is deployed in the same way, as shown in Fig. 7. Firstly, when UA requests to use secure functions from the TEE service, it sends data to handle service which runs as Android system service. Handle service, then transfer input data to the running OPTEE/VTEE instance. While the 9p protocol is a part of the handle service in VTEE, the VTEE model emulated the same file-sharing function in the OP-TEE version, in which input data is stored inside a sharing file between Android App and either OP-TEE or VTEE. This ensures equality in data transferring between the Android application and the TEE component (OP-TEE and VTEE). After receiving data, OPTEE/VTEE calls to deploy a TA corresponding to the requested function. TA verifies and computes the result and then sends back to OPTEE/VTEE instance as well as the UA.

Fig. 8 shows the comparison between executing ten algorithms in OP-TEE and VTEE. The results differed from expectations that direct program execution at the hardware (OP-TEE) would be faster. However, most algorithms (except RSA) give the opposite results.

The unexpected performance in software-based TEE is because the number of data transfer points in the OP-TEE is greater than VTEE. Data transfer points are the points that data passes through the process without memory sharing. OP-TEE OS runs in the TrustZone, which requires data transfer from the Normal world to the Secure world by the UA before being processed in Trusted Application (TA). On the other hand, the VTEE sandbox, which is handled by UA and TA, does not need to switch between two worlds due to the VTEE Sandbox is placed in the same layer with the VTEE service (acts as a data handler). Therefore the total time-consuming: T=TComputing+TData transfer

Another reason is that most algorithms witness the faster VTEE except RSA (sign and verify). This result proves that OP-TEE has a shorter computing time than VTEE because RSA requires harder computing power which always performs better in hardware-based solutions rather than software. Other algorithms, which are lightweight computing functions, consume less TComputing than TData transfer. The consequence is that the total time-consuming T of VTEE (on average) less than OP-TEE.

The above argumentation supports the high feasibility of delivering VTEE as a Trusted Computing Environment in the wide-range commercial product, which can replace hardware-based solutions from the performance perspective.

According to Fig. 3b provided in Section 3, it is obvious that VTEE sandboxes have less security than traditional hardware-based TEE designs. However, in order to evaluate the security of the proposed model, this research contains various methods for inspecting different Android OS layers. For that, the first version moved the private key of an Android app into the VTEE sandbox. Additionally, another Android app was developed for calculating the crypto algorithm inside.

The result is illustrated in Table 1. The evaluation methods include Dalvik VM (DVM) heap dump, root-based, memory forensic, and side-channel attacks. The first method is DVM heap dump which can be implemented by using the Android-integrated Activity Manager (AM) function. The full memory of a target Android app can be exported to HPROF format (a Heap/CPU profiling tool). The memory of the VTEE Android app and normal Android app can be inspected by this method. However, the private key was not leaked in the VTEE model since the key was stored in the VTEE sandbox. The same result was given from a root-based attack. Because the VTEE VM can be custom with different contexts, it is not possible to attack from root privilege without changing the context. According to the design of the Android framework, the SELinux context in Android can only be changed by modifying the AOSP source code or by the vendor.

Another inspection approach is the memory attack, which can be implemented with memory forensic tools and Loadable Kernel Module (LKM) enabled. By satisfying the LKM condition, we were able to disclose VTEE VM memory from known traces. At the time of writing, side-channel attacks are able to exploit all layers of Android. The levels of security in Android OS with the VTEE model after our security evaluations. The table shows that VM-based VTEE sandbox is at the lowest level in untrusted OS. Although the proposed model does not have a large impact on security, it provides an additional layer of isolation and security to the existing Android system.

VTEE, therefore, can play an important role in enhancing the security of Android applications, especially on IoT devices with a lavish hardware-based TEE. For example, cars’ smart screen system running Android OS hosts many applications. Even if there are no transactions have been made on the smart screen application, other applications still contain sensitive personal information of the car owner, such as premium accounts, usage caches, and location information for the recommendation system. These data also need to be safe against cyber attacks but require a lower level of security where VTEE is applicable.

The limitation of entirely software-based TEE solutions is inevitable, even though VTEE cooperates with a hardware-based solution to protect the sandbox. However, VTEE could keep the higher security level for the rest. While most side-channel attacks are impacting the VTEE, other resource manipulation attacks are impeded. The trade-off between convenience and security always happens. VTEE model maintains the security perspective in protecting users from more common attacks while delivering more flexibility in TEE function development.

As discussed in the previous section, software-based TEE solutions cannot provide as strong security as hardware-based ones. However, software-based solutions such as VTEE allow the deployment of multiple virtualized TAs, running parallelly, isolating from each other and from UAs. Thus, when a system does not require strict security enforcement, software-based solutions are suitable to increase computing power.

In this work, VTEE provides two approaches to strike a balance between security and performance. The first approach is privilege-level isolation. As discussed in Section 3.5, VTEE sandbox can be deployed in HYP mode (L2) or OS (kernel) level (L1). When the VTEE sandbox is run at the OS level (i.e., intra-privilege isolation), the isolation is based on virtualization technology and prior SELinux. On the other hand, deploying the VTEE sandbox in HYP mode provides stronger isolation because it provides CPU protection. In other words, a process needs to be granted to access HYP mode computing resources. As a result, switching between L1 and L2 is costly, but more safety is ensured. The second approach is to move the broker provider from the public to the private model, as discussed in Section 3.6. While private service gives more security (e.g., each UA has a separated TA), the performance decreases more than two times, as shown in Fig. 6b.