In [50], Willi and Othmar first proposed the idea of nonlinearity (NL) for an S-box S. The NL evaluates the linear relation between the inputs and outputs of an S-box. Mathematically, NL(S) of an S-box can be computed by:

NL(S)=min{2n−1−12maxu≠0|∑x∈GF(2n)(−1)Si(x)⊕u⋅x|},where, u∈GF(2n), and Si represents the coordinate Boolean function of S, while “⋅” and “⊕” denote the dot product and addition over GF(2), respectively. The higher value of the NL stands for high resistance to linear attacks. The optimal value of NL for an n×n S-box is 2n−1−2(n/2)−1. The NL value of the S-boxes outlined in Tables 3 and 4 are 106 and 112, respectively. These NL results are high enough to thwart the attacks.

Matsui presented the idea of linear approximation probability (LAP) in [51] for evaluating the strength of the S-box against linear attacks. To analyze the security of S, LAP is used to quantify the linear characteristics and correlation between the input and output bits. The following expression is used to compute the LAP of an S-box: LAP⁡(S)=12n(maxρ,σ|#{x∈GF(2n):ρ⋅x=σ⋅S(x)}−2n−1|),where, ρ∈GF(2n), σ∈GF(2n)∖{0}, and “⋅” denotes the dot product over GF(2). The lower value of the LAP indicates high resistance against linear attacks. The LAP results of the S-boxes demonstrated in Tables 3 and 4 are 0.0312 and 0.0078, respectively, which are very low.

In [52], Biham and Shamir first provided the differential approximation probability (DAP) test. The DAP is a well-known test used to determine the security of an S-box against differential attacks. The DAP measures the probability of the precise difference between input and output bits. That is, it quantifies the probability of a specific differential characteristic occurring within the S-box S. The DAP can be calculated by the following equation: DAP(S)=12n(maxΔs1, Δs2#{x∈GF(2n):S(x)⊕S(x⊕Δs1)=Δs2}),here, Δs1 and Δs2 belong to GF(2n), and “⊕” represents the bit-wise addition over GF(2). For the low value of DAP, the S-box is considered more strong against differential attacks. The DAP of the S-boxes exhibited in Tables 3 and 4 are 0.0390 and 0.0156, respectively, which are very low.

The concept of strict avalanche criterion (SAC) was first proposed by Webster and Tavares in [53]. The SAC is a useful tool to determine the security of S against Boolean function attacks by quantifying the substantial change in an S-box output bit when a single input bit is varied. For the Boolean functions Si, and ρp∈GF(2n), the SAC is computed using an n×n dependence matrix N=[dij], whose entry dij is calculated as follows: dij=12n(∑x∈GF(2n)w(Si(x⊕ρj)⊕Si(x)) ),where w(r), for r∈GF(2n), denotes the number of non-zero bits in the vector r. The S-box S is said to satisfy the SAC and has high cryptographic strength against Boolean function attacks if every off-diagonal entry of N is close to the optimal value of 0.50. The SAC values of the S-boxes in Tables 3 and 4 are 0.4218, 0.5937 and, 0.4531, 0.5625, respectively. These values are sufficiently close to the optimal value 0.50, exhibiting strong avalanche properties.

The idea of the Bit independence criterion (BIC) was given by Webster and Tavares in [53]. The BIC is also used to determine the security of S against Boolean function attacks by measuring the variation in a pair of S-box output bits when a single input bit is changed. Let D be a BIC matrix and defined as D=[zqr], where zqr is computed by: zqr=12n(∑x∈GF(2n)1≤q≠ℓ≤nw(Sq(x⊕ρr)⊕Sq(x)⊕Sℓ(x⊕ρr)⊕Sℓ(x))).

The S-box is considered to fulfill the BIC criteria and provide good cryptographic strength against Boolean function attacks if every off-diagonal entry in the matrix D is close to the optimal value of 0.50. The BIC values of the Tables 3 and 4 are 0.4707, 0.5293, 0.4883, and 0.5234, respectively, which are extremely close to the optimal value 0.50.

Sakall et al. [54] proposed the concept of algebraic complexity (AC) to test the resistance of an S-box against an algebraic attack. The AC represents the number of non-zero terms in the linear polynomial representation of an S-box. As 2n−1 is the optimal value of the AC for any n×n S-box. The AC of the S-boxes in Tables 3 and 4 attains the optimal value 255.

It follows from the above-attained values of the proposed S-boxes that the presented algorithm has strong cryptographic strength. However, to rigorously analyze the new algorithm, a comparison with recently developed schemes is conducted below.

We run the algorithms [42–44] for initial valid values of prime p and b to generate the first 1000 S-boxes using the MEC y2≡x3+b(modp), and the experimental results are shown in Table 9. The results indicate that the presented scheme needs fewer input points, a smaller value of prime than the schemes in [42–44] to generate a comparable number of distinct S-boxes. The number of total possible proposed S-boxes is greater than those of [42–44] for every initial valid underlying primes. More explicitly, the current algorithm begins to perform for very small primes, whereas the mentioned schemes are incapable of generating a single S-box for primes less than 257.

Apart from the generation of a large number of distinct S-boxes, an S-box generator is considered to have good dynamic behavior if the mean value of the fixed points in the generated S-boxes is considerably small [61]. The round-off mean values of the fixed points (#Fixed points) for the proposed S-box generator and the generators presented in [42–44] are drawn in Table 9. It is obvious that the proposed algorithm has comparatively more dynamic behavior than those of [42–44].

An S-box algorithm is said to have singular points if it does not generate an S-box for a valid input parameter, and hence, the algorithm becomes singular. An S-box generator is considered favorable for cryptographic applications if it is nonsingular. The comparison of the presented algorithm with respect to the singular points is presented in Table 9. It can be verified that the current algorithm is also nonsingular.

An S-box generator is promising for cryptographic applications if it can efficiently generate the resulting S-box [40]. More precisely, for a favorable S-box construction method, it is necessary to frequently output dynamic S-boxes in a minimal time. We analyzed the computation time of the current and some recent schemes for generating the above 1000 S-boxes. The experimental results are shown in Fig. 3.

It is clear that the average time for the proposed S-box is 0.50 s, whereas the average time required for the S-boxes [42–44] is 0.63, 82.76 and 0.80 s, respectively. Furthermore, the current scheme takes only 0.57 s to generate 1000 S-boxes, whereas the schemes in [42–44] require 0.70, 188.50, and 0.95 s, respectively, to generate the same number of S-boxes. Consequently, our algorithm is faster than those of our competitors. The speed of the current algorithm is superior to that of the algorithm in [43] because the newly developed method does not require generating all points for S-box generation.

Apart from this, we picked only the first 100 S-boxes generated by the new scheme and the aforesaid schemes. The NL ranges for the proposed scheme and the schemes in [42–44] are [86, 104], [86, 104], [90, 104], and [90, 102], respectively. Thus, the range for the proposed scheme is comparable with the ranges of the scheme [42,44]. However, if we choose a bit larger prime for the current algorithm and the chosen parameters illustrated in Section 4, then the NL range for the proposed technique becomes [92,104] and hence the number of S-boxes for each NL value in the corresponding range is shown in Fig. 4. It follows that the minimum NL value (92) for the proposed algorithm is greater than the minimum NL values (86,90,90, respectively) attained by the schemes [42–44]. In addition, the current scheme generates more S-boxes of NL = 104 than the schemes in [42–44]. Hence, the proposed algorithm is more capable of generating highly non-linear S-boxes with small input parameters.

To validate the effectiveness of our S-box generator, we conducted independent repeated trials to produce 100 S-boxes using the various parameters described in Section 4. For each generated S-box, we calculated cryptographic properties using standard metrics, including LAP, DAP, BIC, and SAC, and outlined the results in Fig. 5. The LAP values for 100 S-boxes are shown in Fig. 5a, falling in the range [0.0156, 0.0937] with an average of 0.0497, which are low, indicating that our generator produces secure S-boxes against linear attacks. Similarly, the DAP results for the same set of 100 S-boxes are depicted in Fig. 5b. The range of the DAP is [0.0390, 0.0625], having average of 0.0442. That is low, and these findings confirm that constructed S-boxes exhibit high resistance to differential attacks. Moreover, the SAC (min) and SAC (max) test results for 100 S-boxes are presented in Fig. 5c. The ranges [0.3437, 0.4375] and [0.5468, 0.6562] correspond to the SAC (min) and SAC (max) values, indicating averages of 0.3982 and 0.6053, respectively, which are closer to the desired value of 0.5. Likewise, the BIC (min) and BIC (max) test outcomes of 100 S-boxes are depicted in Fig. 5d. The BIC ranges [0.4433, 0.4882] and [0.5156, 0.5566] with average BIC (min) and BIC (max) values 0.4710 and 0.5330, respectively, which are close to the ideal value 0.5, indicating that our generator can produce S-boxes with the desired confusion and secure against Boolean attacks. As a result of the repeated-trial analysis, the suggested method consistently generates dynamic S-boxes with satisfactory cryptographic properties and is independent of specific parameter choices. Hence, the proposed S-box framework offers considerable resistance against various cryptographic attacks.

Since the proposed algorithm uses six keys that are p,b,c,d,e and f. In which only p is real, while the remaining five keys consist of two parts. The proposed algorithm is implemented by choosing p of eight bits. However, the number of bits may be increased by selecting a large p. In general, if p consists of μ bits, then the key space of the proposed cryptosystem is 211μ, which is much larger than the key spaces 22μ,23μ,24μ,29μ,29μ of the schemes [39–42,44], respectively. Thus, the new scheme offers more security to thwart unauthorized attempts of attackers.

A cryptosystem is considerably secure to key-related attacks if a minor change in any key results in a different output. To analyze this property of the presented algorithm, we generate S-boxes against different altered keys, as listed in Table 10. We have 12 S-boxes, where σ1 is the one for unchanged keys, as shown in Table 3, and σi,i=2,3,…,12, for the slightly changed keys, are shown in Table 10.

Then we compute the correlation coefficient (CC) between each pair (σi,σj) for i≠j of S-boxes generated by the original keys and slightly changed keys. The CC must be close to zero for a highly key sensitive S-box algorithm. The results are shown in Fig. 6, where the average CC (0.0085) indicates that a minor change in any key produces a completely uncorrelated and different output.

In this part of the article, we compare and discuss the results of the encrypted images obtained by the current and existing S-boxes.

If an encryption approach can produce considerably distinct encrypted images from two slightly distinct plain images, then it is assumed to be secure against differential attacks. The resilience of an encryption technique against a differential attack is quantified by two metrics: NPCR (Number of Pixels Changed Rate) and UACI (Unified Average Changing Intensity). The theoretical values of NPCR and UACI for the images of size 256×256 and 512×512 are 99.5693, [33.2824, 33.6447] and 99.5893, [33.3730, 33.5541], respectively. For the aforesaid images, we computed the NPCR and UACI values as listed in Table 11. It may be observed that the NPCR values for the Resolution chart, Chemical plant, Herringbone weave, and San diego are according to the theoretical values and better than the results acquired by the [42,68]. Similarly, the UACI results of the images based on our S-box fall in the theoretical ranges and are better than results based on the S-boxes of [42,68]. Therefore, the encryption approach using our S-box shows stronger resistance against differential attacks than the results for the S-boxes of [42,68].

A secure encryption algorithm must be capable of effectively resisting all kinds of statistical attacks. We applied various tests, including histograms, correlation, and entropy tests, to evaluate the robustness of the presented image cryptosystem against such attacks.

Entropy is a statistical metric used to quantify the randomness of secret data. It depicts the intensity distribution and unpredictability among the image pixels. An encrypted image with a higher entropy can protect more information when compared to an encrypted image with a lower entropy. The ideal value of entropy for an encrypted image is 8. We computed the entropy of all cipher images shown in Fig. 7 and the results are depicted in Table 11. It follows that entropy results for images such as the Resolution chart and Chemical plant when ciphered by the new S-box are comparable with images encrypted by the S-box in [68] and are close to the entropy obtained due to the S-box in [41]. Likewise, for encrypted images, Herringbone weave and San diego, entropy results for the current S-box are better than those of the S-box in [68] and are comparable with those of [41]. Consequently, the encryption method using our S-box exhibits stronger resilience to statistical attacks than [68] and achieves comparable resistance with [41].

The pixels of a plain image are significantly connected to the adjacent pixels. A secure encryption algorithm must generate encrypted images with extremely low inter-pixel correlation between their pixels. Ideally, bring it close to zero so that no information may be obtained by a cryptanalyst. We employed correlation analysis to compute the correlation coefficient in horizontal, diagonal, and vertical directions to examine the pixel correlation in the enciphered images. The outcomes obtained by the correlation test are depicted in Table 11. It follows that correlation results for cipher images of the Resolution chart and Chemical plant along three different dimensions, such as horizontal, diagonal, and vertical, using our presented S-box, are better than results obtained from the S-boxes of [41,68]. Alike, for encrypted images such as Herringbone weave and San diego, correlation results using the new S-box are better than the S-box in [68] and are comparable with the S-box of the scheme [41]. The encryption approach using the newly designed S-box demonstrates stronger resilience to statistical attacks than utilizing the S-boxes in [41,68].

The uniformity of the histogram of encrypted images demonstrates the level of security of an image encryption scheme. An encrypted image with a uniformly distributed histogram can effectively resist statistical attacks. The histograms of the cipher images are exhibited in Fig. 8. It is evident that the histograms of the enciphered images generated using our S-box generator in the proposed encryption scheme are uniform. As a result, an improved cryptosystem provides robust security against statistical attacks and successfully passes the histogram analysis.

The key space refers to the total number of distinct keys that can be utilized during the encryption process. It is important to recognize that the security of a cryptographic system is strongly correlated with the size of its key space. If the key space is large, then the attacker will find it more difficult to break the cryptosystem using a brute-force attack. The S-box algorithm proposed by Hayat et al. [41] has a key space 24μ, and our S-box generator has a key space 211μ. Thus, the encryption algorithm using our S-box generator has strong security against key attacks.

To evaluate the randomness and cryptographic robustness of the proposed scheme, after its integration into our S-box generator, the National Institute of Standards and Technology (NIST) Statistical Test Suite (SP 800–22) [69] is employed to an arbitrary encrypted image. The NIST test suite is a collection of widely recognized, standardized tests which is used to assess the randomness of binary sequences generated by cryptographic schemes. These metrics ensure that cryptographic algorithms produce outputs that are unpredictable and maintain cryptographic security, thereby preventing statistical inference and pattern-based attacks. The results obtained after successfully applying the NIST test suite are demonstrated in Table 12. The test results indicate that the probability (p-values) of all tests exceed the accepted significance threshold of 0.01, demonstrating that they successfully pass each NIST test suite and achieved good randomness. These consistent p-values verify that the encrypted image meets the criteria of statistical randomness, free from detectable patterns or structural biases, which is essential for cryptography. Therefore, the presented S-box algorithm exhibits strong resistance against statistical and pattern-based attacks, proving highly suitable for securing image data with high randomness.

To examine the encryption performance of the presented scheme, three fundamental quantitative metrics are utilized in this study: the Peak Signal-to-Noise Ratio (PSNR), the Structural Similarity Index (SSIM), and the Bit Correct Ratio (BCR). The PSNR is a widely used, standard metric for assessing image quality by comparing the original and decrypted images. A higher PSNR value indicates a more precise decryption process and a higher-quality decrypted image, reflecting that the decrypted image is more similar to the original plain image. The PSNR is quantified utilizing the Mean Square Error (MSE) between the original plain and its decrypted image. On the other hand, the SSIM measures the degree of structural information preserved after decryption. It computes visual similarity, contrast, and structural correlation in plain and ciphered images. Likewise, the BCR quantifies decryption accuracy by computing the ratio of accurately retrieved bits to the total number of bits compared between the plain and encrypted images. It offers a numerical measure of decryption accuracy and the overall encryption process. We computed the MSE, PSNR, SSIM, and BCR values for the plain Cameraman image and the corresponding decrypted image, and summarized the results in Table 13. Under the noise-free scenario, it is evident from Table 13 that the decrypted image generated using the proposed algorithm is exactly similar to the plain image. As a result, the MSE output is zero, the PSNR outcome approaches ∞, and both the SSIM and BCR results are equal to 1. Higher scores for these metrics indicate a more effective encryption process, revealing that negligible data loss occurs during decryption. Hence, the encryption approach, after using our S-box algorithm, removes the visual and structural features of the original plain images.

During transmission, some image data may be lost due to noise. An encryption algorithm must possess strong anti-noise properties, maintain security even in noisy transmission scenarios, and ensure reliable image decryption. To evaluate the robustness of the encryption scheme against noise interference, a Cameraman_256×256 test image is chosen. We add Salt-and-pepper noise to the encrypted image at varying noise densities of 0.01, 0.05, 0.1, and 0.2, respectively. The performance of the presented algorithm against a noise attack is illustrated in Fig. 9. From Fig. 9, we observe that the decrypted images are visually recognizable across distinct noise densities, preserving the essential visual content of the plain images even after sufficient noise is added. This reveals that the generator exhibits strong resilience and high efficiency against noise-based attacks.

In addition to noise interference, partial data loss may also occur during image transmission over networks. For a reliable encryption scheme, it is necessary to recover as much information as possible from encrypted images even in the presence of such losses. To examine the robustness of the presented scheme under these conditions, the ciphered image of Cameraman_256×256 is cropped at varying rates of 1/16%, 1/8%, 1/4%, and 1/2% of the total pixels, and the resulting cropped images are decrypted. The performance results are demonstrated in Fig. 10. It is evident from Fig. 10 that, even with substantial data loss in the ciphered image, the decrypted images successfully reconstruct the primary visual content of the original image. These findings confirm that the proposed algorithm has strong resistance to cropping attacks, demonstrating its ability to effectively recover key image information even when partial data is missing. These results collectively validate the high security and robustness of the proposed encryption method against data loss attacks.