BraTS 2020 [45] is rigorously selected and annotated to facilitate the training and testing of machine-learning methods for brain tumor segmentation. The dataset in Table 1 includes multi-parametric MRI scans, which are vital for modeling to reflect the complexity of the brain tumors.

With the BraTS dataset, there is a solid starting point from which a better AI-based model for detecting brain tumors can be developed. The discriminative and comprehensive multi-modal MRI scans of the dataset help the DenseNet architecture solve the problem of brain tumor diagnosis with higher accuracy due to detailed and less abstract learning of peculiarities.

Preprocessing is essential for making MRI images as close as possible to the neural network training input data and making it every day. MRI images were then normalized to minimize pixel differences. In particular, image pixel intensity values were normalized such that their mean value was equal to zero and the standard deviation was equal to one. Such normalization helps bring all the input images into the same scale, which is essential, especially since we’re using a neural network. The quantitative conversion is made uniform by applying the same scaling factor, thus increasing the train rates while improving the predictions’ accuracy. Each MRI image is initially normalized by subtracting the mean value and dividing it by the standard deviation. This will also enhance the model’s learning speed and convergence, as it is built on a single standard scale. For each image, the overall mean of pixel intensity is computed. This is the average of the absolute value of its pixels divided by the total number of pixels in the image.

The standard deviation, which measures the spread of pixel intensity values around the mean, is also computed. The formulas for the mean (μ) and standard deviation (σ) are: (1)μ=1N∑i=1Nxi (2)σ=1N∑i=1N(xi−μ)2where xi represents the intensity value of the ith pixel and N is the total number of pixels in the image.

Each pixel intensity xi is transformed using the calculated mean (μ) and standard deviation (σ) to get a new value xi′ such that the resulting image has a mean of zero and a standard deviation of one. Normalization calculated by: (3)xi′=(xi−μ)σ

MRI images often contain noise from machine vibrations, thermal fluctuations, or operator variability. Gaussian blurring was applied to reduce noise, a technique that uses a Gaussian function to smooth the image, minimizing minor distortions that could obscure important features like tumor regions. This step helps clarify the MRI scans, allowing the model to focus on relevant image details. One effective method for noise reduction in these images is Gaussian blurring, which uses a Gaussian function to smooth the image and reduce the noise level. The Gaussian filter is defined by the Gaussian function, which in two dimensions has the form: (4)G(x,y)=12πσ2e−x2+y22σ2here, x and y are the distances from the origin in the horizontal and vertical axes, respectively, and σ is the standard deviation of the Gaussian distribution. The parameter σ controls the extent of smoothing: a larger σ results in more blurring as the function’s width increases.

The image is convolved with the Gaussian kernel to produce the smoothed image. The convolution of an image I with the Gaussian kernel G is given by: (5)I′(x,y)=(G∗I)(x,y)=∑i=−kk∑j=−kkG(i,j)⋅I(x−i,y−j)where ∗ denotes convolution, I′(x,y) is the pixel value of the processed image at the position (x,y), and k is the kernel size, typically chosen based on σ (e.g., k=3σ).

Data augmentation techniques are extensively used to increase the diversity of the training data and simulate various real-world scenarios. Augmentation helps combat overfitting and enhances the model’s generalization ability to new, unseen data. Here are the key augmentation strategies employed. Several data augmentation techniques are used to simulate a range of clinical imaging scenarios. Random rotations within a range of −10 to 10 degrees address variations in patient positioning during MRI scans. Translations along the horizontal and vertical axes simulate slight shifts in perspective while scaling adjustments mimic different resolutions encountered in clinical practice. Additionally, the snapshots are scaled by other factors to reflect the diversity in image resolutions encountered in one-of-a-kind clinical settings. Additionally, we carry out horizontal and vertical flips on the snapshots, growing the dataset’s variability and ensuring the version can efficaciously cope with photographs in extraordinary orientations. Lastly, elastic deformation is used, using non-linear ameliorations to the pictures to emulate anatomical variations and the variations that can increase in MRI scan acquisitions. These augmentation strategies decorate our version’s robustness, preparing it to cope with real international imaging variations.

A random angle rotates images θ, where θ is chosen uniformly from a range, e.g., [−10∘,10∘]. The rotation transformation for a point (x,y) in the image can be represented using a rotation matrix: (6)[x′y′]=[cosθ−sinθsinθcosθ][xy]

This transformation rotates the pixel coordinates (x,y) to new coordinates (x′,y′), altering the image’s orientation without changing its dimensions.

Images are translated randomly in both the X and Y axes. If tx and ty represent the translation distances along the X and Y axes, respectively; each pixel (x,y) in the original image is moved to (x+tx,y+ty). This can be expressed as: (7)[x′y′]=[10tx01ty001][xy1]

This matrix operation shifts the location of each pixel, effectively mimicking the shift in imaging perspective.

Scaling is applied to simulate different image resolutions. A scaling factor s uniformly chosen from a predefined range, e.g., [0.9,1.1], resizes the image. The scaling transformation for a point (x,y) is: (8)[x′y′]=[s00s][xy]

This adjusts each pixel’s coordinates to expand or contract the image, reflecting different resolutions encountered in clinical settings. Flipping operations are either horizontal or vertical. For horizontal flipping, each pixel’s x-coordinate is inverted: (9)x′=image width−x−1

For vertical flipping, the y-coordinate is inverted: (10)y′=image height−y−1

These transformations flip the image across the respective axes, enhancing the dataset’s variability by accommodating images in different orientations.

Elastic deformation introduces non-linear transformations that mimic natural anatomical variations. It uses a displacement field. (Δx,Δy), where each displacement component is sampled from a Gaussian distribution. The transformed coordinates (x′,y′) are then given by: (11)x′=x+Δx(x,y),y′=y+Δy(x,y)

A Gaussian kernel typically smooths these fields to ensure realistic deformations and not introduce sharp artifacts.

The DenseNet used in our study is configured with specific characteristics optimized for segmenting and classifying brain tumors from MRI data.

DenseNet architectures are mainly known for their parameter usage and feature propagation efficiency. The core idea is that each layer is directly connected to every other layer in a feed-forward fashion. Mathematically, for a given layer l_l, the feature maps of all preceding layers [x0,x1,…,xl−1] are used as inputs: (12)xl=Hl([x0,x1,…,xl−1])where Hl(⋅) represents a composite function of operations—Batch Normalization (BN), followed by a ReLU activation and a 3 × 3 Convolution (Conv).

To better capture the complexities of MRI data, we increase the depth of the network. Deeper networks generally perform better at learning features from complex data. However, to prevent the vanishing gradient problem, each DenseNet block is equipped with skip connections, ensuring that the gradient can propagate back through the network without significant loss:

Each layer in the DenseNet is designed to receive contributions from all preceding layers. The output of the llth layer (xl) is formulated using a function that combines the outputs of all previous layers through skip connections and a transformation Hl. Mathematically, this is represented as: (13)xl=xl−1+Hl([x0,x1,…,xl−1])here xl−1 the output from the previous layer, and Hl represents a composite function of batch normalization (BN), ReLU activation, and convolution (Conv).

The function Hl applied to the concatenated outputs of all preceding layers is defined as: (14)Hl([x0,x1,…,xl−1]=Conv(ReLU(BN(Concat([x0,x1,…,xl−1])))) (15)xl=xl−1+Conv(ReLU(BN(Concat([x0,x1,…,xl−1])))) (16)Hl([x0,x1,…,xl−1])=Conv(ReLU(BN(Concat([x0,x1,…,xl−1]))))Hl

For attention mechanism application: (17)Al=σ(Conv(Hl))

For combining attention with feature maps: (18)xl=xl−1+Al(Hl([x0,x1,…,xl−1]))where σ denotes the sigmoid function that scales the output of a convolutional layer applied to the feature maps, enabling selective emphasis on relevant features in the image.

We introduce dilated convolutions in specific layers to increase the receptive field without losing resolution or excessively increasing the number of parameters. Dilated convolutions help in expanding the kernel’s footprint, capturing a more significant context without the downsampling typically used in deeper networks: (19)Hl=Convdilated(BN(ReLU([x0,x1,…,xl−1])))where Convdilated represents a convolution operation with a dilation rate greater than 1, expanding the kernel’s reach to capture more spatial context. (20)Hl=Convdilated(BN(ReLU(Concat([x0,x1,…,xl−1]))))

Convdilated represents the convolution operation equipped with a dilation rate greater than 1. This dilation rate expands the spatial coverage of the convolution kernel, allowing it to encompass a larger area of input features without increasing the number of parameters as much as increasing the kernel size directly would.

The attention mechanism in our model is a channel-wise attention module that enhances the model’s ability to focus on relevant MRI regions associated with tumor characteristics. By dynamically adjusting the weights of feature channels, this mechanism emphasizes critical features while suppressing less relevant information, thereby improving segmentation accuracy and diagnostic precision.

Algorithm 1 demonstrates the implementation of segmenting and classifying brain tumors using a modified DenseNet architecture, which proceeds with MRI images by concatenating features, performing batch normalization, applying ReLU activation, performing convolutions, implementing skip connections, and optionally using attention and dilated convolutions for a more excellent detailed capture, followed by tumor classification optimized by the Adam optimizer.

The dilation rate effectively spaces out the kernel elements, enabling the capture of information from a wider field without a direct sampling of every single intermediate point within the receptive field. Thus, resolution is maintained, and computational load is reduced.

A hybrid pooling strategy is used in transition layers to preserve critical spatial details while reducing dimensionality. This strategy combines average pooling and max pooling, aiming to retain prominent features while lowering the output dimensions smoothly: (21)Pl=12(MaxPool(xl−1)+AvgPool(xl−1))

The following steps train our DenseNet model, each incorporating key techniques to optimize performance.

The primary loss function used is the cross-entropy loss for classification tasks, defined as: (22)L(y,y^)=−∑c=1Myo,clog(y^o,c)

yo,c is a binary indicator (0 or 1) if the class label c is the correct classification for observation o. y^o,c is the predicted probability that observation o is of class c. M is the number of classes.

We use the Adam optimizer to update the weights in the network, which computes adaptive learning rates for each parameter. The update rule for Adam is given by: (23)θt+1=θt−ηv^tm^t+ϵ

θ are the parameters of the model. η is the learning rate. v^t and m^t are the bias-corrected estimates of the first and second moments of the gradients, respectively? ϵ is a small constant added to improve numerical stability.

The DenseNet model is well-suited for the complexities of detecting and segmenting brain tumors from MRI images, leveraging the full potential of its architectural innovations.

To evaluate the effectiveness of the DenseNet model in detecting and segmenting brain tumors, we use several key performance metrics that are standard in medical imaging tasks:

Accuracy is the most straightforward and most intuitive performance measure [40]. It is calculated as the ratio of correctly predicted observations to the total observations. Sensitivity measures the proportion of actual positives that are correctly identified as such and is crucial for medical diagnostics, where missing a condition can be critical. Specificity, on the other hand, measures the proportion of actual negatives that are correctly identified. Precision is the ratio of correctly predicted positive observations to the total predicted positives. The F1 Score is the weighted average of Precision and Recall. Therefore, this score takes both false positives and false negatives into account. It is beneficial when the class distribution is uneven. The Area Under the Receiver Operating Characteristic Curve (AUC-ROC) is a graphical plot that illustrates the diagnostic ability of a binary classifier system as its discrimination threshold is varied. The AUC represents the degree or measure of separability.

To make the model’s decisions understandable, especially in a clinical setting where understanding the reasoning behind a diagnosis is as important as the diagnosis itself, we employ Gradient-weighted Class Activation Mapping (Grad-CAM) and Layer-wise Relevance Propagation (LRP) techniques:

Grad-CAM uses the gradients of any target concept (like a decision output) flowing into the final convolutional layer to produce a coarse localization map, highlighting essential regions in the image for predicting the concept:(24)LcGrad−CAM=ReLU(∑αkcAk)where αkc is the gradient importance weight for the class c, and Ak represents the activation maps.

LRP is used to decompose the output predictions of a network by propagating the prediction backward through the layers to assign a relevance score to each input pixel:(25)Ri=∑jaiwij∑iaiwijRjwhere ai are the activations of a layer i, wij are the weights between layers i and j, and Rj is the relevance of neuron j in the subsequent layer. These metrics and techniques provide a robust framework for evaluating and enhancing our DenseNet model’s performance.

This research tested the modified DenseNet algorithm’s performance in detecting brain tumors from MRI images with a well-scripted algorithm and computational tools. This section describes the hardware and software platforms and libraries and the overall environment utilized for the current analyses.

Applying this specific computational model offered a stable and high-performance environment where experiments can be run to assess the modified DenseNet model. Powerful GPUs enabled rapid model iterations and experiments, while an extensive software ecosystem supported all aspects of the machine learning workflow, from preliminary data processing to model building to the results analysis and visualization.

To visualize the training progress and the effects of our adjustments, we summarized them in tables and employed several graphical representations. Initial Model Configuration and Adjustments are outlined in Table 2.

Training adjustment and outcomes are summarized in Table 3.

Fig. 2 illustrates the decline in loss values over time, marked at points where adjustments were made, showing their immediate and long-term effects on model performance.

Training and validation loss over epochs showed a decreasing trend, with noted improvements following each adjustment. Fig. 3 charts the trajectory of accuracy improvements, linking these directly to the specific adjustments in data augmentation, dense block configuration, and attention mechanisms. It demonstrated accuracy improvements, correlating these gains with particular changes in model configuration and training strategy.

The detailed setup and continuous adjustments allowed for fine-tuning the DenseNet model’s capabilities in detecting brain tumors from MRI images. It ensured robustness and accuracy, making it highly effective for clinical applications. These tables and figures provide a clear, structured view of how systematic changes to the training approach directly contributed to performance enhancements.

The results of the model’s performance across different thresholds and conditions are summarized in Table 4.

These metrics indicate a high level of performance, suggesting that the model effectively identifies brain tumors from MRI scans. To further contextualize the proposed model’s performance, we compared it against standard models used in medical imaging, such as the original DenseNet without modifications and a basic CNN model, as shown in Table 5.

The comparison highlights the enhancements achieved with the modified DenseNet, showing marked improvements in all key metrics. The model’s diagnostic capabilities are illustrated visually in Fig. 4. The proposed DenseNet variant demonstrates consistently higher performance compared to the baseline CNN and standard DenseNet models. The accuracy and AUC-ROC curves indicate faster convergence and reduced performance fluctuations during training, suggesting improved stability of the learning process. In particular, the integration of dilated convolutions enables the network to capture multi-scale contextual features, while the channel-attention mechanism enhances feature selection, both of which contribute to the observed performance gains.

This curve helps visualize the trade-offs between precision and recall for different threshold settings, underlining the robustness of our model in handling various operational scenarios. As shown in Fig. 5 for the model, a confusion matrix gives the real positive and negative, the false positive and negative, which provides an insight into the model working in distinguishing between tumor and non-tumor instances. Some tables give detailed visuals of how the metric of interest, or the threshold, impacted the model’s performance, complementing the other tables presented. This comprehensive evaluation demonstrates that the proposed method of modifying the DenseNet model results in improved performance using standard measures of different machine learning models and still gives significant potential for use in clinical practitioner applications of brain tumor detection.

Gradient-weighted Class Activation Mapping (Grad-CAM) is used to improve the clarity and reliability of our modified DenseNet model in detecting brain tumors from MRI images. Grad-CAM provides insight into the areas of the image the model should focus on when making predictions and helps define the model’s decision-making process. Grad-CAM creates a coarse localization map by flowing the gradient of each target concept to the final CNN enhancement layer and highlights the required locations in the image to predict the idea. This method is particularly useful in medical imaging to emphasize that model decisions depend on relevant factors in images.

Grad-CAM was applied to the last convolutional layers of the modified DenseNet to visualize the areas of highest activation corresponding to the detection of brain tumors. This helps verify whether the model focuses on the tumor regions rather than non-relevant areas, which is crucial for clinical acceptance. The implementation of Grad-CAM provided clear visual evidence that our model was concentrating on the correct regions in the MRI scans when identifying and classifying tumors. Table 6 indicates that the model’s focus areas aligned with the tumor locations for various test images, as confirmed by clinicians, validating the model’s interpretative ability.

To visually demonstrate how the model processes and interprets MRI images, we present several Grad-CAM heatmaps, as shown in Fig. 6. A heatmap overlay on an MRI image, where the red-to-yellow gradient indicates areas of the highest importance, confirms that the tumor region is correctly identified. Fig. 7 illustrates the heatmaps generated by the basic CNN, the original DenseNet, and our modified DenseNet, highlighting the improved focus and clarity of our model. Fig. 8 shows a more detailed heatmap for MRI images, where tumors are not easily defined or are located in abnormal areas, illustrating how the model performs under stress.

Fig. 9 illustrates the working of the model to give focus on the tumor region, justifying its diagnostic ability. These Grad-CAM heat map zoning convincingly show that the model draws its predictions from relevant parts of the pattern suitable for diagnosis. It presents a simplified and comprehensible format through which the model’s actions can be easily explained to medical practitioners, thereby motivating clinical integration. Table 7 presents the results of an ablation study analyzing the impact of different architectural modifications on the DenseNet model’s performance.

The baseline DenseNet, without any modifications, achieved an accuracy of 89.0% and an AUC-ROC of 0.94. Introducing dilated convolutions improved the accuracy to 91.2% and the AUC-ROC to 0.96, showcasing better spatial detail capture. Adding attention mechanisms enhanced accuracy to 92.4% and AUC-ROC to 0.97 by improving focus on critical tumor regions. The complete model, integrating dilated convolutions and attention mechanisms, achieved the highest performance, with an accuracy of 94.0% and an AUC-ROC of 0.98, demonstrating the synergistic effect of these enhancements.

The results were subjected to benchmarking tests against several other state-of-the-art deep learning models to evaluate the proposed modified DenseNet architecture for detecting brain tumors in MR images. This section compares performance variables and variables within the framework, presenting interpretive visuals. All models were evaluated on the following five key performance measures: accuracy, precision, recall, F1 score, and AUC-ROC. Table 8 also reaffirms that our proposed DenseNet, with the applied modifications, is undoubtedly superior to the basic CNN and the original DenseNet in all measured performance metrics, demonstrating its superiority for this particular task.

Table 8 compares the performance of our modified DenseNet model in MRI-based brain tumor detection with that of recent state-of-the-art approaches. While techniques like the EfficientNet and Swin Transformer models achieve high accuracy and AUC, our modified DenseNet demonstrates competitive results, achieving 94.0% accuracy and a high AUC of 0.980. These results highlight the model’s effectiveness, particularly in striking a balance between precision and recall, demonstrating its robustness as a reliable tool for tumor identification in MRI scans. The comparisons shown here demonstrate that the changes made to the DenseNet structure significantly improve its performance in MRI images of brain tumors. The synthesized visualization and tabulated comparison support quantitative and qualitative arguments on the effectiveness of our proposed model over regular CNN and DenseNet architectures.

In the literature, we found that a lightweight deep learning based brain tumor model was proposed in [51] and a multilevel brain tumor segmentation model was presented in [52]. In the future, we will enhance our work by comparing it with this research.