Current Issue

The study utilizes the publicly available Chest X-Ray Images (Pneumonia) dataset from Kaggle, comprising 5,863 anonymized posterior-anterior CXR images categorized into two classes: 4,273 pneumonia cases and 1,583 normal cases (Fig. 1). To ensure reproducibility, the dataset is categorized into training (80%, n = 4,690), validation (10%, n = 586), and testing (10%, n = 587) subsets using a stratified split to preserve class distribution across splits and mitigate sampling bias. All images are uniformly resized to 224 × 224 pixels to align with the input dimensions of standard pre-trained CNNs and normalized to the [0, 1] range using min-max scaling to stabilize gradient updates during training. To enhance model generalizability and reduce overfitting, real-time data augmentation techniques are applied to the training set, including random rotations (±15°), horizontal flipping, and zooming using the TensorFlow-Keras ImageDataGenerator API.

Fig. 1. 
Sample CXR (normal and pneumonia) images

The ViT divides each CXR into 16 × 16-pixel patches, flattens them, and projects them into embeddings. Learnable positional encodings are added to retain spatial relationships. A transformer encoder processes these patches using multi-head self-attention (to identify global dependencies) and feed-forward neural networks. Finally, a classification head with a SoftMax layer labels the image as "normal" or "pneumonia" (Fig. 2).

Fig. 2. 
ViT Architecture for Chest X-ray Classification

The model is trained for 30 epochs with a batch size of 16. Binary cross-entropy loss is used and weighted to prioritize the minority class. The Adam optimizer is applied with a learning rate of 1e-5 and gradual decay. Gradient accumulation stabilizes training by simulating larger batches. A full summary of hyperparameter settings is provided (Table 1).

ViT is benchmarked against CNNs such as VGG16, ResNet50, and Densely connected convolutional network with 121 layers (DenseNet121). All baseline models were retrained under identical experimental conditions (epochs: 30, batch size: 16, optimizer: Adam with learning rate 1e-5, decay γ = 0.995, and weighted cross-entropy loss) for fair comparison. The results (Fig. 3) show that ViT outperforms these models in accuracy, sensitivity (recall), and specificity, proving ViTs strength in medical image analysis.

Fig. 3. 
Accuracy Comparison of Vision Transformer with Baseline Models for Pneumonia Detection

The Vision Transformer (ViT) architecture for medical image analysis follows a systematic workflow, as illustrated (Fig. 4). First, chest X-ray images are divided into patches and converted into vector embeddings. Positional encoding is applied to retain spatial relationships between patches, which are then processed by a transformer encoder using self-attention to capture global contextual dependencies. Finally, the output is fed into a classification head to distinguish between Normal and Pneumonia cases. This structured pipeline enables ViT to learn discriminative features while maintaining spatial coherence, contributing to its superior diagnostic performance

Fig. 4. 
ViT Workflow for Pneumonia Detection

• • Global Context Modelling: ViTs capture long-range dependencies in CXRs, improving detection of subtle pneumonia patterns.
• • Scalability: The model handles high-resolution images without downsampling, preserving fine details.
• • Interpretability: Attention maps provide insights into model decisions, enhancing clinician trust.
• • Robustness: Data augmentation and class imbalance handling improve generalization to diverse CXR datasets.

The ViT model displayed remarkable performance, achieving 97.61% accuracy, 95% sensitivity, and 98% specificity in diagnosing pneumonia, with an F1 score of 95.2% (balanced precision and recall). The model was trained for 30 epochs, and the accuracy and loss curves (Fig. 5 (a,b)) demonstrate stable learning without overfitting. To further validate robustness, 5-fold cross-validation on a held-out test subset (20% of the data) yielded a mean accuracy of 97.2% ± 0.3%. The evaluation process was fully automated, with the test set strictly reserved for blind evaluation. The area under the receiver operating characteristic curve (AUC-ROC) value of 0.96 (Fig. 6) confirms the model's strong ability to distinguish pneumonia from normal chest X-rays.

Fig. 5. 
(a) Loss curve demonstrating convergence (b) accuracy progression over 12 epochs

Fig. 6. 
Receiver operating characteristic curve

Statistical tests confirmed the model's reliability. The accuracy of the model is between 96.2% and 98.9%, according to the 95% confidence interval. Its near-perfect Matthews correlation coefficient of 0.9396 further illustrates how regularly it provides predictions. The model achieved a sensitivity of 98.1% and specificity of 95.7%, reflecting its robust ability to minimize both false negatives and false positives in clinical screening contexts. Additionally, precision–recall analysis yielded an F1-score of 0.974, demonstrating harmonized performance in balancing precision (97.3%) and recall (97.9%) despite the dataset's inherent class imbalance.

Using attention maps that correlate to the places that physicians inspect, the model identifies indicators of pneumonia, such as lung opacities. According to the confusion matrix (Fig. 7), the model produced only 6 false positives (thinking normal is pneumonia) and 8 false negatives (missing pneumonia) out of 586 test images, resulting in a 97.61% accuracy.

Fig. 7. 
Confusion Matrix of ViT for Pneumonia vs. Normal Classification on Chest X-ray Test Set

The ViT outperformed popular CNNs such as VGG16 (92.14% accuracy) and ResNet50 (82.37%). It outperformed the best CNN (Quaternion Residual Network, 93.75%) by 3.86% accuracy (Table 2). ViT’s strength lies in analyzing the entire image for global patterns, unlike CNNs that focus on local details.

To address the class imbalance inherent in the dataset (where pneumonia cases significantly outnumbered normal cases), a weighted cross-entropy loss function was implemented. This approach assigns higher penalty weights to misclassifications of the minority class (normal events) during training, effectively prioritizing their learning and mitigating bias toward the overrepresented pneumonia class. The weighting scheme was computed inversely proportional to class frequencies, ensuring the model did not overlook subtle patterns in underrepresented samples.

Adam optimizer was employed with an initial learning rate of 1 × 10−5 (or 1e-5) and an exponential decay schedule (γ = 0.995 per epoch). This configuration stabilized training dynamics by gradually reducing step sizes, avoiding overshooting local minima while maintaining responsiveness to critical gradients.

These methodological choices—coupled with data augmentation and gradient clipping—enhanced the model's sensitivity to latent features associated with early-stage pneumonia. As shown in Fig. 8, the refined architecture achieved superior performance in detecting mild pneumonia manifestations, particularly in cases with ambiguous radiographic signatures.

Fig. 8. 
(a) Precision curve, (b) Recall curve across classification thresholds on the held-out test set.

While ViTs demonstrate superior performance in capturing global contextual relationships within medical images, their computational demands significantly exceed those of CNNs. This disparity stems from ViTs' reliance on self-attention mechanisms, which scale quadratically with input resolution (O(n2) O(n2) complexity), compared to CNNs' spatially invariant convolutional operations (O(n) O(n) complexity). Consequently, ViTs require specialized hardware (e.g., high-memory graphics processing units) and optimized frameworks (e.g., mixed-precision training) for practical deployment—a challenge in resource-constrained clinical environments.

To ensure robustness and generalizability, rigorous validation on smaller, multi-center datasets (e.g., pediatric or geriatric cohorts) and cross-domain imaging modalities (e.g., X-rays from diverse manufacturers) is critical. Future work will incorporate full k-fold cross-validation to further mitigate sampling bias. Such evaluations would quantify the model's resilience to covariate shifts and demographic biases, addressing overfitting to narrowly curated training data.