Date of Submission

6-15-2026

Date of Award

6-15-2026

Institute Name (Publisher)

Indian Statistical Institute

Document Type

Master's Dissertation

Degree Name

Master of Technology

Subject Name

Computer Science

Department

Machine Intelligence Unit (MIU-Kolkata)

Supervisor

Maji, Pradipta

Abstract (Summary of the Work)

Brain tumor analysis is an important application of medical image computing, where accurate segmentation and interpretation of tumor regions can support diagnosis and treatment planning. However, existing methods often address tumor segmentation and radiology report generation as separate tasks. Moreover, one of the major challenges in report generation tasks from MRI is accurate tumor localization. Most models fail to locate the lobe and hemisphere in which the tumor is located, causing incorrect report generation. In this regard, a unified 3D vision-language framework is proposed for simultaneous brain tumor segmentation and report generation from multi-modal MRI. Given T1, T2, T1C, and FLAIR volumes, the proposed model predicts clinically meaningful tumor regions and generates a textual description of tumor location and appearance. In order to encourage consistency between the predicted segmentation and generated report, the proposed model judiciously integrates a Swin UNETR-based 3D encoder-decoder, a multi-scale lesion tokenizer, auxiliary clinical grounding heads, and an iterative crossmodal refinement module. Moreover, anatomy and laterality heads are introduced, which provide clinical hints to the LLM, allowing better tumor localization. Further, an iterative refinement is incorporated so that the generated report and segmented outputs can refine each other, finally producing better outputs. Extensive experimentation on BraTS2020 and TextBraTS data sets shows that the proposed model achieves a mean Dice of 81.60% and HD95 of 6.21. In addition, the model achieves a BERTScore-F1 of 0.9226, clinical laterality F1 of 0.8459, clinical anatomy F1 of 0.7539 and clinical pathology F1 of 0.9976. These results indicate that the proposed framework can generate clinically meaningful reports while accurately localizing tumor regions and maintaining strong alignment between segmentation and textual interpretation.

Control Number

CS2403

DOI

https://dspace.isical.ac.in/items/bb6ea269-a7d2-443d-992e-20f4d1c21070

DSpace Identifier

http://hdl.handle.net/10263/7752

Download

Share

COinS