Date of Submission

6-11-2026

Date of Award

6-23-2026

Institute Name (Publisher)

Indian Statistical Institute

Document Type

Master's Dissertation

Degree Name

Master of Technology

Subject Name

Computer Science

Department

Computer Vision and Pattern Recognition Unit (CVPR-Kolkata)

Supervisor

Bhattacharya, Ujjwal

Abstract (Summary of the Work)

Large Language Models have shown strong performance across reasoning, language understanding, and generation tasks, but their computational and memory requirements make deployment on low resource devices difficult. This dissertation studies efficient performance recovery of pruned LLMs, focusing on whether a pruned model can regain useful task performance through parameter-efficient fine-tuning while preserving the benefits of compression. The work uses Llama-3.2-3B-Instruct as the dense reference model and applies activationaware Wanda pruning at multiple sparsity levels(20%, 30% and 50%). The pruned models are evaluated on language modeling and reasoning tasks using WikiText-2 perplexity and GSM8K (Grade School Math 8000 problems) accuracy. Adaptation is performed using efficient parameter adaptation methods like QLoRA and DoRA. Additionally, the thesis evaluates the balance between sparsity, precision, perplexity, speed, memory usage, and device compatibility. Based on the results obtained from the experiments conducted, we can safely say that the adapter recovery technique improves the performance of sparse models. DoRA performs better in GSM8K recovery compared to QLoRA recovery at sparsity of 20%, 30%, and 50%. Additionally, the WikiText-2 perplexity for QLoRA is slightly lower than DoRA at 30% and 50% sparsity. The largest recovery gain occurs at 50% sparsity, where GSM8K accuracy improves from 0.1800 to 0.4508 using DoRA recovery. However, the best final recovered GSM8K accuracy is obtained at 20% sparsity with DoRA.

Control Number

CS2420

DOI

https://dspace.isical.ac.in/items/522dc9ec-673c-41ea-a249-6961037e22c1

DSpace Identifier

http://hdl.handle.net/10263/7767

Download

Share

COinS