MS thesis — Vision-language AI to assist visually impaired people
Research on BEiT-3, a 746-million-parameter vision-language model, answering 16 structured questions about an image and turning the answers into scene descriptions for visually impaired people.
- Years
- 2023–2025
- Organisation
- NED University of Engineering and Technology
- Role
- MS Artificial Intelligence (research-based)
- Status
- Research
- Input
- Processing step
- Agent / AI step
- Output
Context
My research-based MS in Artificial Intelligence at NED University of Engineering and Technology asked a practical question: can a vision-language model describe a scene well enough to help someone who cannot see it? The work is unpublished.
What I did
I evaluated BEiT-3, a vision-language model with 746 million parameters, for Visual Question Answering (VQA) as assistive technology for visually impaired people.
How it works
Rather than asking for a free-form caption, I designed 16 generic questions that pull out the key visual and contextual information in an image. Using images from the MS COCO dataset, BEiT-3 answers each question. Custom logic then combines the answers into a descriptive scene narrative.
Those narratives were compared with descriptions of the same images from Meta AI and ChatGPT.
Findings
BEiT-3 reached high VQA accuracy and produced coherent, detailed scene descriptions. Future work points to stronger object detection and better description generation. The thread running through it is AI for accessibility and social impact.
Stack
BEiT-3 for VQA, evaluated on MS COCO.