MCQ Analysis Tool

Problem

Medical schools in the UK had to adapt to a new system of a national medical school examination. Support was needed to assist students in improving attainment in these exams.

Solution

This tool analyses a student's responses to a multiple choice, medical finals mock exam. The exam questions were initially analysed using an LLM to produce a database of the topics and subjects linked to each question.

The student responses are processed by the application and scored against the answer schema. Incorrect responses are identified. Details of any subject areas answered incorrectly are then collated. This data can then be processed by the LLM to produce a report alongside a personalised, structured revision plan for the student to ensure gaps in knowledge are improved.

Overview

Key aspects of this project were:

• Required building an 'ingestion' pipeline for the initial analysis and data extraction for the exam papers and mark scheme
• I chose 'traditional' python coding methods for the exam marking process instead of an LLM to avoid hallucinations and unnecessary LLM api calls
• The output summaries can be easily customised using structured prompt templates