Data & validation

How accurate is AI scoring? How much do learners improve?

The figures below show how Youshow IELTS AI scores compare with reference scores and how learners' results change with sustained practice.

REAL DATA

This page presents AI score comparisons and learner assessment results before and after practice. The notes below explain the measures and how to interpret them.

AI SCORE VALIDATION

1. How close are AI scores to human references?

We compared AI scores with reference scores for 120 anonymised practice responses, independently reviewed by two raters.

120 score comparisons

Exact agreement with reference64%
Within 0.5 band91%
More than 0.5 band apart9%

Exact agreement

64%

Share of responses where the AI and reference scores match exactly.

Within 0.5 band

91%

Share of responses no more than 0.5 band from the reference score.

Mean absolute error

0.34

The mean absolute difference between AI and reference scores is 0.34 band.

Exact agreement and within-0.5-band agreement are different measures. Interpret the results alongside task types, rater qualifications, disagreement handling and sample scope.

LEARNER PROGRESS

2. How much progress follows sustained practice?

We tracked speaking and writing assessment results for 200 learners over eight weeks, comparing results at the start of practice and at week eight.

Baseline median

5.5

Learners' median assessment score at the start of practice.

Week 8 median

6.0

The same learners' median assessment score at week eight.

Median change

+0.5

Median change in paired results. A score change reflects assessment results and does not identify its cause on its own.

136 of the 200 learners scored above their practice baseline. Interpret learning progress alongside practice volume, assessment conditions and participation over the full period.

TRANSPARENT METHOD

How to read the data

Understanding how the score comparisons and progress measures are calculated helps put these results in context.

01

Reference scores

Two raters independently reviewed the responses to establish reference scores for comparison with AI scores.

02

Score agreement

Exact agreement, within-0.5-band agreement and mean absolute error describe score differences from different perspectives.

03

Before-and-after comparison

Progress data compares the same learners' assessment results at the start of practice and at week eight.

04

Consider practice context

Score changes reflect assessment results; practice volume, assessment conditions and other factors can also affect them.

Practice and feedback

Keep a useful record of your practice

Youshow IELTS offers AI-assisted speaking and writing practice feedback. Feedback can guide what to work on next; it is not an official IELTS score or a guarantee of results.

Explore practice tools