An interesting and (to this non-statistician) reasonable-looking paper on the effects of generative AI on exam scores (as a proxy for learning). As The Economist pithily puts it:

Homework scores once predicted exam performance; now those who score highest are, perversely, more likely to do worse in exams.

The paper’s own abstract reports:

AI adoption raises homework scores by 18% and reduces completion time by 30%, but lowers monthly exam scores by 20% within six months. High-stakes entrance-exam scores fall by 18 and 24%, with the full penalty emerging only after about two years.

But all is not lost:

Those who used AI but spent as long on assignments as non-users paid little penalty.

Perhaps the crucial ingredient for exam success is simply putting in the effort to become familiar with the material.

These figures from The Economist’s article reproduce figures A11 and A5 from the paper, respectively, and capture the core results well: