As automation and AI re-organise social, political, and economic life, law’s reflex is often towards the human. By ensuring automation is “human-centred” and subject to “human oversight”, the hope is that automation’s economic promises can be safely realised. Despite the intuitive appeal, analysts have started documenting the empirical failure of human oversight to improve decision quality. But this creates a conundrum. There may be little scientific evidence supporting legal human oversight requirements, but abdicating human agency in automated decision processes is ethically and politically unfeasible.

To transcend this impasse, legal scholars have started reformulating human oversight requirements in an effort to improve empirical outcomes. This paper intervenes by suggesting that human oversight’s regulatory value is determined more by its political utility than its empirical impact on decision quality. Instead of concocting new guiding principles or accounts of human capacity that would inform better oversight laws, this paper shows how laws requiring human oversight emerge in service of specific political and economic projects. These are grouped into four domains: affording accountability for automated decisions affecting legal rights; managing the distribution of liability between automated systems and their operators; satisfying product safety requirements for AI software markets; and attracting legal rights and privileges for sociotechnical assemblages. Rather than something that can be gotten right, these examples show that laws requiring human oversight are highly versatile, conditioned by political and economic context, and constantly reconfiguring the human at the centre to facilitate novel technological developments.