Work
AI evaluation.
At the BBC I lead technical work on evaluating AI systems for accuracy, editorial risk and safety. The results inform decisions about whether, and how, tools are used.
The work
I design evaluation suites and harnesses for tools built on large language models. Most combine expert human review with automated checks, and part of the job is testing whether automated evaluators agree with human reviewers. More of the work is now multilingual.
The work runs in two directions. I turn BBC editorial standards into criteria a system can be tested against, and I turn technical evidence into guidance that editors and senior leaders can act on.
AI and society
Evaluation is also how I approach AI's effects on society. The News Integrity study started from a public question: what happens to trust in news when people get it through AI assistants. I've also spoken about what AI means for skills and work, and about what it takes for an organisation to use AI responsibly.
News integrity
News Integrity in AI Assistants
News Integrity in AI Assistants Toolkit
Lead author, with James Fletcher and Dorien Verckist (BBC and EBU). It sets out what a good AI news answer needs, covering accuracy, sourcing, quotes, fact and opinion, and context, with a taxonomy of the ways AI assistants get news wrong.
News Integrity in AI Assistants
Contributor. Journalists from 22 public-service media organisations in 18 countries assessed AI assistants' answers to news questions in 14 languages.
Towards an AI-in-News Evaluation Benchmark
Co-presenter, EBU Data Technology Seminar. I now lead joint BBC and EBU work to turn the Toolkit into a repeatable evaluation capability.
Research
Research
Towards Neuro-Symbolic AI: Knowledge Graph-Enhanced LLM Summarisation
MSc dissertation, University of Reading (Distinction). Compared standard summarisation models with variants that draw on knowledge graphs, using established metrics and a new measure of consistency with the graph.
Sparse vector compression
Doctoral research in computer science at the University of Nottingham, on data compression for industrial imaging and monitoring tools.
Policy and partnerships
Policy and partnerships
Small Islands, Genuine Partnerships
Lead author. Official UN guidance on durable partnerships in Small Island Developing States, published by UN DESA and The Partnering Initiative as an annex to the SDG Partnership Guidebook.
The Partnering Initiative
Senior Research Associate. Research on multi-stakeholder partnerships for sustainable development.
Technical and research consultant
Sustainable development projects for UN agencies, the World Health Organization, the European Commission, the UK Foreign Office and DFID.