The situation
A startup building AI software for newsrooms needed to ship LLM features quickly, without betting everything on a single model provider. New models appear every few months, and each one is better at some tasks and worse at others. The team needed a way to choose with evidence, not hype.
What I did
- Built a benchmarking framework. It evaluates model quality across text, audio, video and image, and guides which model to use for which job.
- Designed a provider-agnostic LLM layer. The team can switch between models and compare them without changing application code.
- Built a transcription tool. Using AWS Transcribe, it turns audio recordings into transcripts journalists use to write their articles.
- Worked across the full stack, including AWS cloud infrastructure, and helped onboard new client organisations onto the product.
What changed
Model choices are now based on measured results instead of guesswork, and switching providers is a configuration change rather than a rewrite. Journalists get transcripts they can work from, and new organisations can get started on the platform faster.
Tech used
Python, several LLM provider APIs, AWS Transcribe and other AWS services, and full-stack web development.