AI-powered research assistant

Problem

Documentary research is slow by nature. A new subject means days of reading before anyone knows whether there is a story in it, and that time caps how many subjects ever get looked at. The obvious question was whether an LLM could shorten it. The better one was whether it could widen it, surfacing angles a researcher would not have thought to go looking for. The target was a researcher with a wider net, not a faster one.

Constraints

This was 2024. We ran on GPT-4 with function calling for structured output, but it still muddied answers and dropped structure often enough to need retries, and an internal creative team has a high bar for what it will trust. Getting responses reliable and structured enough to depend on was the hard part.

Approach

Input a subject name, get back biographical leads, story angles, and threads to pull. Built integration tests that run live OpenAI calls with graded responses, keeping output inside tolerance bands. That harness came before checking LLMs in CI was common practice.

Contribution

The CTO drove the project and we paired on architecture and prompt strategy. The build was mine. I owned the frontend, the data ingestion, the API connections, and the prompting layer, which refused to sit still. It moved from prompt engineering to function calling, with the model pulling the specific facts we needed out of what the pipeline fetched. Every research run went out and got fresh data.

Stabilising it is where the engineering was. The graded-LLM test harness came out of that, and canned data feeds alongside it. A prompt change can only be judged against a fixed input, or you cannot tell whether the output moved because you improved something or because the web did.

Outcome

A capable, working tool that went unused. The research team preferred its traditional workflow and the head of research objected to "AI slop". Underneath the objection sat a positioning problem. A studio built on high-end, long-form documentary had no appetite for stories mined and told at volume, so the thing the tool was good at was the thing they did not want. The tech worked, but adoption was blocked by preference, not capability.

Reflection

The part that outlived the tool is the graded-LLM test harness, holding output to a fixed bar with integration tests when the model would not hold it on its own. That discipline is the transferable engineering, and it carried into later work. The tool worked and still died in-house, because the studio did not want what it was good at, which is a lesson in fit over capability.