The Problem
A client was producing video content at volume. Their process: transcribe audio manually, identify key clips, cut footage, add captions. Each video took hours. The work was purely mechanical — no creative judgment involved — which made it an obvious automation target.
What I Built
A fully automated Python pipeline that takes raw audio as input and outputs finished, captioned video with no human intervention.
Transcription with Whisper: Used OpenAI Whisper to transcribe audio with word-level timestamps. This is the foundation of the entire pipeline — accurate timestamps let every downstream step work with precision.
Intelligent segmentation: Wrote a segmentation algorithm that groups word-level transcription data into sentence-level chunks, respecting natural speech boundaries rather than cutting mid-thought. Factored in pause detection and sentence structure.
Automatic clip selection: Given the segment timings, the pipeline automatically calculates cut points in the source video, adding configurable padding to avoid hard cuts that feel jarring.
Video processing: Used FFmpeg (via Python subprocess) to cut and concatenate video segments with frame-accurate precision. Handled codec compatibility, bitrate normalisation, and output format requirements.
Caption overlay: Generated subtitle files from the transcription data and burned them into the output video with controllable styling. The captions are perfectly synced because they derive from the same word-timestamp data the segmentation uses — there’s no drift.
End to end: Input a file path, get a finished video. No manual steps in between.
Why It Matters
This is what practical AI automation actually looks like: not a demo, not a prototype, but a tool that runs in production and eliminates real, recurring labour. The creativity was in the engineering — the design of the segmentation logic, the reliability of the file handling, the consistency of the output format — not in the choice to use Whisper.
The client estimated it cut their per-video production time from several hours to under five minutes.
Tech Stack
Python · OpenAI Whisper API · Voice AI · FFmpeg · Video/Audio Processing · Automation