Automatic clip editing
A pipeline that cuts long videos into vertical clips, framed on the speaker
Task
The pipeline takes an episode of a show, finds self-contained stories in it and builds a vertical clip from each: framing on the speaker, subtitles, a hook caption, a music second part. The finished clips scroll on a phone like a reels feed: a like sends a clip to upload, a reject sends it to the bin.
How it works
- Download and speech recognition
- Scenes and faces: TransNetV2, SCRFD
- Story search by a model
- Virtual camera: frame path optimisation
- ffmpeg render on VideoToolbox
- Feed on the phone → upload
What had to be solved
The feed took minutes to load — and the internet was not the cause
The connection gave 140 Mbit/s, yet the clips crawled. A measurement showed that the tunnel between the phone and the Mac had never once set up a direct connection and was routing everything through a fallback server in Singapore. Along the way a hole turned up: the tunnel as it stood would have opened clip publishing to the whole internet. The hole was closed first, then the feed was rebuilt: instead of 273 requests and 10.7 MB of posters before the first frame — one screen.
Commits
- 65b11fc01.09.2026the phone reached the Mac via Singapore, and the feed opened with 273 requests
- c46219c30.08.2026the downloader stopped lying “Conversion failed” and hanging at 1%
- 8051b0327.08.2026cancelling a run shuts down the cloud kernel instead of leaving it computing
- ae282f828.08.2026stopping the cutting made an episode cost more than a run with no cap
- a5cd63e27.08.2026an abandoned row held a kernel slot for eleven hours after the kernel died