Skip to content

AutoSegmentor

Turn raw video into a YOLO-ready dataset — with a handful of mouse clicks, not frame-by-frame labeling.

AutoSegmentor bridges raw footage and structured training data. Point at an object once — Meta AI's Segment Anything Model 2 (SAM2) propagates a pixel-accurate mask across every following frame, while CoTracker3 follows its keypoints for pose estimation — and export a complete dataset (detection, instance segmentation, and pose, all at once) in one keystroke.

Why it exists

Hand-labeling video for computer vision is slow: thousands of frames, each needing a mask and a set of keypoints. AutoSegmentor collapses that into a few clicks on a single frame, then lets state-of-the-art tracking models carry the annotation forward — a human stays in the loop to correct drift, not to redraw every frame from scratch.

What you get

  • An interactive PyQt5 annotation tool — point-and-click prompts, live mask preview, undo/redo, per-object keyframe correction.
  • Automatic propagation across long videos in GPU-friendly batches, so hour-long footage runs on consumer hardware.
  • One export, three label types — YOLO-format detection (bbox), instance segmentation, and pose, generated together from the same verified annotations.
  • Two bundled demos (a classic object clip and a dashcam road scene) that run the whole pipeline end to end with zero configuration.

Get started

git clone --recursive https://github.com/thippeswammy/AutoSegmentor.git
cd AutoSegmentor
python -m venv .venv && source .venv/bin/activate   # or .venv\Scripts\Activate.ps1 on Windows
python install.py
python run_main.py --demo cat

Head to Installation for the full walkthrough (Windows and Ubuntu), or straight to Demos to see what running it actually looks like.

Explore the docs

  • Installation — set up on Windows or Ubuntu, one command or manual.
  • Demos — watch the full pipeline run, then try it on your own footage.
  • Architecture — how the UI, SAM2, CoTracker3, and export actually connect.
  • Dataset Manager — turning verified annotations into a training-ready YOLO dataset, plus large-scale synthetic augmentation.

Source on GitHub.