Inference & Predictions

The predict.py script uses your trained model to predict folder assignments for unsorted videos, with configurable confidence thresholds and automatic file organization.

How Predictions Work

The inference pipeline:

Load trained model from artifacts/model.pt (or .pkl)
Load unlabeled embeddings from artifacts/unlabeled_embeddings.pt
Generate predictions with confidence scores (probability distribution)
Display top-k predictions per video
Optionally move files to predicted folders
Save detailed predictions to predictions.json

Running Predictions

Ensure Prerequisites

You need:

artifacts/model.pt and artifacts/model_config.json from training
artifacts/unlabeled_embeddings.pt from feature extraction

ls artifacts/
# Should show: model.pt, model_config.json, unlabeled_embeddings.pt

Preview Predictions (Dry Run)

Run without --move to see predictions without modifying files:

python predict.py

Output:

Predicting folders for 42 unsorted videos
Model: mlp | Categories: ['cooking', 'funny', 'motivational', 'pets', 'quran', 'soccer', 'tiktok', 'travel']
Confidence threshold: 0.0

  [ASSIGN] 7234567890123456.mp4                   → soccer       (94%)  [soccer: 94% | tiktok: 4% | travel: 2%]
  [ASSIGN] 7234567890123457.mp4                   → cooking      (87%)  [cooking: 87% | funny: 8% | pets: 5%]
  [ASSIGN] 7234567890123458.mp4                   → quran        (99%)  [quran: 99% | motivational: 1% | travel: 0%]
  [ASSIGN] 7234567890123459.mp4                   → funny        (62%)  [funny: 62% | tiktok: 31% | cooking: 7%]
  ...

============================================================
Summary:
  cooking        :    8 videos
  funny          :    3 videos
  motivational   :    5 videos
  pets           :    6 videos
  quran          :   10 videos
  soccer         :    7 videos
  tiktok         :    2 videos
  travel         :    1 videos
  SKIPPED        :    0 videos (below 0% threshold)
  TOTAL          :   42 videos

Full predictions saved to artifacts/predictions.json

Move Files to Predicted Folders

Add --move to actually organize files:

python predict.py --move

The script will:

Move each video from data/Favorites/videos/ to data/Favorites/videos/[predicted_folder]/
Skip files that already exist in destination
Update folder counts in summary

Moving 42 files...
  Moved 7234567890123456.mp4 → soccer/
  Moved 7234567890123457.mp4 → cooking/
  ...
Done moving files!

Command-Line Arguments

—move

Actually move files to predicted folders (default: false).

python predict.py --move

Files are moved (not copied). Once moved, they’re in category folders and considered “labeled” on next feature extraction.

—threshold

Minimum confidence (0.0 to 1.0) required to auto-assign a folder.

python predict.py --threshold 0.7 --move

Example output with threshold=0.7:

  [ASSIGN] video1.mp4 → soccer  (94%)  # Moved (above threshold)
  [ASSIGN] video2.mp4 → cooking (87%)  # Moved
  [SKIP  ] video3.mp4 → funny   (62%)  # Not moved (below 70%)
  [SKIP  ] video4.mp4 → tiktok  (58%)  # Not moved

Summary:
  ...
  SKIPPED        :    2 videos (below 70% threshold)

Use cases:

--threshold 0.5: Only auto-sort videos the model is reasonably confident about
--threshold 0.8: Very conservative, only high-confidence predictions
--threshold 0.0: Sort everything (default)

—top-k

Number of predictions to show per video (default: 3).

python predict.py --top-k 5

Output:

  [ASSIGN] video.mp4 → soccer (94%)  [soccer: 94% | tiktok: 4% | travel: 2% | cooking: 0% | pets: 0%]

Useful for understanding model uncertainty and alternative predictions.

Understanding Confidence Scores

Confidence scores are softmax probabilities (sum to 1.0 across all classes):

# MLP example
logits = model(features)  # Raw outputs
probs = F.softmax(logits, dim=1)  # Convert to probabilities

Interpreting Confidence

Confidence	Interpretation	Action
90-100%	Very confident	Trust the prediction
70-90%	Confident	Usually correct, verify if important
50-70%	Uncertain	Review manually, might be ambiguous
<50%	Very uncertain	Definitely review, likely wrong or ambiguous

High Confidence (>90%)

Example:

[ASSIGN] quran_video.mp4 → quran (99%)  [quran: 99% | motivational: 1%]

Interpretation: Model is nearly certain. Categories with strong audiovisual signatures (e.g., Quran recitation with Arabic speech + distinct visual framing) typically get 95%+ confidence.

Moderate Confidence (50-70%)

Example:

[ASSIGN] ambiguous.mp4 → funny (62%)  [funny: 62% | tiktok: 31% | cooking: 7%]

Interpretation: Model leans toward “funny” but sees similarity to “tiktok”. This video might:

Be a funny TikTok (overlap between categories)
Have features of both categories
Be mislabeled or genuinely ambiguous

Action: Review manually or use a higher --threshold to skip auto-sorting.

Confused Predictions (Close Split)

Example:

[ASSIGN] confused.mp4 → cooking (42%)  [cooking: 42% | travel: 40% | tiktok: 18%]

Interpretation: Model can’t decide between cooking and travel. Possible reasons:

Video shows cooking while traveling
Weak category definition
Insufficient training data for this edge case

Action: This is a good candidate for manual labeling to teach the model.

Predictions Output File

All predictions are saved to artifacts/predictions.json for review:

[
  {
    "video": "7234567890123456.mp4",
    "predicted_folder": "soccer",
    "confidence": 0.9423,
    "top_predictions": [
      {"folder": "soccer", "confidence": 0.9423},
      {"folder": "tiktok", "confidence": 0.0387},
      {"folder": "travel", "confidence": 0.0190}
    ]
  },
  {
    "video": "7234567890123457.mp4",
    "predicted_folder": "cooking",
    "confidence": 0.8721,
    "top_predictions": [
      {"folder": "cooking", "confidence": 0.8721},
      {"folder": "funny", "confidence": 0.0812},
      {"folder": "pets", "confidence": 0.0467}
    ]
  }
]

Use cases:

Audit predictions before moving files
Find low-confidence videos for manual review
Analyze which categories the model confuses

Finding Low-Confidence Predictions

# Videos with <70% confidence
cat artifacts/predictions.json | jq '.[] | select(.confidence < 0.7) | {video, predicted_folder, confidence}'

Output:

{
  "video": "7234567890123459.mp4",
  "predicted_folder": "funny",
  "confidence": 0.62
}

Finding Confused Categories

# Videos where top-2 predictions are close (<20% gap)
cat artifacts/predictions.json | jq '.[] | select(.top_predictions[0].confidence - .top_predictions[1].confidence < 0.2)'

Confidence Threshold Strategy

Conservative Strategy (High Precision)

python predict.py --threshold 0.8 --move

Goal: Only auto-sort videos you’re very confident about. Result:

High precision (few errors)
Many videos skipped (low recall)
Manual labeling required for uncertain cases

Best for: Initial sorting to quickly handle obvious cases.

Balanced Strategy

python predict.py --threshold 0.6 --move

Goal: Sort most videos while avoiding obvious errors. Result:

Good precision (~85-90%)
High recall (~70-80% sorted)
Occasional errors on ambiguous videos

Best for: General use after validating model performance.

Aggressive Strategy (High Recall)

python predict.py --threshold 0.0 --move

Goal: Sort everything, accept some errors. Result:

Lower precision (~80-90%)
100% recall (all videos sorted)
Requires post-sorting review

Best for: Bulk organization where occasional errors are acceptable.

Active Learning Workflow

Initial sorting with high threshold:

python predict.py --threshold 0.8 --move

Review skipped videos (low confidence):

ls data/Favorites/videos/*.mp4  # Remaining unsorted videos

Manually label uncertain videos via labeling interface

Re-extract and retrain with new labels:

python extract_features.py
python train.py
python predict.py --threshold 0.8 --move  # New predictions

Repeat until all videos are sorted or accuracy plateaus

Each iteration improves the model by teaching it edge cases. After 2-3 cycles, you typically reach 90%+ accuracy.

Troubleshooting

FileNotFoundError: artifacts/unlabeled_embeddings.pt

No unlabeled videos found during feature extraction. This happens when all videos are already in category folders.Solution: Move some videos to data/Favorites/videos/ root:

mv data/Favorites/videos/soccer/video.mp4 data/Favorites/videos/
python extract_features.py  # Re-extract to create unlabeled embeddings
python predict.py

All predictions go to one category

Model is biased toward the majority class. Possible causes:

Severe class imbalance: One category has 90%+ of data
Weak features: Categories aren’t visually/audibly distinct
Training issue: Class weighting didn’t work

Solutions:

Balance your dataset (add more examples to minority classes)
Verify categories have distinct content
Check training metrics for signs of mode collapse

Video already exists in target folder

The video was already sorted (manually or by a previous prediction run).

Already exists: soccer/7234567890123456.mp4

Not an error - the script skips to avoid overwriting. If you want to re-sort:

mv data/Favorites/videos/soccer/7234567890123456.mp4 data/Favorites/videos/
python predict.py --move

Predictions seem random (all ~same confidence)

Model hasn’t learned meaningful patterns. Check:

Training accuracy: Was it >60%? If not, model didn’t learn
Data quality: Are videos correctly labeled?
Feature extraction: Did it complete successfully?

Re-check the full pipeline:

python extract_features.py  # Verify no errors
python train.py             # Check CV accuracy > 70%
python predict.py

Batch Processing Large Collections

For 500+ unlabeled videos, process in batches to review incrementally:

# Sort high-confidence videos first
python predict.py --threshold 0.9 --move

# Review predictions for next tier
python predict.py --threshold 0.7 > predictions_0.7.txt
# Manually review predictions_0.7.txt

# Sort if satisfied
python predict.py --threshold 0.7 --move

# Repeat with lower thresholds
python predict.py --threshold 0.5 --move

This gradual approach prevents bulk mislabeling.

Next Steps

Labeling Interface

Use the interactive web UI to review predictions and manually label videos

Get Started

Core Concepts

Guides

Advanced

How Predictions Work

Running Predictions

Command-Line Arguments

—move

—threshold

—top-k

Understanding Confidence Scores

Interpreting Confidence

High Confidence (>90%)

Moderate Confidence (50-70%)

Confused Predictions (Close Split)

Predictions Output File

Finding Low-Confidence Predictions

Finding Confused Categories

Confidence Threshold Strategy

Conservative Strategy (High Precision)

Balanced Strategy

Aggressive Strategy (High Recall)

Active Learning Workflow

Troubleshooting

Batch Processing Large Collections

Next Steps

Labeling Interface

Build docs developers (and LLMs) love

Get Started

Core Concepts

Guides

Advanced

​How Predictions Work

​Running Predictions

​Command-Line Arguments

​—move

​—threshold

​—top-k

​Understanding Confidence Scores

​Interpreting Confidence

​High Confidence (>90%)

​Moderate Confidence (50-70%)

​Confused Predictions (Close Split)

​Predictions Output File

​Finding Low-Confidence Predictions

​Finding Confused Categories

​Confidence Threshold Strategy

​Conservative Strategy (High Precision)

​Balanced Strategy

​Aggressive Strategy (High Recall)

​Active Learning Workflow

​Troubleshooting

​Batch Processing Large Collections

​Next Steps

Labeling Interface

Build docs developers (and LLMs) love

How Predictions Work

Running Predictions

Command-Line Arguments

—move

—threshold

—top-k

Understanding Confidence Scores

Interpreting Confidence

High Confidence (>90%)

Moderate Confidence (50-70%)

Confused Predictions (Close Split)

Predictions Output File

Finding Low-Confidence Predictions

Finding Confused Categories

Confidence Threshold Strategy

Conservative Strategy (High Precision)

Balanced Strategy

Aggressive Strategy (High Recall)

Active Learning Workflow

Troubleshooting

Batch Processing Large Collections

Next Steps