Quick Start

Overview

This guide will walk you through processing your first document with the Meta-Data Tag Generator. You’ll learn how to:

Start the application
Configure AI settings
Process a single document
Understand the results
Try batch processing

This guide assumes you’ve already completed the installation. If not, head there first!

Prerequisites

Before you begin, make sure you have:

OpenRouter API key (Get one free)

Docker Compose installed and running

A PDF document to test with

Step 1: Start the Application

Navigate to project directory

cd /path/to/Meta-Data-Tag-Generator/source

Start all services

docker-compose up -d

This starts:

Backend API (port 8000)
Frontend (port 3001)
PostgreSQL (port 5432)
MinIO (ports 9000, 9001)
Redis (port 6379)

Verify services are running

docker-compose ps

All services should show as “healthy” or “running”.

Check API health

curl http://localhost:8000/api/health

Expected response:

{
  "status": "healthy",
  "version": "2.0.0",
  "message": "Document Meta-Tagging API is running"
}

Step 2: Access the Web Interface

Open your browser and navigate to:

http://localhost:3001

You should see the Meta-Data Tag Generator interface with two main sections:

Single Upload: Process one document at a time
Batch Processing: Process multiple documents from CSV

Step 3: Configure AI Settings

Before processing documents, you need to configure the AI tagging settings.

Locate the Configuration Panel

On the main page, find the “Configuration” section in the sidebar.

Enter your OpenRouter API Key

API Key: sk-or-v1-...

Don’t have an API key? Get one free at OpenRouter. The free tier includes limited requests - add credits for production use.

Select AI Model

Choose from available models:

Model	Speed	Cost	Best For
`openai/gpt-4o-mini`	Fast	Low	General documents
`google/gemini-flash-1.5`	Fastest	Lowest	High-volume processing
`anthropic/claude-3-haiku`	Fast	Low	Complex documents
`openai/gpt-4o`	Slower	Higher	Maximum quality

Recommended for getting started: openai/gpt-4o-mini offers the best balance of speed, cost, and quality.

Configure Processing Settings

Number of Pages to Extract: 3 (default, processes first 3 pages)
Number of Tags: 8 (default, generates 8 metadata tags)

Processing more pages increases accuracy but also increases API costs and processing time. For most documents, 3 pages is sufficient.

Step 4: Process Your First Document

Now let’s process a document! You can use either a file upload or a URL.

Upload a File
Process from URL

Select a PDF file

Click “Choose File” or drag and drop a PDF into the upload area.

Maximum file size: 50MB

Supported format: PDF only

Preview the document

Once uploaded, you’ll see a PDF preview on the right side of the screen.

Click 'Process Document'

The system will:

Detect if the document is scanned or digital
Extract text using the optimal method
Send text to AI for tag generation
Apply any exclusion filters
Return structured results

View results

Processing typically takes 2-15 seconds depending on:

Document type (digital vs scanned)
Number of pages
AI model selected

Enter a PDF URL

In the “PDF URL” field, paste a publicly accessible PDF URL:

https://example.com/document.pdf

Supported URL types:

Direct PDF URLs
CloudFront URLs
S3 public URLs
Government/institutional portals

Validate URL

The system automatically validates the URL and shows a preview if accessible.

The URL must be publicly accessible (no authentication required). For S3, ensure the bucket has public read permissions or use signed URLs.

Process the document

Click “Process Document” - the system will:

Download the PDF (60-second timeout)
Extract text
Generate tags
Return results

Step 5: Understanding the Results

After processing completes, you’ll see detailed results:

{
  "success": true,
  "document_title": "Annual Report 2023-24",
  "tags": [
    "ministry social justice empowerment",
    "scheduled castes welfare",
    "annual report 2023 24",
    "national safai karamcharis finance",
    "scholarship scheme",
    "financial assistance",
    "budget allocation",
    "performance indicators"
  ],
  "extracted_text_preview": "Ministry of Social Justice and Empowerment\nAnnual Report 2023-24...",
  "processing_time": 3.45,
  "is_scanned": false,
  "extraction_method": "pypdf2",
  "ocr_confidence": null
}

Result Fields Explained

Step 6: Using Exclusion Lists (Optional)

Exclusion lists help filter out generic terms that appear frequently across documents.

Create an exclusion list file

Create a text file with terms to exclude (one per line):

exclusion-list.txt

# Government organizations (comments start with #)
government-india
ministry-of-social-justice
government-of-india

# Generic document types
annual-report
newsletter
policy-document

# Overly generic terms
empowerment
welfare
scheme

Upload the exclusion file

In the configuration panel, click “Upload Exclusion List” and select your file.

Supported formats: .txt, .pdf

Format: One term per line or comma-separated

Process with filtering

When you process a document, the system will:

Instruct the AI to avoid excluded terms
Filter any excluded terms that slip through
Request additional tags to maintain the target count

If you request 8 tags and 2 are filtered, the system ensures you still get 8 final tags.

Step 7: Try Batch Processing

Process multiple documents at once using CSV input.

Navigate to Batch Processing

Click the “Batch Processing” tab in the navigation.

Download CSV Template

Click “Download Template” to get a sample CSV:

title,description,file_source_type,file_path,publishing_date,file_size
"Training Manual","PMSPECIAL training document",url,https://example.com/doc1.pdf,2025-01-15,1.2MB
"Annual Report 2023","Ministry annual report",url,https://example.com/doc2.pdf,2023-12-31,2.5MB

Prepare your CSV

Required columns:

title: Document title
file_source_type: url, s3, or local
file_path: URL or path to the PDF

Optional columns:

description: Document description
publishing_date: Publication date
file_size: File size

Upload and process

Upload your CSV file
Map columns if needed
Click “Start Processing”
Watch real-time progress via WebSocket updates

Batch processing includes intelligent rate limiting to avoid API throttling. Large batches are processed sequentially with exponential backoff.

Export results

Download results as CSV with all metadata:

Original document info
Generated tags
Extraction method
Processing time
Any errors

Common Issues & Solutions

API Authentication Error

Error: Invalid API key. Please check your OpenRouter API key.Solution:

Verify your API key is correct (starts with sk-or-v1-)
Check if your API key has available credits
Visit OpenRouter Keys to regenerate

Rate Limit Errors

Error: RATE_LIMITED: OpenRouter free tier limit hitSolution:

Free tier has strict rate limits
Add credits to your OpenRouter account: Billing
For batch processing, the system automatically adds delays
Reduce concurrent requests

OCR Produces Gibberish

Issue: Extracted text contains nonsensical charactersSolution:

Document may be very low quality - try rescanning at higher DPI
For complex Indian scripts, the system automatically falls back to EasyOCR
Check OCR confidence score - values below 60% trigger EasyOCR

Document Processing Timeout

Error: Processing takes too long or times outSolution:

Reduce num_pages to 1-2 for faster processing
Large scanned PDFs take longer (10-30 seconds)
EasyOCR downloads models on first use (one-time delay)
Check Docker resource limits (increase RAM to 8GB for OCR)

WebSocket Connection Failed

Issue: Real-time progress not showing in batch processingSolution:

Check if Redis is running: docker-compose ps redis
Verify WebSocket endpoint is accessible
Check browser console for connection errors
Ensure no firewall blocking WebSocket connections

Performance Tips

For Speed

Use google/gemini-flash-1.5 model
Process only first 1-2 pages
Use digital PDFs when possible
Reduce number of tags to 5

For Quality

Use openai/gpt-4o or anthropic/claude-3-opus
Process 3-5 pages
Use exclusion lists to filter noise
Request 10-12 tags for more options

For Cost

Use google/gemini-flash-1.5 (lowest cost)
Process 1-2 pages only
Batch process to amortize overhead
Use smaller num_tags values

For Accuracy

Ensure documents are high quality
For scanned docs, use 300+ DPI
Include document descriptions
Use language-specific models if available

Next Steps

API Integration

Integrate with your applications using the REST API

Features Deep Dive

Learn about advanced features like exclusion lists and multilingual support

Deployment Guide

Deploy to production on AWS, GCP, or your own infrastructure

Example: Processing a Government Document

Here’s a complete example processing an Indian government document:

curl -X POST http://localhost:8000/api/single/process \
  -H "Authorization: Bearer YOUR_JWT_TOKEN" \
  -F "pdf_url=https://socialjustice.gov.in/writereaddata/UploadFile/AnnualReport2023.pdf" \
  -F 'config={"api_key":"sk-or-v1-...","model_name":"openai/gpt-4o-mini","num_pages":3,"num_tags":8}'

Congratulations! You’ve successfully processed your first document with the Meta-Data Tag Generator.

Getting Started

Core Features

User Guides

Deployment

Overview

Prerequisites

Step 1: Start the Application

Step 2: Access the Web Interface

Step 3: Configure AI Settings

Step 4: Process Your First Document

Step 5: Understanding the Results

Result Fields Explained

Step 6: Using Exclusion Lists (Optional)

Step 7: Try Batch Processing

Common Issues & Solutions

Performance Tips

For Speed

For Quality

For Cost

For Accuracy

Next Steps

API Integration

Features Deep Dive

Deployment Guide

Example: Processing a Government Document

Build docs developers (and LLMs) love

Getting Started

Core Features

User Guides

Deployment

​Overview

​Prerequisites

​Step 1: Start the Application

​Step 2: Access the Web Interface

​Step 3: Configure AI Settings

​Step 4: Process Your First Document

​Step 5: Understanding the Results

​Result Fields Explained

​Step 6: Using Exclusion Lists (Optional)

​Step 7: Try Batch Processing

​Common Issues & Solutions

​Performance Tips

For Speed

For Quality

For Cost

For Accuracy

​Next Steps

API Integration

Features Deep Dive

Deployment Guide

​Example: Processing a Government Document

Build docs developers (and LLMs) love

Overview

Prerequisites

Step 1: Start the Application

Step 2: Access the Web Interface

Step 3: Configure AI Settings

Step 4: Process Your First Document

Step 5: Understanding the Results

Result Fields Explained

Step 6: Using Exclusion Lists (Optional)

Step 7: Try Batch Processing

Common Issues & Solutions

Performance Tips

Next Steps

Example: Processing a Government Document