Installation - TikTok Miner

This guide covers everything you need to install and configure TikTok Miner for development or production use.

System Requirements

Minimum Requirements

Node.js: 18.x or higher
Package Manager: Bun 1.2.5+ (recommended), npm, yarn, or pnpm
PostgreSQL: 14 or higher
Redis: 7 or higher
Memory: 2GB RAM minimum, 4GB recommended
Storage: 10GB free space

Recommended Setup

OS: Linux, macOS, or Windows with WSL2
Node.js: 20.x LTS
PostgreSQL: 15 or 16
Redis: 7.2 or higher
Memory: 8GB RAM

Installation Methods

Docker (Recommended)
Local Development
Production Deployment

Docker Installation

The easiest way to get started is with Docker Compose, which includes all dependencies.

Prerequisites

Docker 20.10 or higher
Docker Compose 2.0 or higher

Setup Steps

Clone Repository

git clone <repository-url>
cd tiktok-miner

Configure Environment

Create your environment file:

cp .env.example .env

Edit .env with your credentials:

.env

# Database Configuration
DB_PASSWORD=your_secure_database_password

# Redis Configuration
REDIS_PASSWORD=your_secure_redis_password

# Required API Keys
OPENAI_API_KEY=sk-your-openai-key
GITHUB_TOKEN=ghp_your-github-token

# Supabase (for authentication)
NEXT_PUBLIC_SUPABASE_URL=https://your-project.supabase.co
NEXT_PUBLIC_SUPABASE_ANON_KEY=your-anon-key

# Optional: Social Media APIs
TIKTOK_CLIENT_KEY=your-tiktok-key
TIKTOK_CLIENT_SECRET=your-tiktok-secret
INSTAGRAM_CLIENT_ID=your-instagram-id
INSTAGRAM_CLIENT_SECRET=your-instagram-secret
TWITTER_BEARER_TOKEN=your-twitter-token
YOUTUBE_API_KEY=your-youtube-key

Never commit your .env file to version control. Keep your API keys secure.

Start Services

Launch all services:

docker-compose up -d

This starts:

PostgreSQL on port 5432
Redis on port 6379
TikTok Miner App on port 3000
Pipeline Worker (background processing)

Verify Installation

Check service status:

docker-compose ps

View logs:

docker-compose logs -f app

Test the application:

curl http://localhost:3000/api/health

Optional Services

Enable optional services with profiles:

# Start with pgAdmin for database management
docker-compose --profile development up -d

# Start with Nginx reverse proxy
docker-compose --profile production up -d

pgAdmin will be available at http://localhost:5050:

Email: [email protected] (or set PGADMIN_EMAIL)
Password: admin (or set PGADMIN_PASSWORD)

Local Installation

For development or when you prefer to run services individually.

Step 1: Install Dependencies

First, install Node.js and your preferred package manager:

# Install Bun
curl -fsSL https://bun.sh/install | bash

# Install project dependencies
bun install

Step 2: Set Up PostgreSQL

Option A: Local PostgreSQL

Install PostgreSQL on your system:

brew install postgresql@15
brew services start postgresql@15

Create the database:

CREATE DATABASE tiktok_miner;
CREATE USER tiktok_miner_user WITH PASSWORD 'your_password';
GRANT ALL PRIVILEGES ON DATABASE tiktok_miner TO tiktok_miner_user;

Option B: Supabase (Recommended)

Use Supabase for managed PostgreSQL with built-in auth:

Create a free account at supabase.com
Create a new project
Copy your connection strings from Settings → Database
Use the “Connection Pooling” URL for DATABASE_URL
Use the “Direct Connection” URL for DIRECT_URL

Your .env should have:

DATABASE_URL="postgresql://postgres.xxx:[email protected]:6543/postgres?pgbouncer=true&connection_limit=1&pool_timeout=0"
DIRECT_URL="postgresql://postgres.xxx:[email protected]:5432/postgres"

Step 3: Set Up Redis

brew install redis
brew services start redis

Test Redis connection:

redis-cli ping
# Should return: PONG

Step 4: Configure Environment

Copy and configure your environment:

cp .env.example .env

Edit .env with your settings. See Environment Variables for details.

Step 5: Run Database Migrations

bun x prisma generate
bun x prisma migrate deploy

The prisma generate command creates the TypeScript client for database access. Run it whenever you update the schema.

Step 6: Start Development Server

bun run dev

The app will be available at http://localhost:3000

Production Deployment

For production environments, follow these best practices:

Step 1: Build the Application

bun run build

Step 2: Configure Environment

Set production environment variables:

.env.production

NODE_ENV=production

# Use production database
DATABASE_URL="postgresql://..."
DIRECT_URL="postgresql://..."

# Production URLs
NEXT_PUBLIC_APP_URL=https://your-domain.com
NEXT_PUBLIC_SUPABASE_URL=https://your-project.supabase.co

# Secure passwords
DB_PASSWORD=strong_random_password
REDIS_PASSWORD=strong_random_password

Step 3: Start Production Server

docker-compose --profile production up -d

Step 4: Configure Reverse Proxy

Use Nginx as a reverse proxy (included in Docker setup):

nginx.conf

server {
    listen 80;
    server_name your-domain.com;
    
    location / {
        proxy_pass http://app:3000;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_cache_bypass $http_upgrade;
    }
}

Always use HTTPS in production. Configure SSL certificates with Let’s Encrypt or your certificate provider.

Environment Variables

Complete reference of environment variables:

Database Configuration

Variable	Required	Description
`DATABASE_URL`	Yes	PostgreSQL connection string with pgBouncer
`DIRECT_URL`	Yes	Direct PostgreSQL connection for migrations
`PASSWORD`	No	Database password (legacy)

Example:

DATABASE_URL="postgresql://postgres.xxx:[email protected]:6543/postgres?pgbouncer=true&connection_limit=1&pool_timeout=0"
DIRECT_URL="postgresql://postgres.xxx:[email protected]:5432/postgres"

Supabase Configuration

Variable	Required	Description
`NEXT_PUBLIC_SUPABASE_URL`	Yes	Your Supabase project URL
`NEXT_PUBLIC_SUPABASE_ANON_KEY`	Yes	Supabase anonymous key

AI Configuration

Variable	Required	Description	Default
`OPENAI_API_KEY`	Yes	OpenAI API key	-
`OPENAI_MODEL`	No	Model to use	`gpt-4o`
`OPENAI_TEMPERATURE`	No	Response randomness (0-1)	`0.7`
`OPENAI_MIN_CONFIDENCE`	No	Minimum confidence threshold	`0.5`

GitHub Configuration

Variable	Required	Description
`GITHUB_TOKEN`	Recommended	GitHub personal access token
`GITHUB_TOKENS`	No	Comma-separated tokens for load balancing
`GITHUB_TOKEN_LB_STRATEGY`	No	Load balancing strategy

Variable	Required	Description
`TIKTOK_CLIENT_KEY`	No	TikTok API client key
`TIKTOK_CLIENT_SECRET`	No	TikTok API client secret
`INSTAGRAM_CLIENT_ID`	No	Instagram API client ID
`INSTAGRAM_CLIENT_SECRET`	No	Instagram API client secret
`TWITTER_BEARER_TOKEN`	No	Twitter API bearer token
`YOUTUBE_API_KEY`	No	YouTube Data API key

Email Configuration

SMTP
Azure

Variable	Required	Description
`SMTP_HOST`	No	SMTP server host
`SMTP_PORT`	No	SMTP server port (465 or 587)
`SMTP_USER`	No	SMTP username
`SMTP_PASSWORD`	No	SMTP password
`SMTP_SECURE`	No	Use SSL/TLS (true for 465)
`SMTP_AUTH_METHOD`	No	Authentication method

Variable	Required	Description
`AZURE_EMAIL_CONNECTION_STRING`	No	Azure Communication Services connection string

Application Configuration

Variable	Required	Description	Default
`NEXT_PUBLIC_APP_URL`	Yes	Application URL	`http://localhost:3000`
`LOG_LEVEL`	No	Logging level	`info`
`NODE_ENV`	No	Environment	`development`
`PORT`	No	Server port	`3000`

Database Schema

TikTok Miner uses Prisma ORM with the following main models:

Creator Profiles

model CreatorProfile {
  id                      String   @id @default(uuid())
  name                    String
  email                   String?
  bio                     String?
  profileImageUrl         String?
  category                String?
  tags                    String[]
  isVerified              Boolean  @default(false)
  platformIdentifiers     Json
  compositeEngagementScore Float?
  totalReach              Int
  averageEngagementRate   Float?
  contentFrequency        Float?
  audienceQualityScore    Float?
  lastSync                DateTime @default(now())
  syncStatus              SyncStatus @default(PENDING)
  
  // Relations
  youtubeMetrics          YoutubeMetrics?
  twitterMetrics          TwitterMetrics?
  instagramMetrics        InstagramMetrics?
  tiktokMetrics           TiktokMetrics?
  metricsHistory          CreatorMetricsHistory[]
  engagementAnalytics     EngagementAnalytics[]
}

TikTok Metrics

model TiktokMetrics {
  id                String   @id @default(uuid())
  creatorProfileId  String   @unique
  userId            String   @unique
  followerCount     Int
  followingCount    Int
  videoCount        Int
  heartCount        BigInt
  averageViews      Int
  averageLikes      Int
  averageComments   Int
  averageShares     Int
  engagementRate    Float
  username          String   @unique
  nickname          String?
  profileUrl        String
  bio               String?
  isVerified        Boolean
  totalViews        BigInt
  dailyViewGrowth   Float?
  dailyFollowerGrowth Float?
}

Budget Management

TikTok Miner includes comprehensive budget tracking:

model Budget {
  id                String         @id @default(uuid())
  name              String
  budgetType        BudgetType
  totalAmount       Float
  spentAmount       Float
  remainingAmount   Float
  startDate         DateTime
  endDate           DateTime
  status            BudgetStatus
  alertThreshold80  Boolean
  alertThreshold90  Boolean
  alertThreshold100 Boolean
  autoRenewal       Boolean
}

For the complete schema, see prisma/schema.prisma in the repository.

Verification

Verify your installation:

Check Database Connection

bun x prisma db pull

Test API Health

curl http://localhost:3000/api/health

Run Tests

bun run test:all

Next Steps

Quickstart Guide

Run your first creator search in 5 minutes

Environment Variables

Deep dive into configuration options

CLI Tools

Learn about command-line tools

API Reference

Explore the REST API

Troubleshooting

Common Issues

Prisma generate fails

This usually means the database connection is incorrect or unreachable.Solutions:

Verify DATABASE_URL and DIRECT_URL are correct
Check PostgreSQL is running
Test connection: psql $DATABASE_URL
Ensure database exists

Port already in use

Another service is using port 3000.Solutions:

Stop the conflicting service
Change port: PORT=3001 bun run dev
In Docker: Update docker-compose.yml ports section

Redis connection refused

Redis is not running or not accessible.Solutions:

Start Redis: redis-server
Check if running: redis-cli ping
Verify REDIS_URL in .env
Check firewall rules for port 6379

Supabase authentication errors

Supabase configuration is incorrect.Solutions:

Verify NEXT_PUBLIC_SUPABASE_URL matches your project
Check NEXT_PUBLIC_SUPABASE_ANON_KEY is valid
Ensure Supabase project is active
Test in Supabase dashboard first

Migration errors

Database schema doesn’t match expected state.Solutions:

Reset database: bun x prisma migrate reset
Check migration history: bun x prisma migrate status
Apply pending migrations: bun x prisma migrate deploy
Use DIRECT_URL for migrations, not pooled connection

Getting Help

If you’re still having issues:

Check the logs: docker-compose logs -f or tail -f logs/dev.log
Review the Docker Deployment troubleshooting section
Check Database Setup for migration issues
Search GitHub Issues

Getting Started

Core Features

Platform Integration

Advanced

Deployment

​System Requirements

​Minimum Requirements

​Recommended Setup

​Installation Methods

​Docker Installation

​Prerequisites

​Setup Steps

​Optional Services

​Local Installation

​Step 1: Install Dependencies

​Step 2: Set Up PostgreSQL

​Step 3: Set Up Redis

​Step 4: Configure Environment

​Step 5: Run Database Migrations

​Step 6: Start Development Server

​Production Deployment

​Step 1: Build the Application

​Step 2: Configure Environment

​Step 3: Start Production Server

​Step 4: Configure Reverse Proxy

​Environment Variables

​Database Configuration

​Supabase Configuration

​AI Configuration

​GitHub Configuration

​Social Media APIs

​Email Configuration

​Application Configuration

​Database Schema

​Creator Profiles

​TikTok Metrics

​Budget Management

​Verification

​Check Database Connection

​Test API Health

​Run Tests

​Next Steps

Quickstart Guide

Environment Variables

CLI Tools

API Reference

​Troubleshooting

​Common Issues

​Getting Help

Build docs developers (and LLMs) love

System Requirements

Minimum Requirements

Recommended Setup

Installation Methods

Docker Installation

Prerequisites

Setup Steps

Optional Services

Local Installation

Step 1: Install Dependencies

Step 2: Set Up PostgreSQL

Step 3: Set Up Redis

Step 4: Configure Environment

Step 5: Run Database Migrations

Step 6: Start Development Server

Production Deployment

Step 1: Build the Application

Step 2: Configure Environment

Step 3: Start Production Server

Step 4: Configure Reverse Proxy

Environment Variables

Database Configuration

Supabase Configuration

AI Configuration

GitHub Configuration

Social Media APIs

Email Configuration

Application Configuration

Database Schema

Creator Profiles

TikTok Metrics

Budget Management

Verification

Check Database Connection

Test API Health

Run Tests

Next Steps

Troubleshooting

Common Issues

Getting Help