Proxies - JobSpy JS

Overview

JobSpy JS supports rotating proxies to distribute requests across multiple IP addresses. This is useful for:

Avoiding rate limits
Bypassing IP-based blocking
Accessing geo-restricted job boards
Scaling up scraping volume

Proxies are optional but recommended for high-volume scraping or when dealing with strict rate limits.

Basic Usage

Single Proxy

import { scrapeJobs } from "jobspy-js";

const result = await scrapeJobs({
  site_name: ["indeed", "linkedin"],
  search_term: "software engineer",
  proxies: "user:[email protected]:8080",
});

Multiple Proxies (Rotation)

const result = await scrapeJobs({
  site_name: ["indeed", "linkedin", "glassdoor"],
  search_term: "software engineer",
  proxies: [
    "user:[email protected]:8080",
    "user:[email protected]:8080",
    "user:[email protected]:8080",
  ],
});

Proxies rotate round-robin across concurrent scraper instances. If you’re scraping 3 sites with 3 proxies:

Site 1 uses proxy 1
Site 2 uses proxy 2
Site 3 uses proxy 3

Proxy Formats

JobSpy JS accepts proxies in multiple formats:

Format	Example
`host:port`	`proxy.example.com:8080`
`user:pass@host:port`	`admin:[email protected]:8080`
`http://host:port`	`http://proxy.example.com:8080`
`https://host:port`	`https://proxy.example.com:8080`
`socks5://host:port`	`socks5://proxy.example.com:1080`

Auto-prefixing: Bare host:port strings are automatically prefixed with http://:

// These are equivalent:
proxies: "proxy.example.com:8080"
proxies: "http://proxy.example.com:8080"

Configuration

scrapeJobs()

import { scrapeJobs } from "jobspy-js";

const result = await scrapeJobs({
  site_name: ["linkedin", "indeed"],
  search_term: "developer",
  proxies: [
    "user:[email protected]:8080",
    "user:[email protected]:8080",
  ],
});

proxies

string | string[]

Proxy server(s) for rotating requests. Can be a single proxy string or an array of proxies.

// Single proxy
proxies: "user:[email protected]:8080"

// Multiple proxies
proxies: [
  "proxy1.example.com:8080",
  "proxy2.example.com:8080",
]

fetchJobDetails()

import { fetchJobDetails } from "jobspy-js";

const job = await fetchJobDetails("indeed", "fdde406379455a1e", {
  proxies: "user:[email protected]:8080",
});

fetchLinkedInJob()

import { fetchLinkedInJob } from "jobspy-js";

const job = await fetchLinkedInJob("4127292817", {
  proxies: "user:[email protected]:8080",
});

Proxy Types

HTTP/HTTPS Proxies

Most common proxy type. Works with all scrapers.

proxies: "http://user:[email protected]:8080"
proxies: "https://user:[email protected]:8080"

SOCKS5 Proxies

Supports TCP-level proxying. Works with all scrapers.

proxies: "socks5://user:[email protected]:1080"

Authenticated Proxies

Proxies with username/password authentication:

proxies: "user:[email protected]:8080"
proxies: "http://user:[email protected]:8080"

Unauthenticated Proxies

Proxies without authentication:

proxies: "proxy.example.com:8080"
proxies: "http://proxy.example.com:8080"

How Rotation Works

Round-Robin Rotation

Proxies rotate in order across concurrent scraper instances:

const result = await scrapeJobs({
  site_name: ["indeed", "linkedin", "glassdoor", "ziprecruiter"],
  search_term: "engineer",
  proxies: [
    "proxy1.example.com:8080",  // Used by Indeed
    "proxy2.example.com:8080",  // Used by LinkedIn
    "proxy3.example.com:8080",  // Used by Glassdoor
  ],
});

// ZipRecruiter wraps around to proxy1

Per-Scraper Assignment

Each scraper instance gets a single proxy for all its requests. Proxies are assigned when the scraper is initialized.

// Example with 2 sites and 3 proxies
const result = await scrapeJobs({
  site_name: ["indeed", "linkedin"],
  proxies: ["proxy1", "proxy2", "proxy3"],
});

// Indeed scraper: uses proxy1 for all requests
// LinkedIn scraper: uses proxy2 for all requests
// proxy3 is unused in this run

Examples

Basic Proxy Rotation

import { scrapeJobs } from "jobspy-js";

const result = await scrapeJobs({
  site_name: ["linkedin", "indeed", "glassdoor"],
  search_term: "data scientist",
  location: "San Francisco, CA",
  proxies: [
    "user1:[email protected]:8080",
    "user2:[email protected]:8080",
    "user3:[email protected]:8080",
  ],
});

console.log(`Found ${result.jobs.length} jobs`);

SOCKS5 Proxy

const result = await scrapeJobs({
  site_name: ["linkedin"],
  search_term: "engineer",
  proxies: "socks5://user:[email protected]:1080",
});

Fetching Job Details via Proxy

import { fetchJobDetails } from "jobspy-js";

const job = await fetchJobDetails("indeed", "fdde406379455a1e", {
  proxies: "user:[email protected]:8080",
  format: "markdown",
});

console.log(job?.description);

LinkedIn Job via Proxy

import { fetchLinkedInJob } from "jobspy-js";

const job = await fetchLinkedInJob("4127292817", {
  proxies: "user:[email protected]:8080",
  format: "plain",
});

console.log(job.description);

Loading Proxies from Environment

// Set JOBSPY_PROXIES="proxy1:8080,proxy2:8080,proxy3:8080"
const proxies = process.env.JOBSPY_PROXIES?.split(",") || [];

const result = await scrapeJobs({
  site_name: ["indeed", "linkedin"],
  search_term: "developer",
  proxies: proxies.length > 0 ? proxies : undefined,
});

Combining Proxies with Authentication

import { scrapeJobs } from "jobspy-js";

const result = await scrapeJobs({
  site_name: ["linkedin"],
  search_term: "engineer",
  proxies: "user:[email protected]:8080",
  use_creds: true,
  credentials: {
    linkedin: {
      username: process.env.LINKEDIN_USERNAME!,
      password: process.env.LINKEDIN_PASSWORD!,
    },
  },
});

CLI Usage

Single Proxy

jobspy -s linkedin -q "engineer" -p "user:[email protected]:8080"

Multiple Proxies

jobspy -s linkedin indeed -q "engineer" \
  -p "proxy1.example.com:8080" \
  -p "proxy2.example.com:8080" \
  -p "proxy3.example.com:8080"

With Authentication

jobspy -s linkedin -q "engineer" \
  -p "user:[email protected]:8080" \
  --creds \
  --linkedin-username "[email protected]" \
  --linkedin-password "secret"

Proxy Providers

JobSpy JS works with any standard HTTP/HTTPS/SOCKS5 proxy service. Popular providers include:

Residential proxies: Better for avoiding detection (Bright Data, Oxylabs, Smartproxy)
Datacenter proxies: Faster and cheaper (Webshare, Proxy-Cheap)
Rotating proxies: Automatic IP rotation on each request (GeoNode, ProxyScrape)

Residential proxies from the same country as your target jobs are less likely to trigger rate limits or blocks.

Troubleshooting

Proxy Connection Failures

Symptoms: ECONNREFUSED, ETIMEDOUT, or ENOTFOUND errors Solutions:

Verify proxy is online and reachable
Check proxy credentials are correct
Ensure firewall allows outbound connections to proxy
Try a different proxy from your provider

// Test proxy connectivity
try {
  const result = await scrapeJobs({
    site_name: ["indeed"],
    search_term: "test",
    results_wanted: 1,
    proxies: "user:[email protected]:8080",
    verbose: 2,  // Enable debug logging
  });
  console.log("Proxy works!");
} catch (err) {
  console.error("Proxy failed:", err);
}

Proxy Authentication Errors

Symptoms: 407 Proxy Authentication Required Solutions:

Verify username and password are correct
Check for special characters in credentials (URL-encode if needed)
Ensure proxy supports authentication

// URL-encode credentials with special characters
const username = encodeURIComponent("user@domain");
const password = encodeURIComponent("p@ssw0rd!");
const proxy = `${username}:${password}@proxy.example.com:8080`;

const result = await scrapeJobs({
  site_name: ["indeed"],
  search_term: "test",
  proxies: proxy,
});

Slow Performance

Symptoms: Scraping takes much longer than expected Solutions:

Use datacenter proxies instead of residential (faster but less reliable)
Use proxies geographically close to target job boards
Reduce results_wanted to minimize requests
Check proxy provider’s speed benchmarks

Rate Limiting Even with Proxies

Symptoms: Still getting 429 errors or empty results Solutions:

Use residential proxies (harder to detect)
Add delays between requests
Rotate more proxies
Enable credential fallback with use_creds: true
Reduce scraping volume

Best Practices

1. Use Enough Proxies

Match the number of proxies to the number of sites:

// Good: 3 sites, 3+ proxies
const result = await scrapeJobs({
  site_name: ["indeed", "linkedin", "glassdoor"],
  proxies: ["proxy1", "proxy2", "proxy3"],
});

// Suboptimal: 3 sites, 1 proxy
const result = await scrapeJobs({
  site_name: ["indeed", "linkedin", "glassdoor"],
  proxies: "proxy1",  // All sites share one IP
});

2. Use Residential Proxies for Sensitive Sites

LinkedIn and Glassdoor are more aggressive about blocking datacenter IPs:

const result = await scrapeJobs({
  site_name: ["linkedin", "glassdoor"],
  proxies: [
    "residential-proxy1.example.com:8080",
    "residential-proxy2.example.com:8080",
  ],
});

3. Store Proxies Securely

Never hardcode proxy credentials:

// ❌ Bad
proxies: "user:[email protected]:8080"

// ✅ Good
proxies: process.env.PROXY_URL

4. Monitor Proxy Health

Track which proxies work and which fail:

const proxies = [
  "proxy1.example.com:8080",
  "proxy2.example.com:8080",
  "proxy3.example.com:8080",
];

for (const proxy of proxies) {
  try {
    const result = await scrapeJobs({
      site_name: ["indeed"],
      search_term: "test",
      results_wanted: 1,
      proxies: proxy,
    });
    console.log(`✓ ${proxy} works`);
  } catch (err) {
    console.log(`✗ ${proxy} failed:`, err.message);
  }
}

5. Rotate Proxies Periodically

Don’t reuse the same proxy for extended periods:

// Refresh proxy list every hour
let proxies = await fetchFreshProxies();

setInterval(async () => {
  proxies = await fetchFreshProxies();
}, 60 * 60 * 1000);

Get Started

SDK Guide

CLI

MCP Server

Job Boards

​Overview

​Basic Usage

​Single Proxy

​Multiple Proxies (Rotation)

​Proxy Formats

​Configuration

​scrapeJobs()

​fetchJobDetails()

​fetchLinkedInJob()

​Proxy Types

​HTTP/HTTPS Proxies

​SOCKS5 Proxies

​Authenticated Proxies

​Unauthenticated Proxies

​How Rotation Works

​Round-Robin Rotation

​Per-Scraper Assignment

​Examples

​Basic Proxy Rotation

​SOCKS5 Proxy

​Fetching Job Details via Proxy

​LinkedIn Job via Proxy

​Loading Proxies from Environment

​Combining Proxies with Authentication

​CLI Usage

​Single Proxy

​Multiple Proxies

​With Authentication

​Proxy Providers

​Troubleshooting

​Proxy Connection Failures

​Proxy Authentication Errors

​Slow Performance

​Rate Limiting Even with Proxies

​Best Practices

​1. Use Enough Proxies

​2. Use Residential Proxies for Sensitive Sites

​3. Store Proxies Securely

​4. Monitor Proxy Health

​5. Rotate Proxies Periodically

​See Also

Build docs developers (and LLMs) love

Overview

Basic Usage

Single Proxy

Multiple Proxies (Rotation)

Proxy Formats

Configuration

scrapeJobs()

fetchJobDetails()

fetchLinkedInJob()

Proxy Types

HTTP/HTTPS Proxies

SOCKS5 Proxies

Authenticated Proxies

Unauthenticated Proxies

How Rotation Works

Round-Robin Rotation

Per-Scraper Assignment

Examples

Basic Proxy Rotation

SOCKS5 Proxy

Fetching Job Details via Proxy

LinkedIn Job via Proxy

Loading Proxies from Environment

Combining Proxies with Authentication

CLI Usage

Single Proxy

Multiple Proxies

With Authentication

Proxy Providers

Troubleshooting

Proxy Connection Failures

Proxy Authentication Errors

Slow Performance

Rate Limiting Even with Proxies

Best Practices

1. Use Enough Proxies

2. Use Residential Proxies for Sensitive Sites

3. Store Proxies Securely

4. Monitor Proxy Health

5. Rotate Proxies Periodically

See Also