Skip to main content

Overview

The Job Finder Agent is a sophisticated multi-agent system that analyzes LinkedIn profiles and finds personalized job opportunities. It orchestrates six specialized agents in a sequential pipeline, using Bright Data’s MCP server for web scraping.

6-Agent Pipeline

Sequential agent handoffs for comprehensive analysis

LinkedIn Profiling

Deep professional background analysis

Smart Job Matching

Y Combinator jobs based on skills and experience

MCP Integration

Bright Data MCP server for web scraping

Architecture Pattern

This agent demonstrates the Sequential Multi-Agent Pipeline pattern using OpenAI Agents SDK with MCP integration.

Agent Pipeline

import asyncio
from agents import Agent, Runner, OpenAIChatCompletionsModel
from agents.mcp import MCPServer
from openai import AsyncOpenAI

# Initialize OpenAI client with Nebius
client = AsyncOpenAI(
    base_url="https://api.tokenfactory.nebius.com/v1",
    api_key=os.environ["NEBIUS_API_KEY"]
)

model = OpenAIChatCompletionsModel(
    model="meta-llama/Llama-3.3-70B-Instruct",
    openai_client=client
)

Six Specialized Agents

1

LinkedIn Analyzer

Extracts professional experience, skills, education, and reputation from LinkedIn profiles.
2

Domain Classifier

Identifies primary professional domain (Software Engineering, Design, Product, etc.).
3

URL Generator

Maps domain to Y Combinator job board URL.
4

Job Finder

Scrapes job listings using Bright Data MCP server.
5

URL Parser

Transforms authentication URLs to direct application links.
6

Summary Generator

Creates comprehensive career analysis report.

Multi-Agent Implementation

Agent 1: LinkedIn Profile Analyzer

linkedin_agent = Agent(
    name="LinkedIn Profile Analyzer",
    instructions="""
        Analyze LinkedIn profiles for:
        - Professional experience and career progression
        - Education and certifications
        - Core skills and expertise
        - Current role and company
        - Industry reputation (recommendations/endorsements)
        
        Provide structured analysis with bullet points.
        
        RULES:
        - If no experience, say "No experience found"
        - If information unavailable, say "Not available"
        - Do NOT make up information
        - Call tool ONCE, then use the response
    """,
    mcp_servers=[mcp_server],  # Bright Data MCP for scraping
    model=model
)

Agent 2: Job Suggestions (Domain Classification)

job_suggestions_agent = Agent(
    name="Job Suggestions",
    instructions="""
        Domain classifier that identifies primary professional domain.
        
        Select ONE domain from:
        - Software Engineering (programming, development)
        - Design & UI/UX (design, user experience)
        - Product Management (strategy, roadmap)
        - Recruiting & HR (talent acquisition)
        - Sales (business development)
        - Science (research, data science)
        - Marketing (content, developer relations)
        
        Rules:
        - Choose based on PRIMARY skills and experience
        - Pick most recent/relevant if multiple domains
        - Default to Software Engineering if unclear
        
        Return JSON:
        {
            "selected_domain": "chosen domain",
            "confidence_score": 0-100,
            "selection_reason": "brief explanation"
        }
    """,
    model=model
)

Agent 3: URL Generator

url_generator_agent = Agent(
    name="URL Generator",
    instructions="""
        Creates Y Combinator job board URLs based on domains.
        
        Domain to URL mapping:
        - "Software Engineering" -> "ycombinator.com/jobs/role/software-engineer"
        - "Design & UI/UX" -> "ycombinator.com/jobs/role/designer"
        - "Product Management" -> "ycombinator.com/jobs/role/product-manager"
        - "Recruiting & HR" -> "ycombinator.com/jobs/role/recruiting-hr"
        - "Sales" -> "ycombinator.com/jobs/role/sales-manager"
        - "Science" -> "ycombinator.com/jobs/role/science"
        - "Marketing" -> "ycombinator.com/jobs/role/marketing"
        
        Output:
        {
            "job_board_url": "mapped url",
            "domain": "original domain"
        }
    """,
    model=model
)

Agent 4: Job Search Agent

Job_search_agent = Agent(
    name="Job Finder",
    instructions="""
        Extracts job listings from Y Combinator job board.
        
        Steps:
        1. Take URL from URL generator's JSON response
        2. Fetch job listings ONCE using provided tools
        3. Extract for first 5 relevant jobs:
           - Company name
           - Job title
           - Job type (Full-time/Part-time/Contract)
           - Location (including remote status)
           - Apply URL
        
        Format:
        ## Job Matches for [Domain]
        ### [Job Title]
        - **Company:** [Company Name]
        - **Type:** [Job Type]
        - **Location:** [Location]
        - **Apply:** [Apply URL]
        
        CRITICAL: Call tool EXACTLY ONCE
    """,
    mcp_servers=[mcp_server],  # Uses Bright Data for scraping
    model=model
)

Agent 5: URL Parser

url_parser_agent = Agent(
    name="URL Parser",
    instructions="""
        Transforms Y Combinator auth URLs into direct job URLs.
        
        Process:
        1. Extract job_id from 'signup_job_id=' parameter
           Example: '...signup_job_id=75187...' → '75187'
        
        2. Create direct URL:
           Base: 'https://www.workatastartup.com/jobs/'
           Result: 'https://www.workatastartup.com/jobs/75187'
        
        3. Replace Apply URL in each job listing
        
        Keep all other job information unchanged.
    """,
    model=model
)

Agent 6: Summary Agent

summary_agent = Agent(
    name="Summary Agent",
    instructions="""
        Creates comprehensive career analysis report.
        
        Structure:
        ## 👤 Profile Summary
        [Concise career summary]
        
        ## 🎯 Your Top Skills:
        - [Skill 1]
        - [Skill 2]
        ...
        
        ## 💡 Suggested Roles:
        ### [Role Title]
        - **Why this role?** [Explanation]
        - **Required Skills:** [Skills]
        - **Potential Companies:** [Companies]
        - **Growth Potential:** [Opportunities]
        - **Salary Range:** [Range if available]
        
        ## 💼 Current Job Matches:
        ### [Job Title] at [Company]
        - [Description]
        - Match Score: [Score]%
        - [Apply Here]([URL])
        
        RULE: Do NOT make up information not provided in inputs.
    """,
    model=model
)

Sequential Execution

async def run_analysis(mcp_server: MCPServer, linkedin_url: str):
    query = f"""Analyze LinkedIn profile at {linkedin_url}.
                Focus on professional background.
                Find best jobs based on profile."""
    
    try:
        # Step 1: LinkedIn Analysis
        linkedin_result = await Runner.run(
            starting_agent=linkedin_agent,
            input=query
        )
        
        # Step 2: Domain Classification
        suggestions_result = await Runner.run(
            starting_agent=job_suggestions_agent,
            input=linkedin_result.final_output
        )
        
        # Step 3: URL Generation
        job_link_result = await Runner.run(
            starting_agent=url_generator_agent,
            input=suggestions_result.final_output
        )
        
        # Step 4: Job Search
        job_search_result = await Runner.run(
            starting_agent=Job_search_agent,
            input=job_link_result.final_output
        )
        
        # Step 5: URL Parsing
        parsed_urls_result = await Runner.run(
            starting_agent=url_parser_agent,
            input=job_search_result.final_output
        )
        
        # Step 6: Summary Generation
        summary_input = f"""
            LinkedIn Profile Analysis:
            {linkedin_result.final_output}
            
            Job Suggestions:
            {suggestions_result.final_output}
            
            Job Matches:
            {parsed_urls_result.final_output}
            
            Create comprehensive career analysis report in markdown.
        """
        
        summary_result = await Runner.run(
            starting_agent=summary_agent,
            input=summary_input
        )
        
        return summary_result.final_output
        
    except Exception as e:
        logger.error(f"Error during analysis: {str(e)}")
        raise

MCP Server Integration

Bright Data MCP Server

from agents.mcp import MCPServerStdio
import asyncio

_mcp_server = None

async def initialize_mcp_server():
    """Initialize Bright Data MCP server for web scraping."""
    global _mcp_server
    
    if _mcp_server:
        return _mcp_server
    
    try:
        server = MCPServerStdio(
            cache_tools_list=False,
            params={
                "command": "npx",
                "args": ["-y", "@brightdata/mcp"],
                "env": {
                    "API_TOKEN": os.environ["BRIGHT_DATA_API_KEY"],
                    "WEB_UNLOCKER_ZONE": "mcp_unlocker",
                    "BROWSER_AUTH": os.environ["BROWSER_AUTH"],
                }
            }
        )
        
        await asyncio.wait_for(server.__aenter__(), timeout=10)
        _mcp_server = server
        return server
        
    except Exception as e:
        logger.error(f"Error initializing MCP server: {e}")
        return None

def get_mcp_server():
    """Get the current MCP server instance."""
    return _mcp_server

Streamlit Integration

import streamlit as st
import asyncio
from mcp_server import initialize_mcp_server, get_mcp_server
from job_agents import run_analysis

st.title("Job Finder Agent")

# Initialize MCP server on app start
if 'mcp_initialized' not in st.session_state:
    with st.spinner("Initializing MCP server..."):
        asyncio.run(initialize_mcp_server())
        st.session_state.mcp_initialized = True

linkedin_url = st.text_input(
    "LinkedIn Profile URL",
    placeholder="https://linkedin.com/in/username"
)

if st.button("Analyze Profile"):
    if not linkedin_url:
        st.error("Please enter a LinkedIn URL")
    else:
        with st.spinner("Analyzing profile and finding jobs..."):
            mcp_server = get_mcp_server()
            
            if mcp_server:
                # Run async analysis
                result = asyncio.run(run_analysis(mcp_server, linkedin_url))
                st.markdown(result)
            else:
                st.error("MCP server not initialized")

Advanced Patterns

Async Agent Execution

# All agents run asynchronously
await Runner.run(starting_agent=agent, input=query)

# Enables:
# - Non-blocking execution
# - Efficient MCP server communication
# - Better error handling

Agent-to-Agent Data Flow

# Each agent's output feeds the next
Step 1 Output (LinkedIn data)

Step 2 Input (Domain classification)

Step 2 Output (JSON with domain)

Step 3 Input (URL generation)

... continues through all 6 agents

Structured Output Parsing

# Agents output JSON for structured data transfer
suggestions_result.final_output = '''
{
    "selected_domain": "Software Engineering",
    "confidence_score": 95,
    "selection_reason": "Strong Python and React experience"
}
'''

# Next agent parses JSON and uses data

Configuration

Environment Variables

NEBIUS_API_KEY=your_nebius_api_key
BRIGHT_DATA_API_KEY=your_bright_data_key
BROWSER_AUTH=your_browser_auth_credentials

Model Selection

# All agents use same model for consistency
model = OpenAIChatCompletionsModel(
    model="meta-llama/Llama-3.3-70B-Instruct",
    openai_client=client
)

Use Cases

Career Transition

Find jobs matching transferable skills from LinkedIn

Job Search

Automated job discovery based on experience

Skill Gap Analysis

Identify skills needed for target roles

Career Path Planning

Discover potential career trajectories

Project Structure

job_finder_agent/
├── app.py              # Streamlit web interface
├── job_agents.py       # 6-agent pipeline implementation
├── mcp_server.py       # Bright Data MCP server management
├── requirements.txt    # Dependencies
└── .env               # API keys (not committed)

Candilyzer

Reverse flow: candidate evaluation instead of job finding

Deep Researcher

Similar sequential pipeline pattern

Learn More

OpenAI Agents SDK

Async agent orchestration with Runner

MCP Integration

Model Context Protocol for external tools

Advanced Agents

More advanced agent examples

Build docs developers (and LLMs) love