Prerequisites
Before starting, ensure you have:- Python 3.8 or higher installed
- Google Chrome browser
- pip (Python package manager)
- Git (for cloning the repository)
Installation Steps
Create Virtual Environment
It’s strongly recommended to use a virtual environment to avoid dependency conflicts:
Your command prompt should now show
(venv) prefix, indicating the virtual environment is active.Install Python Dependencies
Install all required packages from This will install:
requirements.txt:- flask - Web framework for the application
- selenium - Browser automation for scraping
- webdriver-manager - Automatic ChromeDriver management
- beautifulsoup4 - HTML parsing
- pandas - Data manipulation
- openpyxl - Excel file generation
- openai - OpenAI API client
- python-dotenv - Environment variable management
Verify Chrome Installation
The application uses Chrome for web scraping. Verify Chrome is installed:
ChromeDriver is automatically managed by
webdriver-manager, so you don’t need to download it manually.Required Environment Variables
The application requires the following environment variable:OPENAI_API_KEY
Required: YesFormat: String starting with
sk-Purpose: Authenticates requests to OpenAI’s GPT API for job description analysis
Chrome/ChromeDriver Configuration
The application is configured to use Chrome with specific options for optimal scraping:Automatic ChromeDriver Management
The project useswebdriver-manager which automatically:
- Downloads the correct ChromeDriver version for your Chrome browser
- Manages driver updates
- Handles platform-specific configurations
Chrome Options Used
The scraper (linkedin_scraper.py:23-32) configures Chrome with:If you encounter issues with Chrome, ensure you’re running the latest version. ChromeDriver compatibility is automatic but requires a recent Chrome installation.
Virtual Environment Best Practices
Why Use Virtual Environments?
- Isolation: Keeps project dependencies separate from system Python
- Reproducibility: Ensures consistent behavior across different machines
- Clean uninstall: Easy to remove by deleting the venv folder
- Multiple Python versions: Different projects can use different Python versions
Managing Your Virtual Environment
Troubleshooting
Chrome/ChromeDriver Issues
Problem: “ChromeDriver can’t be found”Solution: Ensure Chrome is installed and webdriver-manager has internet access to download drivers Problem: “Session not created: version mismatch”
Solution: Update Chrome to the latest version, webdriver-manager will download the matching driver
Python Dependency Issues
Problem: Package installation failsSolution:
Virtual Environment Not Activating
Problem: Command not found or permission deniedSolution (Linux/Mac):
Verification
Verify your setup is complete:Next Steps
OpenAI API Setup
Configure your OpenAI API key and understand usage costs
Running the Application
Learn how to start and use the job analyzer