Skip to content

[FIX] Update LinkedIn scraper for current DOM structure, Selenium 4.x compatibility, Dev enhancements (UV, Ruff, pre-commit) #252

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 9 commits into
base: master
Choose a base branch
from

Conversation

stickerdaniel
Copy link
Contributor

Hey, I am using this Repository for my LinkedIn MCP Server, which wraps the provided scraping functions of this repository with tools that LLMs can access. As users experience issues with the provided tools, here is a list of changes that make this project more usable again and simplify future contributions.

Changes

Issues

  • Fixed deprecated Selenium methods (find_element_by_* → find_element(By.*)) Update/Fix for the Issue: Not able to scrape jobs list because of depreciated selenium function #190
  • Updated CSS selectors for the current LinkedIn DOM structure:
    • job-card-container (jobs)
    • artdeco-entity-lockup__subtitle (company names)
    • job-card-container__link (job titles)
  • Resolved stale element reference issues in job scraping
  • Implemented error handling/fallback selectors
  • Refactored recommended jobs logic to work with the current page structure

Examples Added

  • Job search with keyword filtering
  • Recommended jobs from the LinkedIn homepage
  • Individual job details scraping
  • Company profile and people extraction
  • Person contacts and profile data

Testing

All scraping functionality from the added examples works.

Dev Enhancements

  • Update the project to use UV package manager instead of pip
  • added ruff (linting & formatting)
  • added pre-commit hook

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant