Project Overview:
We are seeking an experienced Python developer to create a robust web scraping and automation system for a web-based tool that extracts information job applications via url. Our tool currently utilizes CSS tags to scrape job descriptions from web pages. However, due to the dynamic nature of web pages and frequent changes in their structure, this method has proven unreliable. We aim to develop a new system that employs a headless browser approach for more resilient and effective data extraction.

Core Responsibilities:

Design and Implement Web Scraping System: Develop a new web scraping service that utilizes a headless browser to accurately and efficiently extract job description information from provided URLs.

Integration with Existing Tools: Ensure seamless integration of the new scraping service with our existing Python-based web tool, enabling automated resume editing based on job descriptions.

API Development: Create a Node.js endpoint to allow access to the scraping service, ensuring compatibility and ease of use within our existing infrastructure.

Data Extraction Accuracy: Implement mechanisms to validate and ensure the accuracy of the extracted data from various job posting websites, with considerations for handling different website layouts and structures.

Maintenance and Scalability: Develop the system with scalability in mind, including the ability to update or modify the scraping logic as web standards evolve. Additionally, provide documentation and support for future maintenance and updates.
Skills and Qualifications:

Strong Python Programming Skills: Proficient in Python, with extensive experience in web scraping and automation.
Experience with Headless Browsers: Demonstrated experience in using headless browsers for web scraping, such as Selenium or Puppeteer.

Familiarity with Node.js: Capable of creating and managing endpoints in Node.js to integrate Python services within a broader application context.

Web Technologies: Good understanding of web technologies, including HTML, CSS, JavaScript, and how web pages are structured and rendered.

Problem-Solving Skills: Ability to tackle challenges related to web scraping, such as handling dynamic content, pagination, and site-specific obstacles.

Communication: Excellent communication skills to clearly articulate technical challenges and solutions with the team.

Project Duration and Payment:
Please provide an estimated timeline for the completion of the project along with your bid. Include any assumptions or prerequisites needed for the project’s success.
Payment terms and milestones will be discussed and agreed upon before project initiation.

Application Process:
Interested candidates are invited to submit a proposal that includes:

A brief overview of your experience with similar projects.
– A high-level approach or strategy you would consider for this project.
– Any questions or clarifications you may need to better understand the project scope.

Conclusion:
This project offers an exciting opportunity to work on a unique web-based tool that has a real impact on job seekers. We look forward to collaborating with a skilled developer who can help us overcome our current challenges and contribute to the continued success of our tool.

Posted On: March 07, 2024 14:20 UTC
Category: Back-End Development
Skills:Python, Data Scraping, Automation, API, Python Script, Node.js

Country: United States

click to apply