We are developing a Chrome extension designed to enhance the online shopping experience by automatically detecting and selecting clothing sizes on various e-commerce websites. The project involves scraping e-commerce websites to understand their DOM structure, identifying size selection elements, and developing algorithms to automate the size selection process. We are looking for a highly skilled Web Scraping Expert to lead the data collection and analysis phase of this project.

Key Responsibilities:

1. Web Scraping and Data Extraction:

Scrape E-Commerce Websites: Develop and deploy web scraping scripts to collect data from various e-commerce websites, focusing on elements related to size selection (e.g., dropdowns, radio buttons, and clickable divs).

Handle Dynamic Content: Effectively scrape content from sites that use AJAX, JavaScript, or other dynamic content loading techniques.

Extract DOM Structures: Collect and analyze the DOM structures from multiple e-commerce platforms to identify patterns and commonalities in size selection elements.

2. DOM Analysis and Mapping:

Analyze HTML/CSS: Perform detailed analysis of HTML and CSS to understand the structure and presentation of size selection elements across different websites.
Identify Size Selection Elements: Map out and document the specific DOM elements (e.g., IDs, classes, attributes) used for size selection on each platform.

Develop Detection Logic: Collaborate with the front-end development team to create algorithms that can detect and interact with these size selection elements automatically.

3. Anti-Scraping Measures:

Bypass Anti-Scraping Techniques: Develop solutions to handle and bypass common anti-scraping measures like CAPTCHAs, IP blocking, and rate limiting, while adhering to legal and ethical standards.

Monitor Site Changes: Set up monitoring tools or methods to detect when the target websites update their DOM structure, ensuring the scraping logic remains functional over time.

4. Integration with Chrome Extension:

Collaborate with Developers: Work closely with front-end and back-end developers to integrate the scraping and DOM detection logic into the Chrome extension.

Optimize for Performance: Ensure that the scraping processes are optimized for speed and efficiency to provide a seamless user experience in the Chrome extension.

5. Documentation and Reporting:

Document Processes: Thoroughly document all scraping scripts, DOM analysis findings, and the logic used for detecting size selection elements.

Provide Regular Updates: Report progress regularly to project managers and stakeholders, highlighting any challenges encountered and proposed solutions.

Hourly Range: $30.00-$60.00

Posted On: August 18, 2024 06:18 UTC
Category: Scripting & Automation
Skills:Data Scraping, Data Mining, Python, JavaScript, Scrapy

Country: Israel

click to apply