We’re looking for a Python developer to build a scalable backend for a new social app. There are several stages we want to go through:

1. Scrape data at scale (min 500K, max 5M requests per day) from multiple websites/webapps
2. Analyze the scraped data
3. Build a sign up/login flow and store user information such as name, email, phone number etc
4. An API to create/retrieve data about groups that each user creates
5. Allow users to upload videos and pics together with metadata (location, hashtags etc)
6. Use open-source generative AI models to create stickers and songs. We will use pre-trained models for this so there’s no need to fine-tune or build one ourselves
7. Allow users to send messages to each other (text, audio, stickers, pics)
8. Generate unique codes for each user, similar to Google Auth

The proposed techstack for the project is as follows:

– Python
– Scraping tools/APIs. We prefer to use out of the box APIs/services but we might need to build some infrastructure ourselves and/or use open-source projects
– AWS
– Supabase

Please do NOT apply to this job if you’ve never built resilient scraping infrastructure. We must be able to handle at least 500K requests per day and scrape various websites without getting blocked.

We will start with a one week trial during which we will assess whether we can build the desired scraping infra.

Hourly Range: $40.00-$70.00

Posted On: January 31, 2024 21:28 UTC
Category: Back-End Development
Skills:Amazon Web Services, Python, REST API, Web Scraping, Web Proxy, Data Scraping, Supabase

Country: United Kingdom

click to apply