Scrapy Jobs

I need a robust yet easy-to-maintain web scraper that pulls player statistics from four different sports sites—a blend of official league pages, sports news outlets, and a couple of well-known fan forums. All scraped data should flow into a single database and surface through a lightweight web dashboard where I can search by player, season, and team, compare numbers side by side, and export results to CSV. My ideal flow looks like this: enter or schedule the URLs, run or auto-run the scraper, watch progress logs, and then immediately view fresh stats inside the dashboard—no command-line work once everything is deployed. If any source changes its HTML, the scraper should fail gracefully and flag the issue in the UI so I can react quickly. Tech stack is flexible; Python with Be...

I need a robust AI-driven scraper that reliably pulls product text, images, SKUs, and pricing from selected e-commerce websites and our own admin-controlled catalog portals. The script must handle pagination, dynamic content, and, where required, authenticated sessions without manual intervention. Core expectations • Extract and store: product titles/descriptions, all associated images, SKU codes, and current prices. • Export: structured CSV or JSON for data, separate folder (or S3 bucket) for images, with clear file naming that links each image back to its SKU. • Tech stack: Python with libraries such as Scrapy, Playwright/Selenium, BeautifulSoup, or a comparable approach—whatever you can prove is most efficient and resilient. Basic computer-vision or OCR hoo...

Need experienced dev to fix my claude-coded python playwright automation script. Note that I don't know any coding. So need your help to review and then fix. Details will be provided. Reply with "claude" so I know you've really read this very short job description.

I need a clean, well-structured scrape of vendor details from jiii.ng. The goal is straightforward: pull each vendor’s name and contact information and hand the results back to me in a single Excel workbook. Categories The focus is on the main marketplace sections. I will confirm the exact categories before you begin, but expect the usual high-traffic areas (Electronics, Real Estate, Automobiles) to be included. Data points • Vendor name • All available contact details (phone, email, WhatsApp, or any other channel the site exposes) Delivery You’ll give me: 1. The Excel file, one sheet per category, neatly labeled and deduplicated. 2. The script or notebook you used (Python with requests / BeautifulSoup, Scrapy, or a comparable tool is fine) so I c...

Project Overview I have an Excel dataset containing approximately 11,440 licensed plumbing contractors in California (C-36 classification). The dataset includes contractor license information such as business name, city, address, and phone number. I am seeking an experienced data scraping / lead enrichment specialist to locate verified business email addresses associated with these contractors and append them to the spreadsheet. The goal is to identify publicly available contact emails belonging to the contractor’s business and integrate them into the existing dataset. ⸻ Source Dataset The spreadsheet contains approximately 11,440 contractor records and includes fields such as: • License Number • BusinessName • BUS-NAME-2 (DBA name, if present) • FullBus...

I need an automated service that visits eight real-estate platforms every day—, Zillow (FSBO only), , Hubzu, HomePath, HUDHomestore, InvestorLift, and Crexi—pulls both property details and price data, and drops the results into a single clean dataset. Each site has its own search criteria that I will supply (zip codes, status flags, auction dates, etc.). The scraper must respect those individual rules and still be easy for me to adjust later without having to rewrite code—think a clearly labeled config section or similarly simple mechanism. Key expectations • Runs on a schedule without manual intervention and retries gracefully if a site is temporarily unavailable. • Captures all standard listing fields (address, beds, baths, square footage, list/asking ...

I need every piece of publicly available data pulled from three separate websites and packaged so I can work with it offline. The moment the contract is awarded I’ll hand you the URLs; from there I expect you to spin up an automated solution—Python with Scrapy, BeautifulSoup, Selenium, or a comparable stack—that captures all records, follows pagination, and handles any lazy-loaded content. Time is tight: the full job must be wrapped up within 3 calendar days. My total budget is fixed at US $300, so please be sure your approach fits comfortably inside that limit. Deliverables • A clean, well-structured file for each site (CSV or Excel, whichever you prefer) containing every scraped field. • The runnable scripts or notebooks you used, clearly commented so I ...

We are building a high-accuracy OCR system that extracts structured data from scanned documents and images such as invoices, receipts, ID cards, and application forms. This is not just text recognition — the goal is to build a reliable pipeline that converts messy, real-world documents into clean, validated, structured JSON data ready for databases or ERP systems. -Objective Develop an intelligent OCR engine that: Extracts text from scanned PDFs and images (JPG, PNG) Detects document type automatically (invoice, ID card, form, etc.) Identifies key-value pairs (e.g., Name, Date, Total Amount, Invoice No) Handles noisy images (low resolution, skewed, shadows) Returns structured JSON output Achieves high accuracy (≥95% on test dataset) -Technical Scope - Image Preprocessin...

Long-Term Project Idea: AI-Powered Web Scraping + SaaS App for Lead Intelligence - Project Overview We are building a scalable system that: Scrapes structured & unstructured data from thousands of websites Extracts business contact information Enriches and validates data Provides a dashboard for clients Runs continuously as a subscription-based SaaS platform This is not a one-time scraping script — This is a long-term product development project. - Phase 1 – Smart Web Scraping Engine Objective: Build a scalable scraping infrastructure that can: Crawl 50,000+ websites Detect relevant pages (Contact, About, Team, Directory) Extract: Names Emails Phone numbers Social media Company info Handle: Dynamic websites (JavaScript-rendered) Rate limiting Anti-bot ...

I need clean, reliable product-level data pulled from popular e-commerce sites and delivered in a structured format I can feed straight into my analysis pipeline. The focus is on product details only—titles, descriptions, specifications, images, SKU, category tags and anything else that properly describes the item itself. Price information or customer reviews are not essential right now, but building the scraper in a way that could be extended to capture those later would be appreciated. Your crawler should respect , rotate proxies or user-agents as needed to avoid blocks, and export the results as either CSV or JSON; if you can offer both, even better. I’ll point you to the first two target sites once we start, then we can iterate quickly on additional domains that share a si...

I need a clean, one-off scrape of a large online retailer (I’ll share the exact store and URLs once we agree). The job is straightforward: pull three data points for every product I specify—current price, stated delivery time, and offer type (Retail, Seller, or Seller but shipped by Retail). The site employs standard anti-bot measures, so rotating proxies, respectful delays, or headless browsing may be required to keep requests under the radar. Please build the script with Python—BeautifulSoup, Scrapy, Playwright, or any reliable stack is fine—as long as it can be rerun on my side without extra fees or licences. Deliverables • A single Excel workbook.  – One sheet per category or URL I provide  – Columns: Product identifier (ASIN...