Walmart Multi-Keyword Parallel Scraper

eCommerce Seller, California

Project overview

Multi-window parallel scraper for Walmart seller data across hundreds of keywords simultaneously. Handles CAPTCHA challenges, deduplicates against existing CSV files, and outputs clean structured Excel files per keyword category.

The challenge

Client needed product data for 200+ keyword categories from Walmart but manual collection was impossible. Single-threaded scrapers were too slow and kept getting blocked.

My solution

Built a multi-threaded architecture using undetected-chromedriver with separate browser windows per keyword group. Added intelligent deduplication, CAPTCHA detection and retry logic, and column-preserving CSV merging.

Result

10,000+ products scraped daily across 200+ keywords. Zero duplicates. Clean Excel output ready for analysis.

Tech stack

Python, undetected-chromedriver, Threading, Pandas, Excel, CAPTCHA handling

What was built