🚀 The Ultimate Multi-Platform AI Universal Scraper AI SaaS Ecosystem

Universal Scraper is a production-ready, full-stack SaaS platform designed for high-performance data extraction. Built with a powerful Python Flask backend and a modern React (Vite) frontend, this platform offers a turnkey solution for launching a professional scraping business.

🤖 Advanced AI & Multi-Platform Support
Unlike simple scripts, this platform features specialized scrapers for major platforms and a Universal Scraper AI engine:

Platform-Specific Scrapers: Dedicated logic for Shopify, WooCommerce, and Salla.
Universal Scraper AI: Uses LLMs to understand and extract data from any website autonomously.
Multi-AI Integration: Supports OpenAI, Google Gemini, Groq, and Anthropic.
🛡️ Professional Stealth & Extraction Tech
Playwright Automation: Uses Playwright with Stealth Mode to bypass bot detection.
Smart Resource Management: Blocks heavy resources (images, fonts, media) to optimize speed and reduce server load.
Proxy & Captcha: Advanced management for rotating proxies and automated captcha solving.
Headless Execution: Runs complex scraping tasks in the background using Celery & Redis.
💼 Complete Business Features (SaaS)
Full Membership System: User registration, profile management, and 2FA (OTP/QR) security.
Payment Gateways: Fully integrated with Stripe and PayPal for subscriptions.
Interactive Dashboards: Modern React-based dashboards for both Users and Admins.
API & Webhooks: Developers can integrate the scraping engine via REST API or receive data via webhooks.
Multi-Format Export: Export data to JSON, CSV, Excel, or sync directly to Google Sheets and WooCommerce.
🛠️ Technical Excellence
Clean Architecture: Modular Python backend and component-based React frontend.
Installation Wizard: Includes a built-in wizard for easy server setup.
Internationalization: Supports multiple languages (English and Arabic included).
Docker Ready: Includes Docker Compose for seamless deployment.

Features

🤖 Next-Gen AI & Extraction Core
RAG-Powered Learning: Advanced Retrieval-Augmented Generation system that learns site structures over time.
Multi-Model AI Support: Integrated with OpenAI (GPT-4), Google Gemini, Anthropic (Claude), and Groq (Llama 3).
Visual Selector & No-Code Mapping: A sophisticated point-and-click interface for custom data field extraction.
Autonomous Platform Detection: Instantly recognizes and adapts to Shopify, WooCommerce, and Salla stores.
Smart Workflow Engine: Build complex, multi-step scraping pipelines with conditional logic.
Price Watch & Monitoring: Automated system to track price changes and trigger alerts.

🛡️ Stealth & Performance Suite
Playwright Stealth Integration: Mimics human behavior to bypass the most advanced anti-bot systems.
Proactive Resource Blocking: Intelligently blocks images, fonts, and media to save bandwidth and boost speed.
Advanced Proxy Rotator: Support for residential and mobile proxies with automatic failover.
Captcha Auto-Solver: Built-in support for multiple Captcha solving services.
Browser Fingerprinting: Randomizes headers and browser signatures to avoid IP flagging.

💼 Full SaaS & White-Labeling
Complete White-Label Panel: Change logos, branding, and colors directly from the Admin UI.
Stripe & PayPal Ready: Full subscription lifecycle management with integrated payment gateways.
Advanced Admin Overview: Monitor system health, user activity, and server logs in real-time.
Subscription Tier System: Create unlimited plans with custom limits on requests and threads.
SEO Management Tools: Built-in tools for admins to manage meta tags and site indexing.

📊 Connectivity & Export Power
Direct Store Syncing: One-click export and sync directly to any WooCommerce store.
Google Sheets Integration: Real-time data pushing to Google Sheets for live reporting.
Telegram Bot Integration: Control the scraper and get task alerts via a dedicated Telegram bot.
Developer API & Webhooks: Full REST API and webhooks for 3rd party integrations.
Bulk Exporting: Support for high-volume exports in JSON, CSV, Excel, and Markdown.

🔒 Security & Technical Excellence
2FA Security: Two-Factor Authentication via OTP and QR codes for maximum account safety.
React 19 Frontend: Built with the latest React version for insane UI performance.
Background Processing: Handles millions of requests using Celery and Redis workers.
Automated Installation Wizard: A step-by-step UI to set up the entire platform on your server.
Translation Ready: Fully supports multiple languages, including English and Arabic.

Requirements
Backend Requirements
Python: 3.10 or higher.
RAM: 4 GB Minimum (8 GB Recommended for production).
Database: SQLite (Dev) or PostgreSQL/MySQL (Production).
Services: Redis Server (Required for Celery tasks).
Operating System: Linux (Ubuntu recommended) or Windows.
Frontend Requirements
Node.js: 18 LTS or higher.
Build Tool: Vite (Included).

Instructions
Backend Setup:
Navigate to /backend and create a virtual environment: python -m venv venv.
Install dependencies: pip install -r requirements.txt.
Install Playwright browsers: playwright install chromium.
Configure your .env file with API keys and DB URL.
Frontend Setup:
Navigate to /frontend and install packages: npm install.
Build the production assets: npm run build.

Launch:
Start Redis server.
Start the Celery worker: celery -A app.tasks worker.
Run the Flask app: python web_dashboard.py.
Wizard: Access the domain in your browser to complete the setup via the Installation Wizard.

Youtube : https://www.youtube.com/watch?v=A-S1VINZTXo