Run Application

You can run the Application on Docker with

docker-compose up --build

You can run the Application locally with Python with python -m src.core

The Application will initialize the Database, load the Configuration config.json and start the scraping. (Screenshot)

You can see the incoming scraped in the logs (Scnreenso)

and first results in the mysql database (screenshot)

after scraping, which takes differenz amount of time per vendor, you can see downloading files (screenshot)

Re-Running the Application

By re-running the Application, all Vendors on the schedule will be executed and start scraping sequentially. Only new products will be stored in the database and downloaded.

Known Issues

• Vendors can block your IP
• Vendors can block this Script
• Vendors Website not available, not performant enough
• Vendors Website