A robust, simple Python scraper that takes a list of sitemap URLs as an input in a CSV, and returns another CSV with the same sitemaps and all the URLs that have been scraped from them. Only takes <loc> values from each sitemap.
Steps To Use:
Note: Python 3 is needed to run this script, as well as all libraries and packages contained within the script. As of 15 Nov 2023, there is no requirements.txt file created to easily download all packages.
- Download the folder to your computer.
- Open the CSV labelled 'input-urls.csv' and paste all the sitemap URLs that you'd like to scrape.
- Run the script - it is advised to do this with an IDE like VS Code.
- Wait for the script to execute.
- Check the script folder and you'll find an output file with all the data.
Simple as that! Any issues, contact me @ https://www.linkedin.com/in/jpgarbaccio/.