pywikiscrape is a user-friendly but powerful open-source Wikipedia scraper written in Python that lets users export articles to an sqlite3 database using Beautiful Soup.
Currently, the only way to use pywikiscrape is to clone the Github repository and manually run the script. To do so, follow this code block. You must ensure that you have all required dependencies available to your Python interpreter.
git clone https://github.com/tyleruploads/pywikiscrape.git
cd pywikiscrape
pip install -r requirements.txt # Installs requirements — Linux users: Manage the dependencies yourself; I was able to, so you should too
python3 src/main.pyThe script saves its output to a sqlite3 database with the name of pywikiscrape.db. You do not need to pass any arguements to the script.
Contributions are what make open-source projects one of a kind. All contributions are highly appreciated.
- Found a bug or issue: Open an Issue and show the output of the script, the steps to reproduce it, and as much information as possible
- Have an idea: Open an Issue and explain your idea as much as possible, why you think it would be a good addition to the project, and any other important information.
This project is licensed under the MIT License - see the LICENSE file for more information.