Scrapy for Beginners

How to install Scrapy

Introduction-

Scrapy is a new Python package that aims at easy, fast, and automated web crawling, which recently gained much popularity. The Scrapy framework for building web-crawling software in Python is extremely easy to use.
It offers great flexibility in terms of functionality, as well as solid performance, programmatically going over a collection of web pages and extracting data using APIs or as a general purpose web crawler and powerful tool for working with data on the web. Using Scrapy is not a difficult task, but installing it can be exceedingly challenging, potentially even a barrier to many users.

Installation steps:

1.First of all open “command prompt” and after that run the command “pip install scrapy” which has been shown in the figure.

Step 1

2.If error occurs which has been shown in the above figure then download VCForPython27 from official website “https://www.microsoft.com/en-in/download/details.aspx?id=44266” and install VCForPython27.

Step 2

3. After that again go to the “command prompt” and install scrapy using “pip install scrapy”.

Step 3

4.After installation of scrapy open “command prompt” go to the folder where we want to create project as shown in the figure.

Step 4

To create project in scrapy use following command
1. For creating project run command “scrapy startproject testproject” in the “command prompt” . This will create a tutorial directory with the following contents:

Step 5

            • __init__.py
          • __init__.py
          • items.py
          • middlewares.py
          • pipelines.py
          • settings.py

for more details of the project visit “https://doc.scrapy.org/en/0.24/intro/tutorial.html#creating-a-project”