Scrapy: Unresolved Import Errors

Question

I'm using Python 2.7 with Eclipse. I'm doing a tutorial that builds a basic web scraper with Scrapy. Here is the link.

http://www.youtube.com/watch?v=4fbvkMhvsWY

Before launching the scraper in command prompt I received "unresolved import" errors when attempting the following lines of code:

from scrapy.spider import BaseSpider

from scrapy.selector import HtmlXPathSelector

When I attempt to crawl in command prompt with the following command:

scrapy crawl myfile

I get the error, "Spider not found: myfile".

I also get another unresolved import error in my items.py file. "Field" not only gets the "unresolved import" error, but it also gets the "unused import" error.

code from items.py file:

from scrapy.item import Item, Field

Here is the code from the spider file:

Spider file(named Tutorial1.py)

from scrapy.spider import BaseSpider

from scrapy.selector import HtmlXPathSelector

class Tutorial1 (BaseSpider):
    name="Tutorial1"

    allowed_domains=['http://wikipedia.org']
    start_urls = ["http://en.wikipedia.org/wiki/Home_page",]

    def parse(self, response):
        hxs = HtmlXPathSelector(response)
        print hxs.select('//div/a').extract()

Also when attempting to do other tutorials I experience the same issues leading me to believe that this has something to do with my directory. I'm not sure though.

I've found other individuals are having similar problems.

Scrapy: ImportError: No module named items

Scrapy spider is not working

My system path looks like this:

C:\Python27;C:\Python27\Scripts

I do not get errors when importing the following:

import zope.interface

import twisted

import lxml

import OpenSSL

import scrapy

Please help me figure this out. Thanks in advance.

Talvalin Talvalin · Accepted Answer · 2013-01-12T20:35:32

The name of your spider is the parameter that should be used in your scrapy crawl command. This name is set in your spider code (name = "Tutorial1") so running the command scrapy crawl Tutorial1 should fix the command line problem.

As for the import errors, I've noticed that you're on Windows. Installing scrapy on Windows (7) can be more involved than for other operating systems. This article recommends additionally installing pyopenssl, w3lib and pywin32.

What version of scrapy are you using?

Scrapy: Unresolved Import Errors

1 Answers