2
votes

I have (as far as I can tell) installed all the dependencies for Scrapy correctly. I am running the Scrapy.org, 64 bit version of Python on Windows Vista 64 bit. I have so far installed:

  1. Visual C++ Redistributables (64 bit)
  2. Win32 Open SSL (with the DLL's copied to bin directory -as per the instructions above)
  3. Environment variables added for C:\Python27, C:\Python27\Scripts and openssl-win32\bin
  4. Visual C++ Studio 2008
  5. Pywin32
  6. Zope
  7. Twisted
  8. lxml
  9. PyopenSSL
  10. w3lib

Is this a complete list of what I need to make Scrapy run correctly? So far, when I try running the command:

scrapy startproject myproject

I get the following error in Command Shell:

Microsoft Windows [Version 6.0.6002]
Copyright (c) 2006 Microsoft Corporation.  All rights reserved.

C:\Users\Oleg Salenko>scrapy startproject mrscrap
Traceback (most recent call last):
  File "c:\Python27\Scripts\scrapy", line 3, in <module>
    from scrapy.cmdline import execute
  File "c:\Python27\lib\site-packages\scrapy\cmdline.py", line 9, in <module>
    from scrapy.crawler import CrawlerProcess
  File "c:\Python27\lib\site-packages\scrapy\crawler.py", line 3, in <module>
    from twisted.internet import reactor, defer
  File "c:\Python27\lib\site-packages\twisted\internet\reactor.py", line 38, in
<module>
    from twisted.internet import default
  File "c:\Python27\lib\site-packages\twisted\internet\default.py", line 56, in
<module>
    install = _getInstallFunction(platform)
  File "c:\Python27\lib\site-packages\twisted\internet\default.py", line 50, in
_getInstallFunction
    from twisted.internet.selectreactor import install
  File "c:\Python27\lib\site-packages\twisted\internet\selectreactor.py", line 1
8, in <module>
    from twisted.internet import posixbase
  File "c:\Python27\lib\site-packages\twisted\internet\posixbase.py", line 24, i
n <module>
    from twisted.internet import error, udp, tcp
  File "c:\Python27\lib\site-packages\twisted\internet\tcp.py", line 29, in <mod
ule>
    from twisted.internet._newtls import (
  File "c:\Python27\lib\site-packages\twisted\internet\_newtls.py", line 21, in
<module>
    from twisted.protocols.tls import TLSMemoryBIOFactory, TLSMemoryBIOProtocol
  File "c:\Python27\lib\site-packages\twisted\protocols\tls.py", line 41, in <mo
dule>
    from OpenSSL.SSL import Error, ZeroReturnError, WantReadError
  File "c:\Python27\lib\site-packages\OpenSSL\__init__.py", line 8, in <module>
    from OpenSSL import rand, crypto, SSL
  File "c:\Python27\lib\site-packages\OpenSSL\rand.py", line 11, in <module>
    from OpenSSL._util import (
  File "c:\Python27\lib\site-packages\OpenSSL\_util.py", line 4, in <module>
    binding = Binding()
  File "c:\Python27\lib\site-packages\cryptography\hazmat\bindings\openssl\bindi
ng.py", line 87, in __init__
    self._ensure_ffi_initialized()
  File "c:\Python27\lib\site-packages\cryptography\hazmat\bindings\openssl\bindi
ng.py", line 106, in _ensure_ffi_initialized
    libraries=libraries,
  File "c:\Python27\lib\site-packages\cryptography\hazmat\bindings\utils.py", li
ne 80, in build_ffi
    extra_link_args=extra_link_args,
  File "c:\Python27\lib\site-packages\cffi\api.py", line 341, in verify
    lib = self.verifier.load_library()
  File "c:\Python27\lib\site-packages\cffi\verifier.py", line 75, in load_librar
y
    return self._load_library()
  File "c:\Python27\lib\site-packages\cffi\verifier.py", line 151, in _load_libr
ary
    return self._vengine.load_library()
  File "c:\Python27\lib\site-packages\cffi\vengine_cpy.py", line 138, in load_li
brary
    raise ffiplatform.VerificationError(error)
cffi.ffiplatform.VerificationError: importing 'c:\\Python27\\lib\\site-packages\
\cryptography\\_Cryptography_cffi_444d7397xa22f8491.pyd': DLL load failed: %1 is
 not a valid Win32 application.

C:\Users\Oleg Salenko>

Can anyone see what is causing these errors? I'm fairly new to Python and Scrapy is by far the hardest thing I have tried so far to get working.

Thanks

3
@padraic cunningham any ideas on this? its still the same error i had last night.gdogg371

3 Answers

1
votes

This was resolved by installing OpenSSL 64 bit, instead of 32 bit. A bit of a schoolboy error, but I'm pretty pleased I managed to work out the root cause!

0
votes

I solve it by installing pyOpenSSL 64 bit using the installer from here: http://www.egenix.com/cryptodownload/?file=egenix-pyopenssl-0.13.0_1.0.0g_1.win-amd64-py2.7.msi

Installing OpenSSL 64 bit didn't resolve my problem.

0
votes

I just answered this in another thread, but just in case.

I just managed to solve this just now and for people who might stumble upon this, the problem is you need PyWin32. For windows users:

  1. pip install PyWin32 here, choose the right python version and 32/64bit. Now in cmd:

    pip install c:\users\desktop\pywin32-220.1-cp27-cp27m-win_amd64.whl

  2. Now, install pywin32 system files, run "python.exe Scripts\pywin32_postinstall.py -install" from an elevated command prompt. Run cmd as admin and in the command prompt

    python "C:\Python27\Scripts\pywin32_postinstall.py" -install