Installing cryptography (part of Scrapy) on Mac OS X

I'm leaving this tip as a note for myself, but I hope someone will find it useful too.

If you try to install Scrapy or cryptography on Mac OS X, your pip install will likely fail with the message similar to this:

1
2
3
4
5
6
7
8
9
  1. build/temp.macosx-10.9-x86_64-2.7/_openssl.c:433:10: fatal error: 'openssl/opensslv.h' file not found
  2. #include <openssl/opensslv.h>
  3. ^
  4. 1 error generated.
  5. error: command 'clang' failed with exit status 1

The problem lies within OpenSSL that comes with Mac OS X. We can easily solve the problem in two steps

1. You'll need to install another one from Homebrew at first:

1
  1. brew install openssl

2. Install cryptography using newly installed openssl, like this:

1
  1. env LDFLAGS="-L$(brew --prefix openssl)/lib" CFLAGS="-I$(brew --prefix openssl)/include" pip install cryptography

Now run your Scrapy installation once again and get ready to scrape the World:

1
  1. pip install Scrapy