CentOS 6.4 安装搭建 Scrapy 0.22 环境

Scrapy是一个开源的机遇twisted框架的python的单机爬虫，该爬虫实际上包含大多数网页抓取的工具包，用于爬虫下载端以及抽取端。CentOS 6.4 安装搭建 Scrapy 0.22 环境过程记录，希望对大家有所帮助。一、安装Python2.7.61.更新CentOS lib库文件yum -y update2.安装开发工具包yum groupinstall -y development3.安装扩展包yum install -y zlib-dev openssl-devel sqlite-devel bzip2-devel4.下载Python2.7.6wget http://www.python.org/ftp/python/2.7.6/Python-2.7.6.tar.xz5.解压Python2.7.6tar -xvf Python-2.7.6.tar.xz6.进入Python2.7.6目录cd Python-2.7.67.编译安装Python2.7.6./configuremake && make altinstall二、解决Yum不支持Python2.7的问题1.将老的Python程序进行备份mv /usr/bin/python /usr/bin/python2.6.62.建立新的Python链接ln -s /usr/local/bin/python2.7 /usr/bin/python3.修改/usr/bin/yum文件vi /usr/bin/yum将第一行的 #！/usr/bin/python 改为 #！/usr/bin/python2.6.6三、安装setuptools 和 pip1.下载setuptoolswget --no-check-certificate https://pypi.python.org/packages/source/s/setuptools/setuptools-1.4.2.tar.gz2.解压setuptoolstar -xvf setuptools-1.4.2.tar.gz3.进入setuptools目录cd setuptools-1.4.24.通过已安装的Python2.7.6安装setuptoolspython setup.py install5.安装 pipeasy_install pip6.curl https://raw.github.com/pypa/pip/master/contrib/get-pip.py | python2.7 -四、安装libxslt-devel支持lxmlyum install libxslt-devel五、安装Scrapypip install scrapy本文永久更新链接地址