wescraper可用于爬取公众号文章。
安装
资料:在国内使用 pip 命令
pact install libxml2-devel libxslt-devel
pact install libffi-devel
pact install openssl-devel
pip install scrapy -i https://pypi.tuna.tsinghua.edu.cn/simple
pip install tornado
命令行抓取
python wescraper/scraper.py account xxx > we.json # 查询liriansu和miawu相关的公众号
python wescraper/scraper.py key-day xxx > we.json # 查询liriansu和miawu相关的文章(一天内)
account
key-all
key-day
key-week
key-mon
key-year
#浏览器查看
python wescraper/server.py –port==8080
http://localhost/account/
http://localhost/key-all/
http://localhost/key-day/
http://localhost/key-week/
http://localhost/key-mon/
http://localhost/key-year/