Python爬取新浪微博评论数据，写入csv文件中

提示：扫一扫查出行
【扫一扫了解最新限行尾号】
复制提示

因为新浪微博网页版爬虫比较困难，故采取用手机网页端爬取的方式

操作步骤如下：

1. 网页版登陆新浪微博

2.打开m.weibo.cn

3.查找自己感兴趣的话题，获取对应的数据接口链接

4.获取cookies和headers

# -*- coding: utf-8 -*-
import requests
import csv
import osbase_url = 'https://m.weibo.cn/api/comments/show?id=4131150395559419&page={page}'
cookies = {'Cookie':'xxx'}
headers = {'User-Agent':'xxx'}path = os.getcwd()+"/weibo.csv"
csvfile = open(path, 'a+', encoding='utf-8',newline='')
writer = csv.writer(csvfile)
writer.writerow(('username','source','comment'))for i in range(0,83):
try:
url = base_url.format(page=i)
resp = requests.get(url, headers=headers, cookies=cookies)
jsondata = resp.json() data = jsondata.get('data')
for d in data:
created_at = d.get("created_at")
source = d.get("source")
username = d.get("user").get("screen_name")
comment = d.get("text")
print((username,source,comment))
writer.writerow((username, source, comment))
except:
print('*'*1000)
passcsvfile.close()