示例URL地址
http://httpbin.org/get?a=1&b=2async def fetch(client):async with client.get('http://httpbin.org/get',params='q=aiohttp+python&a=1') as resp:return await resp.text()示例URL地址
http://httpbin.org/get?q=aiohttp+python&a=1请求头aiohttp 在自定义请求头时,类似于向 URL 传递参数的方式
async def fetch(client):headers = {'content-type': 'application/json', 'User-Agent': 'Python/3.7 aiohttp/3.7.2'}async with client.get('http://httpbin.org/get',headers=headers) as resp:return await resp.text()COOKIEScookies 是整个会话共用的,所以应该在初始化 ClientSession 对象时传递
async def fetch(client):async with client.get('http://httpbin.org/get') as resp:return await resp.text()async def main():cookies = {'cookies': 'this is cookies'}async with aiohttp.ClientSession(cookies=cookies) as client:html = await fetch(client)print(html)POST 方式在前面的示例中都是以 GET 方式提交请求,下面用 POST 方式请求
async def fetch(client):data = https://www.isolves.com/it/cxkf/yy/Python/2022-07-11/{'a': '1', 'b': '2'}async with client.post('http://httpbin.org/post', data = data) as resp:return await resp.text()示例结果
{"args": {},"data": "","files": {},"form": {"a": "1","b": "2"},"headers": {"Accept": "*/*","Accept-Encoding": "gzip, deflate","Content-Length": "7","Content-Type": "application/x-www-form-urlencoded","Host": "httpbin.org","User-Agent": "Python/3.7 aiohttp/3.6.2"},"json": null,"origin": "49.80.42.33, 49.80.42.33","url": "https://httpbin.org/post"}aiohttp版爬虫花费时间为:0:00:00.514402在示例结果中可以看到 form 中的内容就是模拟 POST 方式提交的内容
超时在请求网站时,有时会遇到超时问题,aiohttp 中使用 timeout 参数设置,单位为秒数,aiohttp 默认超时时间为5分钟
async def fetch(client):async with client.get('http://httpbin.org/get', timeout=60) as resp:return await resp.text()总结aiohttp 以异步的方式爬取网站耗时远小于 requests 同步方式,这里列举了一些 aiohttp 常用功能,希望对大家有所帮助 。
【Python异步之aiohttp】
推荐阅读
- 大学生|丈母娘中意的“女婿职业”排名,榜首意料之中,公务员没进前3
- 朱之文|朱小伟半个月减30斤,大衣哥奖励百万豪车一辆,网友:溺爱孩子
- 子婴是秦始皇什么人?
- 一鼓作气出自哪个战役?
- 君子之交打一台湾地名?
- 万园之园指的是哪个园?
- 关之琳其实不漂亮,关之琳长得漂亮吗-
- 龙灯之乡是哪里?
- 毫克和克怎么换算?
- 干净文艺的短句有哪些?
