Python异步之aiohttp( 二 )

示例URL地址
http://httpbin.org/get?a=1&b=2async def fetch(client):async with client.get('http://httpbin.org/get',params='q=aiohttp+python&a=1') as resp:return await resp.text()示例URL地址
http://httpbin.org/get?q=aiohttp+python&a=1请求头aiohttp 在自定义请求头时,类似于向 URL 传递参数的方式
async def fetch(client):headers = {'content-type': 'application/json', 'User-Agent': 'Python/3.7 aiohttp/3.7.2'}async with client.get('http://httpbin.org/get',headers=headers) as resp:return await resp.text()COOKIEScookies 是整个会话共用的,所以应该在初始化 ClientSession 对象时传递
async def fetch(client):async with client.get('http://httpbin.org/get') as resp:return await resp.text()async def main():cookies = {'cookies': 'this is cookies'}async with aiohttp.ClientSession(cookies=cookies) as client:html = await fetch(client)print(html)POST 方式在前面的示例中都是以 GET 方式提交请求,下面用 POST 方式请求
async def fetch(client):data = https://www.isolves.com/it/cxkf/yy/Python/2022-07-11/{'a': '1', 'b': '2'}async with client.post('http://httpbin.org/post', data = data) as resp:return await resp.text()示例结果
{"args": {},"data": "","files": {},"form": {"a": "1","b": "2"},"headers": {"Accept": "*/*","Accept-Encoding": "gzip, deflate","Content-Length": "7","Content-Type": "application/x-www-form-urlencoded","Host": "httpbin.org","User-Agent": "Python/3.7 aiohttp/3.6.2"},"json": null,"origin": "49.80.42.33, 49.80.42.33","url": "https://httpbin.org/post"}aiohttp版爬虫花费时间为:0:00:00.514402在示例结果中可以看到 form 中的内容就是模拟 POST 方式提交的内容
超时在请求网站时,有时会遇到超时问题,aiohttp 中使用 timeout 参数设置,单位为秒数,aiohttp 默认超时时间为5分钟
async def fetch(client):async with client.get('http://httpbin.org/get', timeout=60) as resp:return await resp.text()总结aiohttp 以异步的方式爬取网站耗时远小于 requests 同步方式,这里列举了一些 aiohttp 常用功能,希望对大家有所帮助 。

【Python异步之aiohttp】


推荐阅读