一文掌握requests库：Python中的爬虫神器

2024-11-16

这是专栏优秀的第三方库的第4篇原创文章。

大家好，这里是程序员晚枫，正在all in 各种AI项目。

requests 是一个非常流行的 Python HTTP 库，用于发送各种 HTTP 请求。以下是 requests 的一些基本用法：

安装

首先，确保你已经安装了 requests 库。如果没有安装，可以通过以下命令安装：

1	pip install requests

发送 GET 请求

import requests

# 发送 GET 请求
response = requests.get('https://python-office.com/data')

# 检查请求是否成功
if response.status_code == 200:
    data = response.json()  # 将响应内容解析为 JSON
    print(data)
else:
    print('请求失败，状态码：', response.status_code)

发送 POST 请求

import requests

# 发送 POST 请求
payload = {'key1': 'value1', 'key2': 'value2'}
response = requests.post('https://python-office.com/submit', data=payload)

# 检查请求是否成功
if response.status_code == 200:
    result = response.json()
    print(result)
else:
    print('请求失败，状态码：', response.status_code)

发送带有 Headers 的请求

import requests

headers = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.3'
}

response = requests.get('https://python-office.com/data', headers=headers)

if response.status_code == 200:
    print(response.text)
else:
    print('请求失败，状态码：', response.status_code)

发送带有 Cookies 的请求

import requests

cookies = {
    'session_id': '123456789',
    'token': 'abcdefg'
}

response = requests.get('https://python-office.com/data', cookies=cookies)

if response.status_code == 200:
    print(response.text)
else:
    print('请求失败，状态码：', response.status_code)

发送带有认证信息的请求

import requests

auth = ('username', 'password')

response = requests.get('https://python-office.com/protected', auth=auth)

if response.status_code == 200:
    print(response.text)
else:
    print('请求失败，状态码：', response.status_code)

发送文件

import requests

files = {'file': open('report.xls', 'rb')}

response = requests.post('https://python-office.com/upload', files=files)

if response.status_code == 200:
    print('文件上传成功')
else:
    print('文件上传失败，状态码：', response.status_code)

异常处理

import requests
from requests.exceptions import HTTPError

try:
    response = requests.get('https://python-office.com/data')
    response.raise_for_status()  # 如果响应状态码不是 200，将抛出 HTTPError 异常
    print(response.text)
except HTTPError as http_err:
    print(f'HTTP error occurred: {http_err}')
except Exception as err:
    print(f'Other error occurred: {err}')