问题简介
对于最终用户（前端）来说，最糟糕的情况是点击某个按钮或进行上传、下载或其他类似操作后的等待。
尤其是当你需要处理巨大的文件时，这简直让人头疼，后端需要保存或发送这些大文件。一些现代框架，例如 FastAPI 和 Flask，提供了一些后台任务库，帮助你后台处理这些任务，不影响正常操作。但这也带来了一个问题：这会使主程序变得更重，因为你将所有重量级任务都移到后台。此外，如果任务失败、出错，或者主程序停止（或崩溃），你可能会丢失这些任务。
解答
那我们到底该干嘛呢？
如果我们能找到一种方法将重负荷操作（比如有时是内存密集，有时是网络密集）单独运行，我们就可以放心地运行我们的程序，无需多虑。
关键词是 Celery。
什么是Celery，我只是从原文复制粘贴了一下
任务队列用于作为一种机制，将工作分配给线程或机器。
任务队列的输入是一段称为任务的工作单元。专用的工作者进程不断监视任务队列，寻找新的工作来执行。
Celery 通过消息进行通信，通常使用消息代理来介于客户端和工作者之间。要启动一个任务，客户端将一条消息添加到队列中，消息代理随后将该消息传递给一个工作者进程。
一个 Celery 系统可以包含多个工作者和消息代理，从而实现高可用性和水平扩展。
Celery 是用 Python 编写的，但协议也可以用任何语言实现。除了 Python 之外，还有为 Node.js 编写的 node-celery 和 node-celery-ts，以及一个 PHP 客户端。
语言互操作性也可以通过暴露一个 HTTP 端点并通过请求该端点的任务来实现（Webhook）。
行动起来，让我们开始写代码。
你可以fork这个仓库，但我还是会继续写。
让我们从头开始创建一个FastAPI项目。我使用的是poetry来进行包管理，我觉得这是最好的包管理工具。如果你还没有安装的话，可以看看这个链接 https://python-poetry.org/，里面有详细的文档。
mkdir celery-app # 创建名为celery-app的目录
cd celery-app # 进入该目录 poetry init
poetry add fastapi, uvicorn, celery, redis, python, pydantic-settings, boto3, python-multipart, logging简单来说，我们需要三样东西：
要发送请求，我们需要创建一个控制器函数，一个用于 S3 实现的服务模块，一个 Celery 配置文件，一个 .env 文件，最后，还需要一个 Celery 任务函数。
这是我们的文件结构，我尽量让它简单；但这不一定是最优的结构，这要视项目而定。
从celery导入Celery
celery_app = Celery(
'worker',
broker='redis://redis:6379/0',
backend='redis://redis:6379/0'
)
celery_app.autodiscover_tasks(['tasks.uploads'], force=True)重要的一点，（tasks.upload）表示你创建的任务函数的位置。
否则， celery 就无法正常工作。
from pydantic_settings import BaseSettings
class Settings(BaseSettings):
AWS_ACCESS_KEY_ID: str
AWS_SECRET_ACCESS_KEY: str
AWS_S3_BUCKET_NAME: str
AWS_REGION: str
class Config:
env_file = ".env"
settings = Settings()我们创建了一个 Settings 类来管理环境配置。
import io
import boto3
from fastapi import Depends
from service.s3.s3_config import S3Config
from service.s3.s3_config import get_s3_config
class S3Service:
"""处理与Amazon S3相关的操作的服务。"""
def __init__(self, config: S3Config):
"""初始化S3客户端并设置存储桶名称。"""
self.client = boto3.client(
's3',
aws_access_key_id=config.aws_access_key_id,
aws_secret_access_key=config.aws_secret_access_key,
region_name=config.region_name
)
self.bucket_name = config.bucket_name
def upload_file(self, file_path: str, file_content: bytes, content_type: str = None) -> str:
"""将文件上传到S3。
参数说明:
file_path (str): 文件在S3存储桶中的存储路径。
file_content (bytes): 要上传的文件内容。
content_type (str, 可选): 文件的内容类型。
返回:
str: 已上传文件的路径。
抛出:
Exception: 如果上传失败。
"""
try:
self.client.upload_fileobj(
io.BytesIO(file_content),
self.bucket_name,
file_path,
ExtraArgs={"ContentType": content_type}
)
return file_path
except Exception as e:
raise Exception(f"上传文件到S3时出错: {e}")
def get_s3_service(config: S3Config = Depends(get_s3_config)) -> S3Service:
"""为S3Service注入依赖。
参数说明:
config (S3Config): S3配置对象。
返回:
S3Service: S3Service的实例。
"""
return S3Service(config)from config.env.env_config import settings
class S3Config:
def __init__(self):
self.aws_access_key_id = settings.AWS_ACCESS_KEY_ID
self.aws_secret_access_key = settings.AWS_SECRET_ACCESS_KEY
self.region_name = settings.AWS_REGION
self.bucket_name = settings.AWS_S3_BUCKET_NAME
def 获取S3配置() -> S3Config:
return S3Config()S3 类和配置用来连接到 S3 桶并上传文件。
import logging
from fastapi import Depends
from config.celery.celery_config import celery_app
from service.s3.s3_config import S3Config
from service.s3.s3_service import S3Service
@celery_app.task
def upload_file_to_s3(file_content: bytes, filename: str, content_type: str):
"""上传文件到S3。
参数：
file_content (bytes): 要上传的文件内容。
filename (str): 要上传的文件名。
content_type (str): 文件类型。
返回：
str: 上传文件的路径。
"""
s3_config = S3Config()
s3_service = S3Service(s3_config)
try:
uploaded_file_path = s3_service.upload_file(filename, file_content, content_type)
return uploaded_file_path
except Exception as e:
logging.error(f"文件上传失败: {e}")
raise这是我们的 Celery 任务，所以它将在队列中运行，并由单独的工作者处理。
# 从tasks.uploads.upload_tasks导入上传文件到S3的函数。
from tasks.uploads.upload_tasks import upload_file_to_s3我们也需要将其添加到名为init.py的文件中，确保celery配置可以识别此函数。
从 fastapi 导入 APIRouter, UploadFile, File, Depends, HTTPException
从 tasks.uploads.upload_tasks 导入 upload_file_to_s3
从 service.s3.s3_service 导入 get_s3_service
从 service.s3.s3_config 导入 get_s3_config, S3Config
导入 logging
router = APIRouter()
@router.post("/upload/")
async def upload_file(file: UploadFile = File(...)):
"""上传一个文件并触发一个 Celery 任务来处理它."""
file_content = await file.read()
try:
uploaded_file_path = upload_file_to_s3.delay(file_content, file.filename, file.content_type)
返回 {"message": "文件上传成功！", "task_id": uploaded_file_path.id}
except Exception as e:
logging.error(f"文件上传期间出错: {e}")
抛出 HTTPException(status_code=500, detail='文件上传失败')请求发送控制器。
我正在用路由器，所以我们需要在 main.py 文件里加上 include_router 这一行。
从fastapi导入FastAPI模块
从controller.s3导入s3_controller模块
app = FastAPI()
app.include_router(s3_controller.router, prefix="/api/content", tags=["文件上传标签"])最后，我们来创建 Dockerfile 和 docker-compose 文件，这样我们就可以把系统放进容器里运行了。
services:
fastapi:
build: .
container_name: fastapi_app
command: uvicorn main:app --host 0.0.0.0 --port 8000 --reload
# 命令：启动uvicorn服务，监听所有网络接口的8000端口，并开启热重载
volumes:
- .:/app
ports:
- "8000:8000"
depends_on:
- redis
- celery_worker
# 快速API服务
redis:
image: redis:6.2
container_name: redis
ports:
- "6379:6379"
# Redis服务
celery_worker:
build: .
container_name: celery_worker
command: celery -A config.celery.celery_config.celery_app worker --loglevel=DEBUG
# 命令：启动Celery工作节点，加载配置文件中的Celery应用并设置日志级别为DEBUG
volumes:
- .:/app
depends_on:
- redis
# Celery工作节点从python:3.9
工作目录设置为/app
复制pyproject.toml和poetry.lock文件到/app目录下
运行pip install poetry
运行poetry config virtualenvs.create false && poetry install --no-dev
复制当前目录下的所有文件到/app目录
命令设置为运行 uvicorn main:app --host 0.0.0.0 --port 8000让我们启动应用吧。
docker-compose up --build：启动并构建服务然后发送一下请求吧。
查看控制台，你应该看到类似下面的输出，
fastapi_app | 信息: 192.168.97.1:58732 - "POST /api/content/upload/ HTTP/1.1" 200 OK
celery_worker | [2024-11-04 10:21:33,495: 信息/MainProcess] 任务 tasks.uploads.upload_tasks.upload_file_to_s3[44368494-d85d-4a92-a399-511915e9b36a] 收到
celery_worker | [2024-11-04 10:21:33,505: 调试/MainProcess] 任务池: 执行 <function fast_trace_task at 0xffffbbcb2d30> 参数:('tasks.uploads.upload_tasks.upload_file_to_s3', '44368494-d85d-4a92-a399-511915e9b36a', {'lang': 'py', 'task': 'tasks.uploads.upload_tasks.upload_file_to_s3', 'id': '44368494-d85d-4a92-a399-511915e9b36a', 'shadow': None, 'eta': None, 'expires': None, 'group': None, 'group_index': None, 'retries': 0, 'timelimit': [None, None], 'root_id': '44368494-d85d-4a92-a399-511915e9b36a', 'parent_id': None, 'argsrepr':... kwargs:{})
celery_worker | [2024-11-04 10:21:33,535: 调试/ForkPoolWorker-8] 事件 choose-service-name: 调用处理程序 <function handle_service_name_alias at 0xffffbadad700>
celery_worker | [2024-11-04 10:21:33,550: 调试/ForkPoolWorker-8] 事件 creating-client-class.s3: 调用处理程序 <function add_generate_presigned_post at 0xffffbab8e160>
celery_worker | [2024-11-04 10:21:33,550: 调试/ForkPoolWorker-8] 事件 creating-client-class.s3: 调用处理程序 <function lazy_call.<locals>._handler at 0xffffba331940>
celery_worker | [2024-11-04 10:21:33,551: 调试/ForkPoolWorker-8] 事件 creating-client-class.s3: 调用处理程序 <function add_generate_presigned_url at 0xffffbab8cee0>
celery_worker | [2024-11-04 10:21:33,553: 调试/ForkPoolWorker-8] 查找 s3 端点通过 environment_service
celery_worker | [2024-11-04 10:21:33,553: 调试/ForkPoolWorker-8] 查找 s3 端点通过 environment_global
celery_worker | [2024-11-04 10:21:33,553: 调试/ForkPoolWorker-8] 查找 s3 端点通过 config_service
celery_worker | [2024-11-04 10:21:33,553: 调试/ForkPoolWorker-8] 查找 s3 端点通过 config_global
celery_worker | [2024-11-04 10:21:33,554: 调试/ForkPoolWorker-8] 未找到配置端点
celery_worker | [2024-11-04 10:21:33,558: 调试/ForkPoolWorker-8] 设置 s3 超时为 (60, 60)
celery_worker | [2024-11-04 10:21:33,581: 调试/ForkPoolWorker-8] 为服务 s3 注册重试处理程序
celery_worker | [2024-11-04 10:21:33,581: 调试/ForkPoolWorker-8] 注册 S3 区域重定向处理程序
celery_worker | [2024-11-04 10:21:33,582: 调试/ForkPoolWorker-8] 注册 S3Express 身份解析器
celery_worker | [2024-11-04 10:21:33,582: 调试/ForkPoolWorker-8] 选择不使用 CRT 转移管理器。首选客户端: auto, CRT 可用: False, 实例优化: False.
celery_worker | [2024-11-04 10:21:33,582: 调试/ForkPoolWorker-8] 使用默认客户端。进程 ID: 15, 线程 ID: 281473870299168
celery_worker | [2024-11-04 10:21:33,584: 调试/ForkPoolWorker-8] 获取 0
celery_worker | [2024-11-04 10:21:33,589: 调试/ForkPoolWorker-8] UploadSubmissionTask(transfer_id=0, {'transfer_future': <s3transfer.futures.TransferFuture object at 0xffffba5e6c10>}) 即将等待如下未来 []
celery_worker | [2024-11-04 10:21:33,590: 调试/ForkPoolWorker-8] UploadSubmissionTask(transfer_id=0, {'transfer_future': <s3transfer.futures.TransferFuture object at 0xffffba5e6c10>}) 等待依赖任务完成
celery_worker | [2024-11-04 10:21:33,590: 调试/ForkPoolWorker-8] 执行任务 UploadSubmissionTask(transfer_id=0, {'transfer_future': <s3transfer.futures.TransferFuture object at 0xffffba5e6c10>}) 参数 {'client': <botocore.client.S3 object at 0xffffb9a801f0>, 'config': <boto3.s3.transfer.TransferConfig object at 0xffffba553550>, 'osutil': <s3transfer.utils.OSUtils object at 0xffffbaaaf9a0>, 'request_executor': <s3transfer.futures.BoundedExecutor object at 0xffffbaaafd90>, 'transfer_future': <s3transfer.futures.TransferFuture object at 0xffffba5e6c10>}
celery_worker | [2024-11-04 10:21:33,590: 调试/ForkPoolWorker-8] 提交任务 PutObjectTask(transfer_id=0, {'bucket': '', 'key': 'INVOICE.pdf', 'extra_args': {'ContentType': 'application/pdf'}}) 到执行器 <s3transfer.futures.BoundedExecutor object at 0xffffbaaafd90> 以请求传输请求: 0.
celery_worker | [2024-11-04 10:21:33,590: 调试/ForkPoolWorker-8] 获取 0
celery_worker | [2024-11-04 10:21:33,592: 调试/ForkPoolWorker-8] PutObjectTask(transfer_id=0, {'bucket': '', 'key': 'INVOICE.pdf', 'extra_args': {'ContentType': 'application/pdf'}}) 即将等待如下未来 []
celery_worker | [2024-11-04 10:21:33,593: 调试/ForkPoolWorker-8] PutObjectTask(transfer_id=0, {'bucket': '', 'key': 'INVOICE.pdf', 'extra_args': {'ContentType': 'application/pdf'}}) 等待依赖任务完成
celery_worker | [2024-11-04 10:21:33,593: 调试/ForkPoolWorker-8] 执行任务 PutObjectTask(transfer_id=0, {'bucket': '', 'key': 'INVOICE.pdf', 'extra_args': {'ContentType': 'application/pdf'}}) 参数 {'client': <botocore.client.S3 object at 0xffffb9a801f0>, 'fileobj': <s3transfer.utils.ReadFileChunk object at 0xffffbadd28b0>, 'bucket': '', 'key': 'INVOICE.pdf', 'extra_args': {'ContentType': 'application/pdf'}}
celery_worker | [2024-11-04 10:21:33,593: 调试/ForkPoolWorker-8] 事件 before-parameter-build.s3.PutObject: 调用处理程序 <function validate_ascii_metadata at 0xffffbaac40d0>
celery_worker | [2024-11-04 10:21:33,594: 调试/ForkPoolWorker-8] 释放获取 0/None
celery_worker | [2024-11-04 10:21:33,594: 调试/ForkPoolWorker-8] 事件 before-parameter-build.s3.PutObject: 调用处理程序 <function sse_md5 at 0xffffbaac54c0>
celery_worker | [2024-11-04 10:21:33,594: 调试/ForkPoolWorker-8] 事件 before-parameter-build.s3.PutObject: 调用处理程序 <function convert_body_to_file_like_object at 0xffffbaac49d0>
celery_worker | [2024-11-04 10:21:33,594: 调试/ForkPoolWorker-8] 事件 before-parameter-build.s3.PutObject: 调用处理程序 <function validate_bucket_name at 0xffffbaac5430>
celery_worker | [2024-11-04 10:21:33,594: 调试/ForkPoolWorker-8] 事件 before-parameter-build.s3.PutObject: 调用处理程序 <function remove_bucket_from_url_paths_from_model at 0xffffbaac3310>
celery_worker | [2024-11-04 10:21:33,594: 调试/ForkPoolWorker-8] 事件 before-parameter-build.s3.PutObject: 调用处理程序 <bound method S3RegionRedirectorv2.annotate_request_context of <botocore.utils.S3RegionRedirectorv2 object at 0xffffbaaaf8b0>>
celery_worker | [2024-11-04 10:21:33,594: 调试/ForkPoolWorker-8] 事件 before-parameter-build.s3.PutObject: 调用处理程序 <bound method ClientCreator._inject_s3_input_parameters of <botocore.client.ClientCreator object at 0xffffba6883a0>>
celery_worker | [2024-11-04 10:21:33,594: 调试/ForkPoolWorker-8] 事件 before-parameter-build.s3.PutObject: 调用处理程序 <function generate_idempotent_uuid at 0xffffbaac5280>
celery_worker | [2024-11-04 10:21:33,595: 调试/ForkPoolWorker-8] 事件 before-endpoint-resolution.s3: 调用处理程序 <function customize_endpoint_resolver_builtins at 0xffffbaac34c0>
celery_worker | [2024-11-04 10:21:33,595: 调试/ForkPoolWorker-8] 事件 before-endpoint-resolution.s3: 调用处理程序 <bound method S3RegionRedirectorv2.redirect_from_cache of <botocore.utils.S3RegionRedirectorv2 object at 0xffffbaaaf8b0>>
celery_worker | [2024-11-04 10:21:33,595: 调试/ForkPoolWorker-8] 使用提供者调用端点参数: {'Bucket': '', 'Region': '', 'UseFIPS': False, 'UseDualStack': False, 'ForcePathStyle': False, 'Accelerate': False, 'UseGlobalEndpoint': False, 'Key': 'INVOICE.pdf', 'DisableMultiRegionAccessPoints': False, 'UseArnRegion': True}
celery_worker | [2024-11-04 10:21:33,595: 调试/ForkPoolWorker-8] 端点提供者结果:
celery_worker | [2024-11-04 10:21:33,595: 调试/ForkPoolWorker-8] 从端点提供者的 auth schemes 列表中选择 "sigv4"。用户选择的 auth scheme 是: "None"
celery_worker | [2024-11-04 10:21:33,595: 调试/ForkPoolWorker-8] 选择 auth 类型 "v4" 为 "v4"，带有签名上下文参数: {'region': ', 'signing_name': 's3', 'disableDoubleEncoding': True}
celery_worker | [2024-11-04 10:21:33,598: 调试/ForkPoolWorker-8] 事件 before-call.s3.PutObject: 调用处理程序 <function conditionally_calculate_checksum at 0xffffbac03dc0>
celery_worker | [2024-11-04 10:21:33,601: 调试/ForkPoolWorker-8] 事件 before-call.s3.PutObject: 调用处理程序 <function add_expect_header at 0xffffbaac5790>
celery_worker | [2024-11-04 10:21:33,601: 调试/ForkPoolWorker-8] 为请求添加 expect 100 continue header。
celery_worker | [2024-11-04 10:21:33,601: 调试/ForkPoolWorker-8] 事件 before-call.s3.PutObject: 调用处理程序 <bound method S3ExpressIdentityResolver.apply_signing_cache_key of <botocore.utils.S3ExpressIdentityResolver object at 0xffffbaaaf3a0>>
celery_worker | [2024-11-04 10:21:33,601: 调试/ForkPoolWorker-8] 事件 before-call.s3.PutObject: 调用处理程序 <function add_recursion_detection_header at 0xffffbaac7e50>
celery_worker | [2024-11-04 10:21:33,601: 调试/ForkPoolWorker-8] 事件 before-call.s3.PutObject: 调用处理程序 <function inject_api_version_header_if_needed at 0xffffbaac4af0>
celery_worker | [2024-11-04 10:21:33,601: 调试/ForkPoolWorker-8] 发送请求给 OperationModel(name=PutObject) 参数: {'url_path': '/INVOICE.pdf', 'query_string': {}, 'method': 'PUT', 'headers': {'Content-Type': 'application/pdf', 'User-Agent': 'Boto3/1.35.54 md/Botocore#1.35.54 ua/2.0 os/linux#6.10.12-orbstack-00282-gd1783374c25e md/arch#aarch64 lang/python#3.9.20 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.35.54', 'Content-MD5': 'ia1vZMnLrvWuEx8LzM8CDQ==', 'Expect': '100-continue'}, 'body': <s3transfer.utils.ReadFileChunk object at 0xffffbadd28b0>, 'auth_path': '//INVOICE.pdf', 'url': '/INVOICE.pdf', 'context': {'client_region': '', 'client_config': <botocore.config.Config object at 0xffffb9a80340>, 'has_streaming_input': True, 'auth_type': 'v4', 'unsigned_payload': None, 's3_redirect': {'redirected': False, 'bucket': '', 'params': {'Bucket': '', 'Key': 'INVOICE.pdf', 'Body': <s3transfer.utils.ReadFileChunk object at 0xffffbadd28b0>, 'ContentType': 'application/pdf'}}, 'input_params': {'Bucket': '', 'Key': 'INVOICE.pdf'}, 'signing': {'region': '', 'signing_name': 's3', 'disableDoubleEncoding': True}, 'endpoint_properties': {'authSchemes': [{'disableDoubleEncoding': True, 'name': 'sigv4', 'signingName': 's3', 'signingRegion': 'eu-north-1'}]}}}
celery_worker | [2024-11-04 10:21:33,602: 调试/ForkPoolWorker-8] 事件 request-created.s3.PutObject: 调用处理程序 <function signal_not_transferring at 0xffffb9ab2940>
celery_worker | [2024-11-04 10:21:33,602: 调试/ForkPoolWorker-8] 事件 request-created.s3.PutObject: 调用处理程序 <bound method RequestSigner.handler of <botocore.signers.RequestSigner object at 0xffffb9a802b0>>
celery_worker | [2024-11-04 10:21:33,602: 调试/ForkPoolWorker-8] 事件 choose-signer.s3.PutObject: 调用处理程序 <function set_operation_specific_signer at 0xffffbaac50d0>
celery_worker | [2024-11-04 10:21:33,602: 调试/ForkPoolWorker-8] 事件 before-sign.s3.PutObject: 调用处理程序 <function remove_arn_from_signing_path at 0xffffbaac3430>
celery_worker | [2024-11-04 10:21:33,602: 调试/ForkPoolWorker-8] 事件 before-sign.s3.PutObject: 调用处理程序 <bound method S3ExpressIdentityResolver.resolve_s3express_identity of <botocore.utils.S3ExpressIdentityResolver object at 0xffffbaaaf3a0>>
celery_worker | [2024-11-04 10:21:33,602: 调试/ForkPoolWorker-8] 使用 v4 auth 计算签名。
celery_worker | [2024-11-04 10:21:33,602: 调试/ForkPoolWorker-8] CanonicalRequest:
celery_worker | PUT
celery_worker | /INVOICE.pdf
celery_worker |
celery_worker | content-md5:ia1vZMnLrvWuEx8LzM8CDQ==
celery_worker | content-type:application/pdf
celery_worker | host:.amazonaws.com
celery_worker | x-amz-content-sha256:UNSIGNED-PAYLOAD
celery_worker | x-amz-date:20241104T102133Z
celery_worker |
celery_worker | content-md5;content-type;host;x-amz-content-sha256;x-amz-date
celery_worker | UNSIGNED-PAYLOAD
celery_worker | [2024-11-04 10:21:33,602: 调试/ForkPoolWorker-8] StringToSign:
celery_worker | AWS4-HMAC-SHA256
celery_worker | 20241104T102133Z
celery_worker | 20241104/eu-north-1/s3/aws4_request
celery_worker | b3a6aa06ac572dc376aa47aea1987a0ee93724103bead2c9dedb636abc322e6e
celery_worker | [2024-11-04 10:21:33,603: 调试/ForkPoolWorker-8] Signature:
celery_worker | eafd210245ac82fa88e83fc2f637772c6c2962e76fcb0baa2ffd2ea91df3e550
celery_worker | [2024-11-04 10:21:33,603: 调试/ForkPoolWorker-8] 事件 request-created.s3.PutObject: 调用处理程序 <function signal_transferring at 0xffffb9ab29d0>
celery_worker | [2024-11-04 10:21:33,603: 调试/ForkPoolWorker-8] 事件 request-created.s3.PutObject: 调用处理程序 <function add_retry_headers at 0xffffbaac3280>
celery_worker | [2024-11-04 10:21:33,603: 调试/ForkPoolWorker-8] 发送 HTTP 请求: <AWSPreparedRequest stream_output=False, method=PUT, url=https://north-1.amazonaws.com/INVOICE.pdf, headers={'Content-Type': b'application/pdf', 'User-Agent': b'Boto3/1.35.54 md/Botocore#1.35.54 ua/2.0 os/linux#6.10.12-orbstack-00282-gd1783374c25e md/arch#aarch64 lang/python#3.9.20 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.35.54', 'Content-MD5': b'ia1vZMnLrvWuEx8LzM8CDQ==', 'Expect': b'100-continue', 'X-Amz-Date': b'20241104T102133Z', 'X-Amz-Content-SHA256': b'UNSIGNED-PAYLOAD', 'Authorization': b'AWS4-HMAC-SHA256 Credential=AKIAZQ3DUZRN2FHZ2A6K/20241104/eu-north-1/s3/aws4_request, SignedHeaders=content-md5;content-type;host;x-amz-content-sha256;x-amz-date, Signature=eafd210245ac82fa88e83fc2f637772c6c2962e76fcb0baa2ffd2ea91df3e550', 'amz-sdk-invocation-id': b'445175a4-f8a3-47d6-acde-daabfb644591', 'amz-sdk-request': b'attempt=1', 'Content-Length': '549743'}>
celery_worker | [2024-11-04 10:21:33,604: 调试/ForkPoolWorker-8] 证书路径: /usr/local/lib/python3.9/site-packages/certifi/cacert.pem
celery_worker | [2024-11-04 10:21:33,605: 调试/ForkPoolWorker-8] 新建 HTTPS 连接 (1): 1.amazonaws.com:443
celery_worker | [2024-11-04 10:21:33,856: 调试/ForkPoolWorker-8] 等待 100 Continue 响应。
celery_worker | [2024-11-04 10:21:33,957: 调试/ForkPoolWorker-8] 见到 100 Continue 响应，现在发送请求体。
celery_worker | [2024-11-04 10:21:34,481: 调试/ForkPoolWorker-8] https://amazonaws.com:443 "PUT /INVOICE.pdf HTTP/1.1" 200 0
celery_worker | [2024-11-04 10:21:34,482: 调试/ForkPoolWorker-8] 事件 before-parse.s3.PutObject: 调用处理程序 <function _handle_200_error at 0xffffbaac3790>
celery_worker | [2024-11-04 10:21:34,482: 调试/ForkPoolWorker-8] 事件 before-parse.s3.PutObject: 调用处理程序 <function handle_expires_header at 0xffffbaac35e0>
celery_worker | [2024-11-04 10:21:34,482: 调试/ForkPoolWorker-8] 响应头部: {'x-amz-id-2': 'rtnQgYRDKADz4t/MqdRPLtXV/YOD1N1evlGBU3987M5kgSh7Noqbuv4ntF9HW5tFcFTs/HdfiIE=', 'x-amz-request-id': '62GMVRASSY9DEVMA', 'Date': 'Mon, 04 Nov 2024 10:21:34 GMT', 'x-amz-server-side-encryption': 'AES256', 'ETag': '"89ad6f64c9cbaef5ae131f0bcccf020d"', 'Server': 'AmazonS3', 'Content-Length': '0'}
celery_worker | [2024-11-04 10:21:34,482: 调试/ForkPoolWorker-8] 响应体:
celery_worker | b''
celery_worker | [2024-11-04 10:21:34,484: 调试/ForkPoolWorker-8] 事件 needs-retry.s3.PutObject: 调用处理程序 <function _update_status_code at 0xffffbaac38b0>
celery_worker | [2024-11-04 10:21:34,484: 调试/ForkPoolWorker-8] 事件 needs-retry.s3.PutObject: 调用处理程序 <botocore.retryhandler.RetryHandler object at 0xffffb9b3ae20>
celery_worker | [2024-11-04 10:21:34,484: 调试/ForkPoolWorker-8] 不需要重试。
celery_worker | [2024-11-04 10:21:34,484: 调试/ForkPoolWorker-8] 事件 needs-retry.s3.PutObject: 调用处理程序 <bound method S3RegionRedirectorv2.redirect_from_error of <botocore.utils.S3RegionRedirectorv2 object at 0xffffbaaaf8b0>>
celery_worker | [2024-11-04 10:21:34,485: 调试/ForkPoolWorker-8] 释放获取 0/None
celery_worker | [2024-11-04 10:21:34,493: 信息/ForkPoolWorker-8] 任务 tasks.uploads.upload_tasks.upload_file_to_s3[44368494-d85d-4a92-a399-511915e9b36a] 在 0.9695482780007296s 完成: 'INVOICE.pdf' 祝你好运哦
随时随地看视频
