⚡️(backend) optimize media_auth cpu usage

Once the sql queries improved we have still a bottleneck on large
concurrent requests on this endpoint. We notive in the profiles generated
that lot of time was spent in creating a new s3 client instance on each
request. django_storage use a thread local cache for signed and unsigned
connection, but using uvicorn we have a new thread for each request, so
on each request a new s3 client is generated and it appears to be an
expensive operation. To fix this issue, we cache the client and share it
accross all the thread and requests.
This commit is contained in:
Manuel Raynaud
2026-08-20 17:28:31 +02:00
parent 7372c4610f
commit 4111e4e5ed
5 changed files with 65 additions and 4 deletions
+4 -2
View File
@@ -13,6 +13,8 @@ import botocore
from lasuite.oidc_login.decorators import refresh_oidc_access_token
from rest_framework.throttling import BaseThrottle
from core.utils.s3 import get_s3_client, get_unsigned_s3_client
def nest_tree(flat_list, steplen):
"""
@@ -76,14 +78,14 @@ def generate_s3_authorization_headers(key):
- access control is truly realtime
- the object storage service does not need to be exposed on internet
"""
url = default_storage.unsigned_connection.meta.client.generate_presigned_url(
url = get_unsigned_s3_client().generate_presigned_url(
"get_object",
ExpiresIn=0,
Params={"Bucket": default_storage.bucket_name, "Key": key},
)
request = botocore.awsrequest.AWSRequest(method="get", url=url)
s3_client = default_storage.connection.meta.client
s3_client = get_s3_client()
# pylint: disable=protected-access
credentials = s3_client._request_signer._credentials # noqa: SLF001
frozen_credentials = credentials.get_frozen_credentials()