mirror of
https://github.com/suitenumerique/docs.git
synced 2026-09-23 10:05:10 +02:00
⚡️(backend) optimize media_auth cpu usage
Once the sql queries improved we have still a bottleneck on large concurrent requests on this endpoint. We notive in the profiles generated that lot of time was spent in creating a new s3 client instance on each request. django_storage use a thread local cache for signed and unsigned connection, but using uvicorn we have a new thread for each request, so on each request a new s3 client is generated and it appears to be an expensive operation. To fix this issue, we cache the client and share it accross all the thread and requests.
This commit is contained in:
@@ -73,6 +73,7 @@ from core.tasks.mail import send_ask_for_access_mail
|
||||
from core.utils.analytics import PosthogEventName, posthog_capture
|
||||
from core.utils.dicts import lowercase_keys
|
||||
from core.utils.paths import filter_descendants
|
||||
from core.utils.s3 import get_s3_client
|
||||
from core.utils.s3_response_stream import content_stream
|
||||
from core.utils.treebeard import create_tree_node_with_retry
|
||||
from core.utils.users import users_sharing_documents_with
|
||||
@@ -2043,8 +2044,12 @@ class DocumentViewSet(
|
||||
logger.debug("User '%s' lacks permission for attachment", user)
|
||||
raise drf.exceptions.PermissionDenied()
|
||||
|
||||
# Check if the attachment is ready
|
||||
s3_client = default_storage.connection.meta.client
|
||||
# Check if the attachment is ready. Use the process-global S3 client
|
||||
# (see core.utils.s3): django-storages caches its client per thread, so
|
||||
# relying on default_storage.connection here rebuilds the boto3 client
|
||||
# on every fresh thread -- profiling showed that client construction,
|
||||
# not the DB, dominated this endpoint's CPU under load.
|
||||
s3_client = get_s3_client()
|
||||
bucket_name = default_storage.bucket_name
|
||||
try:
|
||||
head_resp = s3_client.head_object(Bucket=bucket_name, Key=key)
|
||||
|
||||
Reference in New Issue
Block a user