The celery beat configuration was not working correctly, we installed
the database scheduler but we didn't configure it correctly. We prefer
removing this scheduler to use the default one. Instead, the crontab
part can be configured using environment variable. By default it will be
run every night at 3AM.
We want to add documentation showing how to use DS_Proxy with Drive.
With proxy is fully optionnal and is here if you want to an encryption
layer between Drive and the object storage.
Allong the allowed extension list we added an allowed mimetype list. We
combine both to ensure that the user is allowed to make the upload. If
the mimetype is not allowed, then the file and the item are deleted.
We want to limit what type of file can be uploaded. For this we use an
allowed list of file extension. This allowed list can be disabled using
a setting and this list is configurable using a setting. When enabled,
file with no extension and hidden file are also rejected.
Add setting FEATURES_INDEXED_SEARCH that allows to disable the
search of indexed files while using the indexation tools.
Signed-off-by: Fabre Florian <ffabre@hybird.org>
When indexer service is not configured, the search view should work
event with a disabled OIDC_STORE_ACCESS_TOKEN.
Disable token storage for the unit tests.
Add bin/fernetkey that generates a key for the OIDC_STORE_REFRESH_TOKEN_KEY
setting.
Signed-off-by: Fabre Florian <ffabre@hybird.org>
Use SEARCH_INDEXER_CONTENT_MAX_SIZE as limit (in bytes) for the file content.
Fix default configuration of OIDC_STORE_ACCESS_TOKEN
Signed-off-by: Fabre Florian <ffabre@hybird.org>
Use nb_results instead of page/page_size argument for /search API.
Add --batch-size argument to the index command.
Fix an issue in SearchIndexer.has_text when item.mimetype is empty.
Signed-off-by: Fabre Florian <ffabre@hybird.org>
Reduce the number of Find API calls by grouping all the latest changes
for indexation : send all the items updated or deleted since the
triggering of the task.
Signed-off-by: Fabre Florian <ffabre@hybird.org>
Add SearchIndexer service that handles indexation & search API calls to Find
Add SEARCH_INDEXER_* settings to configure it.
Signed-off-by: Fabre Florian <ffabre@hybird.org>
Add a new Django app 'demo' that contains the command 'create_demo'
Generate dummy users and files with existing factories.
Signed-off-by: Fabre Florian <ffabre@hybird.org>
We have some instances that requires more open resource server features.
This change adds a setting that allows to cherry-pick
which api routes needs to be available or not.
The entitlements are computed from an abstract backend in
order to allow for custom implementations, api calls, etc.
Let's implement those in the appropriate spots.
The pagination class Viewsets.Pagination has the max_page_size set to
200. We want to be able to configure this value using a setting. A new
one MAX_PAGE_SIZE is created for this. By default its value is 200 to
not change the current behavior.
We want to distinguish the url and the url_preview. The url_preview
property will be provided only if the mimetype is listed in the
ITEM_PREVIEWABLE_MIME_TYPES settings. Also an other route is added to
nginx forcing the content-disposition with the value attachment for
download url.
Wopi launch url given in the discovery process can have placeholders.
These placeholders are documented in the wopi documentation and we are
able to manage few of them.
https://learn.microsoft.com/en-us/microsoft-365/
cloud-storage-partner-program/online/discovery#placeholder-values
All the info needed to compute the wopisrc url was sent to the front
application and the front application will have to compute it. Instead
of this, everything is made in the back application, the front just have
to use it.
in the wopi protocol, a wopi client should expose a discovery url
allowing the host to configure the url to use based on the extension or
the file mimetype. We use now this discovery url to configure the wopi
host and remove all the existing mechanism made in the settings.
The first method to implement is the CheckFileInfo method. Its
documentation can be consulted with this link:
https://learn.microsoft.com/en-us/microsoft-365/
cloud-storage-partner-program/rest/files/checkfileinfo
In order to start a WOPI lifecycle we have to manage an access token.
This access token will be used by the wopi client to check information
about the file but also doing action like fetching the file content,
saving it etc.
This service is here to generate this access token, manage its lifetime,
manage if the user has access to the item. The info related to the
access token (item and user) are saved in the cache.
We want to implement the wopi protocol. We create a django app dedicated
to manage all the wopi protocol implementation. The first thing made is
to set wopi client configuration.
The file upload to the object storage backend was made using a presigned
post. But not many object storage solution implement this method. They
all used presigned url combined with the put_object method. We choose to
change to this method to be more widely compatible.
Allow configuration variables that handles secrets, like
`DJANGO_SECRET_KEY` to be able to read from a file which is given
through an environment file.
For example, if `DJANGO_SECRET_KEY_FILE` is set to
`/var/lib/meet/django-secret-key`, the value of `DJANGO_SECRET_KEY` will
be the content of `/var/lib/meet/django-secret-key`.
We need a new endpoint for the search feature. Using existing filters on
the list or children endpoint does not match what we want. We want to
search for all accessible items for the current user.
We want to use the malware_detection module from lasuite library. We add
a new setting MALWARE_DETECTION to configure the backend we want to use.
The callback is also added. It removes the file if it is not safe and
update the item upload state accordongly.
We want to split the cache config between the app cache and the session
cache. In the app cache, the default one, we allow to configure a
prefix. By default this prefix is a fixed string so the cache will be
never revoked because it is changing but it allow every instance to
implement its own strategy like prefixing the keyx cache with a
timestamp.
To not impact session, the session cache is splitted in the settings.
We want to be able to login users quickly from the e2e frontend tests.
Because sometimes we may need to reset the DB to run the test, instead
of spending 5sec of going through the login process via keycloak, just
calling this new api route cost 50ms, and voilà. Really great for DX
and tests performances.
We want to customize the theme by using a configuration file. This
configuration file path can be defined using the settings
THEME_CUSTOMIZATION_FILE_PATH. If this file does not exists or is an
invalid json, an empty json object will be added in the config endpoint.
We want to be able to specify which origins are allowed to pull the
event route. This way we introduce SDK_ALLOWED_ORIGINS.
I did not found a clean and easy to maintain way to to do the same
behavior with django-cors-headers which aim to provide a global
behavior accross the API, here we want a special behavior only for
the sdk relay routes.
Also remove the possibility to use the POST route from cross
domain.
We need to be able to provide a custom redirect url when logging in
from the file picker, we want to be redirected to the specific popup
page, not to the default home route.
This route is intended to be used anonymously from any domain in
order to pass data from the file picker popup to the parent window as
the reference to the parent window is lost when logging via ProConnect.
The `BrowsableAPIRenderer` generates a form to test POST/PUT/... actions
and fill the FK fields with unfiltered data. This issue has been spoted
on visio and fixedsuitenumerique/meet#508
The db engine postgresql_psycopg2 does not exists anymore in django but
for BC compat it is possible to use it in the configuration and it is
replace by postgresql at runtime. We changed this settings to use the
good one.
Manuel Raynaud