A user with at leat editor role, connected can duplicate a file item
which has its upload_state ready. The duplicate_file task is used to do
the copy of the file on the storage.
We plan to add an action to duplicate an item. We want to use a celery
task to make the copy using the s3 client. The task is retried 10 times
if failing and then the item is deleted if the copy fail after 10
retriees.
In order to add in next commit a duplicate action in the ItemViewset, we
add a `duplicate` entry in the item abilities. A user can duplicate a
file if he can update it.
In the item model the properties numchild and numchild_folder must be
maintain by doing a +/- 1 operation in several actions. Maintaining is
error prone, hard to catch and can have lot of root cause we can't
identified. We choose to use annotation instead and to compute the value
when needed.
When a user with priviledged role on an item but inherited he can not
invite external person. We refacto the invitation viewset, serializer
and permission to check roles against all the tree.
The media url is subject to change if the file is renamed. If someone is
using it as a permalink, it will lose the ability to download the file.
We created a new download action, always returning a redirection to the
media url.
Some identity providers might change the case, but in our
products we don't consider case variation to be consider as
different email addresses.
Next step would be to normalize the DB value of email to
be lower-case.
In order to be more fault tolerant, to not lose data if the redis is
reset, etc. We decided to put the item to mirror in a new table and then
the task will manage the status in the database.
This is the first step to then add tasks in the admin to retry or check
why the task has failed and also implement a retry strategy.
The Dockerfile must also be modified, we need the libmagic library to
build the image at the collect stage because the magic module is
imported in the storage module and all tasks are imported when the
application starts.
The user's short_name field length was set to 20. This is not enought
and we have some users who cannot register because of that. We changed
this length to a higher one, 100, like the full_name.
For the same reason we not create an access when a child is created, we
have to not set value to link_reach allowing null value. A root will
have the link_reach set to restricted when created and then every
children are set to null. when the link_configuration will be updated,
descendants will be sync when the new link_reach is higher than the
previous one. Also move scenarios are impacted. When moving to the root,
we force the link_reach to restricted and when moving in an existing
item, we sync it with parents.
Once all accesses filtered, we need to check it we have to change the
max_ancestors_role for accesses that are inherited. An inherited access is will
not have the same item_id than the current item used in this viewset. In that
case, we use the role of the current access and the item_id it is related to.
The ItemAvvessViewset list endpoint is really complicated and half of
the algo is made to cache the role the current user have on the tree.
This code is now removed and replaced by a queryset annotation allowing
to fetch the user_roles and then determine in the model the max role the
user have on the tree. It reduces the number of sql query by 1 and ease
the code comprehension
For a given access we are able to compute the max ancestors role. We
also need the item_id related to this max_ancestors_role. We compute it
both in the vewset and in the models like we already did for the
max_ancestors_role
The set_role_to was computed to have only roles strictly higher than the
max_ancestors_role. We want this behavior only for inherited accesses.
When the role is explicit, we want to allow the lower role in
set_role_to equal to max_ancestors_role
The max_ancestors_role was only computes in the ItemAccessViewSet.list
method to avoid having to compute it multiple times in the list and then
in the serializer. The result is save in the model instance has a cache,
but if not present whe computing the abilities outside the list method,
then the expected abilities are wrong. The set_role_to can't be computed
without it. So we search for the max_ancestors_role in the class itself
if not set.
We need to manage inheritence on item accesses. We backport all the work
made on the docs project. To list accesses, we look for all accesses on
all the item ancestors and the item itself.
Some permissions were made in the BaseAccessSerializer. Permissions
should not occur in serializers. We refactor this, moving permission
check in permissions classes but also in the ItemAccessViewset. Also,
there is no need to keep the BaseAccessPermission
We need to compute the abilities based on ancestors link_reach and
link_role. Gor this we added methods on the Item model computing them
and storing them in a local cache to not compute them again and again in
the same request.
The choices RoleChoices, LinkReachChoices and LinkRoleChoices are moved
in the djano-lasuite project, we can remove them from drive and use them
from the library.
The link reach/role on an item is derived from its ancestors. Pre-computing
values for each item allows passing them in a mapping that avoids repeated
queries when computing each item's abilities.
We can calculate a child's number of accesses by adding the count for its
parent item and adding the count directly attached to it. This addition
can be done in the same list query via an annotation so we don't generate
N+1 queries (N equals 200 by default on the children view).
We want to allow instance to not setup email sending without causing
crashes. Also add the envvar to the test setup so that the existing
tests that were running without DJANGO_EMAIL_HOST to None still
work.
For a given item we want to retrieve its breadcrumb, to create a
breadcrum component in the frontend application. This breadcrumb return
a flat list of items order by their path depth.
A user can be invited and no control is made on the email case. Then,
when a new user is created, we are looking if there are pending
invitation and the lookup used is case sensitive. We change it using
__iexact which is case insensitive.
in the wopi protocol, a wopi client should expose a discovery url
allowing the host to configure the url to use based on the extension or
the file mimetype. We use now this discovery url to configure the wopi
host and remove all the existing mechanism made in the settings.
In order to start a WOPI session, we have to fetch an endpoint on the
item viewset returing the access token, the access token ttl and the
wopi client launch url.
Added the creator id field to ListItemSerializer and ItemSerializer to
enhance data representation and facilitate access to the creator's
identifier in the serialized output.
We want to use the malware_detection module from lasuite library. We add
a new setting MALWARE_DETECTION to configure the backend we want to use.
The callback is also added. It removes the file if it is not safe and
update the item upload state accordongly.
For now it wasn't possible to create twice an item with the same title
in the same path and an exception was raised. We change it by
incrementing a version number in the item title. It is still not
possible to have twice the same title but the item is created with a
different one.
In the deletion process we maintain the numchild property by decreasing
when the item is soft deleted. But the delete method is also decreasing
it. We must only rely on the soft delete process and ensure that an item
can not be deleted if it was not solft_deleted first.
Fixes#261
The check constraint on the filename can be moved in the save method. A
check constraint will execute a sql query and in this case it is not
needed, we have all the element needed to do this check in the python
code and not in the database. This will reduce the number queries on
every save.
The link in the link was not correct, also added french translation
for the sent mails. The language used was not the language of the user,
I replaced with the most up to date approach used on Docs.
Thomas Ramé