Change Log
All notable changes to this project are documented in this file. This project adheres to Semantic Versioning.
Unreleased
Added
Add
BackgroundTask
model to enable support for observable long running backend tasksAdd
BackgroundTaskConsumer
class to handle starting background jobs
Changed
BACKWARD INCOMPATIBLE Remove the duplicated
collection
subfield from theentity
field when serializingData
objectsBACKWARD INCOMPATIBLE: Replace
duplicate
methods onData
,Entity
and
Collection
objects with the call to the corresponding background duplicate job.
BACKWARD INCOMPATIBLE: The
duplicate
endpoints onData
,Entity
andCollection
return the details of theBackgroundTask
instance that is used to track the duplication statusWhen subscribing to an object for notifications the
CREATE
notifications are also sent along. This is necessary to receive CREATE notifications triggered from the data inside the container on the container.Remove the redundant prefetch in the
BaseCollectionViewSet
Annotate the queryset in the
BaseCollectionViewSet
withdata_statuses
,entity_count
anddata_count
attributes to reduce the number of queriesRemove the redundant prefetch in the
EntityViewSet
Annotate the queryset in the
EntityViewSet
withdata_statuses
and anddata_count
attributes to reduce the number of queriesModify the
DataViewSet
to remove redundant queriesAdd
duplicate_data
,duplicate_entity
andduplicate_collection
methods toBackgroundTaskConsumer
classSet
is_active
toTrue
for all available processes every timeregister
management command is run even if the same version of the process already exists in the database
Fixed
Send observer notifications when data object is automatically assigned to the entity via
entity_type
in the processRemove observer from subscription if object is deleted
Handle empty iterable in
observe_instance_container
methodRemove call to
close
in observers consumer that causedRuntimeError
33.0.0 - 2023-03-13
Added
Add
suppress_observer_notifications
andmove_to_container
decorators to observersAdd types
ChannelsMessage
andWebsocketMessage
describing type of observers messagesAdd property
containers
on thePermissionObject
that returns all the containers for the given instanceAdd
observe_instance_containers
method to observer models
Changed
BACKWARD INCOMPATIBLE: Drop support for
Python 3.6
,Python 3.7
,Python 3.8
andPython 3.9
. The processing container continues to supportPython >= 3.6
.Add
source
attribute to observer notificationsWhen subscribing to observer endpoint without explicit ids subscribe to CREATE and DELETE notifications by default
Send
CREATE
andDELETE
notifications to the containers of the given object when permissions to it are gained/lost or it is moved between containers. The notifications also contain thesource
andsource_id
context containing content type and id of the object that triggered the change.
32.1.0 - 2023-02-13
Added
Add new entity annotation framework
Add set_permission method on PermissionQuerySet
Add notify_create method on Subscription model in observers to enable sending notification when object is created
Allow ordering data endpoint by entity name
Add
with_superusers
argument tousers_with_permission
method
Changed
When slug collision occurs in the listener when creating new objects retry up to ten times before raising the exception
Enable overriding user and group id of the processing container with environmental variable
Fixed
Notify user when object (data, entity) is created in the container
Do not delete observers in use by other subscriptions when unsubscribing
Notify subscribers to the collection if object inside them is modified
Notify superusers without explicit permissions when object is created
32.0.0 - 2022-11-14
Changed
BACKWARD INCOMPATIBLE: Drop support for Python 3.4 in the processing container
BACKWARD INCOMPATIBLE: Rewrite the listener to allow more than one of them to run at the same time
Rewrite the processing container code to make it more stable
Rewrite the commnication container code to make it more stable
Add
resolve_url
call to there_import.sh
bash script
Changed
Migrate base docker images to Ubuntu 22.04 and Fedora 36
31.3.1 - 2022-10-19
Fix
Make sure signals are triggered for data objects when their sample is moved to a new collection
31.3.0 - 2022-10-17
Added
Add handler
resolve_url
to listenerAdd support for resolving urls in Python processes
Add
url
method to theBaseConnector
class and override it inS3
andlocal
connectors
31.2.1 - 2022-09-20
Changed
Add
cleanup_callback
toretry
decoratorClean
kubernetes
temporary files with credentials on config load error
31.2.0 - 2022-09-19
Added
Add
descriptor
anddescriptor_schema
fields to theRelation
modelSetting
KUBERNETES_DISPATCHER_CONFIG_LOCATION
specifying the location of the kubernetes config to load in theKubernetes
workload connector
Changed
Retry loading
Kubernetes
configuration couple of times before giving up in the kubernetes workload connectorAdd model observers that notify clients about model changes via a websocket connection
31.1.0 - 2022-08-23
Fixed
Clear
custom_messages
array on auditlog reset call
31.0.0 - 2022-07-18
Changed
BACKWARD INCOMPATIBLE: Always try to load kubernetes configuration via
load_kube_config
befofe falling back toload_incluster_config
Rename
docker-compose.yml
tocompose.yaml
Listener can always modify the data object that it is processing
Add original objects to
post_duplicate
callInstall
asgiref
version based on the version of the installedPython
interpreter
Fixed
Fix a typo in
listener
permission handling when creating data model: a check was performed on the wrong object type
Added
Make requests and limits for the
communication
container configurable
30.3.0 - 2022-06-13
Added
Support custom user model in Python processes
30.2.0 - 2022-05-16
Added
Send custom signal
post_duplicate
when objects are duplicated since regular Django signals are not sentAdd
auditlog
application to log user actions
30.1.0 - 2022-04-15
Added
Custom signal
resolwe.flow.signals.before_processing
is sent when data object is ready for processingSupport setting
descriptor
andDescriptorSchema
on Data during process runtimeSupport filtering Data, Entity and Collections based on permissions (view, edit, owner, group, shared_with_me)
Support filtering Data and Entity objects by relation id
Create
upload_config
API endpoint that specifies upload connector type and credentials client can use for optimized upload
Fixed
Do not return multiple version of the same process while checking for permissions in Python processes
Change misleading error message when importing file if the response with status code indicating error was received from the upstream server
Changed
Use
data_id
instead ofdata.id
when notifying dispatcher to avoid potential database query inside async context
30.0.0 - 2022-03-14
Added
Add support for
Python
3.10Add health checks support for deploy in Kubernetes
Add
collecttools_kubernetes
management commandAdd
COMMUNICATION_CONTAINER_LISTENER_CONNECTION
to separate settings for listener (where to bind to) and containers (where to connect to)Support
docker_volume
setting in connector configSupport use of named volumes as processing or input volume in Docker executor
Support SSL connection to Redis
Changed
BACKWARD INCOMPATIBLE: Require
Django
3.2BACKWARD INCOMPATIBLE: Require
Django Priority Batch
version 4Do not prepare tools configmaps in Kubernetes workload connector
Enable Docker containers to connect to the custom network
Auto-delete completed jobs in Kubernetes after 5 minutes
Optionally add affinity to the Kubernetes job
Remove support for setting permissions using old syntax
29.3.0 - 2022-02-15
Added
Add MD5 checksum to
DataBrowseView
view
29.2.0 - 2022-01-17
Added
Support
range
parameter in fields of Python proces
Changed
Flush stdout/stderr on Python processes on every write
Fixed
Add attribute as a field to a
resolwe.process.fields.GroupField
in Python process only if it is an instance ofresolwe.process.fields.Field
.
29.1.0 - 2021-12-12
Changed
Do not fail in case of missing files in
UriResolverView
Fixed
Remove references to temporary export files from the database and make sure they are not created anymore
Wrap
move_to_collection
in transaction and only call method if collection has changed
29.0.0 - 2021-11-11
Added
BACKWARD INCOMPATIBLE: New permission architecture: it is not based on Guardian anymore. The main benefits of new architecture are speed gains in common operations, such as setting a permission and retrieving objects with the given permission.
Allow overriding process resources in data object
Changed
Allow mounting connectors into pods as persistent volume claim instead of volume of type
hostPath
Fixed
use the same connector inside pod to handle files and directories
When data object was deleted listener did not receive the terminate message and pod did not terminate immediatelly
28.5.0 - 2021-09-13
Added
Add
compare_models_and_csv
management script to check if all ``ReferencedPath``s point to a valid file in the aws databaseAdd method
get_latest
toProcess
class in Python Processes returning the latest version of the process with the given slugSupport assuming role in S3 connector
Changed
Set hashes during upload to avoid creating multiple versions of the object in S3 bucket with enabled versioning
28.4.0 - 2021-08-16
Changed
Remove dependency on EFS/NFS when running on Kubernetes
When running on Kubernetes the runtime volume configuration can be omitted
28.3.0 - 2021-07-20
Fixed
Prepare release 28.3.0 due to preexisting 28.3.0a1 pre-release
28.2.1 - 2021-07-13
Fixed
Speed up deleting storage locations by considering only referenced paths belonging to the given storage location
Temporary pin
asteval
to version0.9.23
due to compatibility issues with Python 3.6
Changed
Improve logging in cleanup manager
28.2.0 - 2021-06-15
Fixed
Create
ReferencedPath
objects during transfer only when needed
Changed
Retry data transfer if
botocore.exceptions.ClientError
is raised during transfer
Added
Add
FLOW_PROCESS_MAX_MEM
Django setting to limit the ammount of memory used by a processSupport disabled fields in Python processes
Add method
get_latest
to theProcess
class inPython
processes which retrieves latest version of the process with the givenslug
28.1.0 - 2021-05-17
Fixed
Do not raise exception when terminating
runlistener
management commandChange concurrency issue in the listener causing processes to sometitimes get incorrect value for
RUNTIME_VOLUME_MAPS
settings
Changed
Make S3 connectors use system credentials when they are not explicitely given in settings
Added
Make it possible to rewrite container image names in kubernetes workload connector
28.0.4 - 2021-05-04
Fixed
Use per process storage overrides
28.0.3 - 2021-05-04
Changed
Make
gp2
default EBS storage class
28.0.2 - 2021-05-03
Fixed
Log peer activity on every received message to avoid declaring otherwise healthy node as failed
Fix possible data loss caused by parallel command processing when uploading log files interfered with processing command from a script
Fix deadlock when uploading empty files
28.0.1 - 2021-04-28
Changed
Make logger level inside init and communication containers configurable via environmental variable
Change default logger level inside init and communication containers for AWS S3 and Google Cloud Storage components to WARNING
Fixed
Stop timer that uploads log files in the processing container immediatelly after the processing is finished to avoid timing issues that could cause the data object to be marked as failed
28.0.0 - 2021-04-19
Changed
BACKWARD INCOMPATIBLE: Require
Django 3.1.x
BACKWARD INCOMPATIBLE: Require
Django Channels
version 3.0.xBACKWARD INCOMPATIBLE: Require
asgiref
version 3.3.xBACKWARD INCOMPATIBLE: Require
jsonschema
version 3.2.xBACKWARD INCOMPATIBLE: Require
Sphinx
version 3.5.xBACKWARD INCOMPATIBLE: Require
django-guardian
version 2.3.xRefresh versions of the other dependencies
Replace
django-versionfield2
withdjango-versionfield
Overhaul of the storage configuration
Remove per-process runtime directory
Increase socket timeouts in the processing and the communication container
Added
Add multipart upload capability to
LocalFilesystemConnector
andAwsS3Connector
Support uploading files to
LocalFilesystemConnector
orAwsS3Connector
Add support for
Python 3.9
27.1.2 - 2021-03-22
Fixed
Bump version of
upload-dir
process to use the default version of processing image instead of the previous oneUse Signature Version 4 when generating presigned URLs in S3 connector
Fix possible socket timeout when uploading files in the processing container
Remove static
rnaseq
image from list of docker images
Changed
Use tagged base image in
upload-file
processAllow to change Entity descriptor in Python process.
27.1.1 - 2021-03-21
Fixed
Fix connection timeout in communication container when sending initial message to the listener
27.1.0 - 2021-03-15
Fixed
Account for file system overhead when processing Data objects with large inputs
Changed
Improve storage manager to only process applicable storage locations instead of iterating through all of them
Skip hash computation when connector itself provides data integrity check
Remove
job
prefix from kubernetes job nameMake error messages in Python processes more useful
Added
Add label
job_type
to Kubernetes job to separate interactive jobs from batch jobs
27.0.0 - 2021-02-22
Fixed
Fixed progress reporting in Python processes
Do not override content-type of S3 object when storing hashes
Support upload of files larger than 80G to AWS S3
Changed
Download input data in init container
Storage objects are sent to the listener over socket instead of using files on the shared filesystem
Make it possible to run the platform without shared filesystem. All inputs for processed data object are prepared in input container and all outputs are uploaded to the chosen storage connector when they are referenced.
Overcommit CPU in kubertenes processing container by 20%
Move docker images from Docker Hub to Amazon ECR
Added
Make automatic removal of Docker containers configurable
Terminate processing immediately when data object is deleted
Make default processing image configurable
26.0.0 - 2021-01-20
Changed
BACKWARD INCOMPATIBLE: Remove
resolwe/upload-tab-file
andresolwe/archiver
Docker imagesBACKWARD INCOMPATIBLE: Remove obsolete processes:
archiver
,upload-tab-file
andupload-image-file
BACKWARD INCOMPATIBLE: Python process syntax has changed: all the attributes of Data object are now available in Python process and therefore accessing outputs using syntax
data_object.output_name
is no longer valid sinceoutput_name
could be the name of the attribute. The new syntax isdata_object.output.output_name
.BACKWARD INCOMPATIBLE: Communication between the processing script and listener has changed from printing to stdout to sending messages over sockets. Messages printed to stdout or sent using old version of the
resolwe-runtime-utils
(YAML processes) are no longer processed. YAML processes need new version ofresolwe-runtime-utils
while Python processes require a rewrite to the new syntax and Python version 3.4 or higher in the container (resolwe-runtime-utils
package is no longer needed).Use Github Actions to run the tests
Listener communicates with containers through ZeroMQ instead of Redis
Start two containers for each process instead of one: the second one is used to communicate with the listener
Move settings for Python processes from files to environmental variables
Added
Add Kubernetes workload connector
Support running process instant termination
Support registering custom command handlers in listener and exposing data objects (possibly defined in other Django applications) to Python processes
Support Django-like syntax in Python processes to create, filter or access attributes of the exposed data objects
Support creating new base classes for Python processes
25.2.0 - 2020-12-15
Fixed
Allow retrieval of Storage object that was linked to more than one
Data
object
Changed
Migrate docker images to Fedora 33 and Ubuntu 20.04
25.1.0 - 2020-11-16
Added
Support Python processes in Sphinx
autoprocess*::
directive
25.0.0 - 2020-10-16
Added
BACKWARD INCOMPATIBLE: Only copy parent relations when duplicating
Data
objectsAdd duplicate data dependency to indicate from which object the
Data
object was duplicatedSupport accessing Data name in Python processes through
self.name
Add
permission
filter tocollection
,entity
anddata
that returns only objects on which current user has given permission
Changed
Make relations in collection visible to public user if he has view permissions on the collection
24.0.0 - 2020-09-14
Changed
BACKWARD INCOMPATIBLE: Terminate Python process immediately after
self.error
method is called inside the process
Fixed
Make sure to terminate Docker container before executor exits
Speed-up duplication of
Data
objects,Enteties
, andCollections
Lock inputs’ storage locations while the process is waiting and processing to make sure that they are not deleted
Don’t validate input objects when
Data
object is marked as done as they may already be deleted at that point
23.0.0 - 2020-08-17
Fixed
Fix ordering options in Data viewset to enable ordering by
process__name
andprocess__type
Handle exception when processing deleted Data object
Changed
BACKWARD INCOMPATIBLE: Delete
elastic
applicationDon’t pass undefined values to steps of workflows
Added
Add relations property to
data:
andlist:data:
fields to support relations on clientAdd
entity_id
property doDataField
in Python processesAdd relations in Python processes
22.1.3 - 2020-07-13
Fixed
When deciding which StorageLocation objects will be deleted consider only completed StorageLocation objects.
Add
google.resumable_media.common.DataCorruption
exception totransfer_exceptions
tuple.
22.1.2 - 2020-06-30
Fixed
Celery sometimes starts more than one worker for a given Data object. In such case the download and purge part of the worker must be skipped or errors processing Data objects might occur.
22.1.1 - 2020-06-16
Changed
Remove
asgiref
version pin due to new release that fixed previous regression.
22.1.0 - 2020-06-15
Changed
Rename
transfer_rec
totransfer_objects
and change its signature to accept dictionary objects with information about name, size and hashes of objects to transferMove part of
Data
object validation to listenerImprove loading time of collection endpoint
Added
Add
move_to_collection
method toData
viewsetReport registration failure in
ProcessTestCase
Add a pseudo Python process to serve as a template
Add
validate_urls
method to storageBaseConnector
classValidate storage connector settings on registraton
Add
transfer_data
method toStorageLocation
classRemove data when
StorageLocation
object is deletedStore file hashes inside
ReferencedPath
model and connect it toStorageLocation
modelAdd
get_hashes
method to storage connectorsAdd
open_stream
method to storage connectorsAdd
compute_hashes
function tostorage.connectors.hasher
moduleUse threads when transfering files with
AwsS3Connector
Add
duplicate
method to storage connectorsAdd pre-processing and post-processing hooks to storage connectors
Use multiple threads for file transfer
Fixed
Add missing decorator
validate_url
toAwsS3Connector
Always import exceptions from
requests
libraryFix bug that sometimes caused objects inside workflow to fail with
Failed to transfer data.
Fix dependency handling bug in listener: when checking for missing data listener must only consider depencies with kind
KIND_IO
instead of all depencies.Raise exception when data transfer failed.
22.0.0 - 2020-05-18
Changed
BACKWARD INCOMPATIBLE: Move purge code inside worker, remove old purge code
Various code fixes to make code work with the new storage model
Use storage connectors in workers to download data not available locally
Added
Add
resolwe.storage
application, a framework for storage managementAdd storage connectors for Google Cloud Storage, Amazon Simple Storage service and local filesystem.
Add migrations to move from old storage model to the new one
Add storage manager
Add management command to start storage manager
Add cleanup manager for removing unreferenced data
Add
isnull
related lookup filterAdd
entity_count
to theCollection
serializerAdd
inherit_collection
toData
viewsetAdd entity_always_create in
Process
serializer
21.1.0 - 2020-04-14
Added
Add support for the
allow_custom_choice
field property in Python processesAdd ordering by contributor’s first and last name to Collection and Data viewsets
Add
data_count
andstatus
fields to theCollection
serializer
Fixed
Enable all top-level class definitions in Python processes
Make filtering via foreign key more ‘Django like’: when foreign key does not exist return empty set instead of raising validation exception. Also when filtering using list of foreign keys do not raise validation exception if some foreign keys in the list do not exist.
Reduce number of database queries in API viewsets by prefetching all required data
21.0.0 - 2020-03-16
Changed
BACKWARD INCOMPATIBLE: Use Postgres filtering instead of Elasticsearch on API endpoints
BACKWARD INCOMPATIBLE: Remove filtering by
year
,month
,day
,hour
,minute
andsecond
on API endpointsMigrate docker images to Fedora 31
Use
DictRelatedField
forcollection
field inRelationSerializer
. In practice this fixes inconsistency comparing with how other serializers handle collections field.
Added
Add a custom database connector to optimize queries and enable them to use database indexes
Add database indexes to improve search performance
Add database fields and triggers for full-text search in Postgres
Add support for annotating entities in processes
Add support for Python 3.8
20.2.0 - 2020-02-17
Added
Support workflows as inputs to Python processes
Support retrieval of
Data.name
in Python processAdd
name_contains
,contributor_name
, andowners_name
collection and data filtering fields on APIAdd
username
tocurrent_user_permissions
field of objects on APIAdd
delete_chunked
method to Collection, Entity and Storage managers
Fixed
Delete orphane Storage objects in chunks in purge command to prevent running out of memory
20.1.0 - 2019-12-16
Added
Add
description
field to Collection full-text search
20.0.0 - 2019-11-18
Changed
BACKWARD INCOMPATIBLE: Remove
download
permission from Data objects, samples and collections andadd
permission from samples and collectionsBACKWARD INCOMPATIBLE: Remove
Entity.descriptor_completed
field
Fixed
Fix Docker executor command with
--cpus
limit option. This solves the issue where process is killed before the timeout 30s is reached
19.1.0 - 2019-09-17
Added
Support filtering by
process_slug
inDataViewSet
Fixed
Fix
DictRelatedField
so it can be used in browsable-APIFix access to subfields of empty
GroupField
in Python processes
19.0.0 - 2019-08-20
Changed
BACKWARD INCOMPATIBLE: Change relations between
Data
,Entity
andCollection
fromManyToMany
toManyToOne
. In practice this means thatData.entity
,Data.colllecton
andEntity.collection
are nowForeignKey
-s. This also implies the following changes:CollectionViewSet
methodsadd_data
andremove_data
are removedEntityViewset
methodsadd_data
,``remove_data``,add_to_collection
andremove_from_collection
are removedEntityQuerySet
andEntity
methodduplicate
argumentinherit_collections
is renamed toinherit_collection
.EntityFilter
FilterSet fieldcollections
is renamed tocollection
.
BACKWARD INCOMPATIBLE: Change following fields in
DataSerializer
:process_slug
,process_name
,process_type
,process_input_schema
,process_output_schema
are removed and moved inprocess
field which is nowDictRelatedField
that usesProcessSerializer
for representationRemove
entity_names
andcollection_names
fieldsadd
entity
andcolection
fields which areDictRelatedField
-s that use corresponding serializers for representationRemove support for
hydrate_entities
andhydrate_collections
query parameters
BACKWARD INCOMPATIBLE: Remove
data
field inEntitySerializer
andCollectionSerializer
. This implies that parameterhydrate_data
is no longer supported.BACKWARD INCOMPATIBLE: Remove
delete_content
paremeter indelete
method ofEntityViewset
andCollectionViewSet
. From now on, whenEntity
/Collection
is deleted, all it’s objects are removed as wellGather all
Data
creation logic intoDataQuerySet.create
method
Added
Enable sharing based on user email
Support running tests with live Resolwe host on non-linux platforms
Add
inherit_entity
andinherit_collection
arguments toData.duplicate
andDataQuerySet.duplicate
methodImplement
DictRelatedField
18.0.0 - 2019-07-15
Changed
BACKWARD INCOMPATIBLE: Remove
parents
andchildren
query filters from Data API endpoint
Added
/api/data/:id/parents
and/api/data/:id/children
API endpoints for listing parents and children Data objects of the object with givenid
Add
entity_always_create
field toProcess
model
Fixed
Make sure that Elasticsearch index exists before executing a search query
17.0.0 - 2019-06-17
Changed
BACKWARD INCOMPATIBLE: Use Elasticsearch version 6.x
BACKWARD INCOMPATIBLE: Bump
Django
requirement to version2.2
BACKWARD INCOMPATIBLE: Remove not used
django-mathfilters
requirement
Added
Support Python 3.7
Support forward and reverse many-to-one relations in Elasticsearch
Add
collection_names
field toDataSerializer
Add test methods to
ProcessTestCase
that assert directory structure and content:assertDirExists
,assertDir
, andassertDirStructure
Add
upload-dir
process
16.0.1 - 2019-04-29
Fixed
Pin
django-priority-batch
to version1.1
to fix compatibility issues
16.0.0 - 2019-04-16
Changed
BACKWARD INCOMPATIBLE: Access to DataField members (in Python process input) changed from dict to Python objects. For example,
input_field.file_field['name']
changed toinput_field.file_field.path
.BACKWARD INCOMPATIBLE: Filters that are based on
django-filter
FilterSet
now use dict-declaring-syntax. This requires that subclasses of respective filters modify their syntax too.Interactively save results in Python processes
Added
Add get_data_id_by_slug method to Python processes’ Process class
Python process syntax enhancements:
Support
.entity_name
in data inputsEasy access to process resources through
self.resources
Raise error if ViewSet receives invalid filter parameter(s)
Report process error for exceptions in Python processes
Report process error if spawning fails
Automatically export files for spawned processes (in Python process syntax)
Import files of Python process FileField inputs (usage: inputs.src.import_file())
Fixed
Interactively write to standard output within Python processes
Fix writing to integer and float output fields
Allow non-required
DataField
as Python process input
15.0.1 - 2019-03-19
Fixed
Fix storage migration to use less memory
15.0.0 - 2019-03-19
Changed
Log plumbum commands to standard output
Change storage data relation from many-to-one to many-to-many
Moved
purged
field fromData
toDataLocation
model
Added
Add
run_process
method toProcess
to support triggering of a new process from the running Python processAdd DataLocation model and pair it with Data model to handle data location
Add
entity_names
field toDataSerializer
Support duplication of
Data
,Entity
andCollection
Support moving entities between collections
Support relations requirement in process syntax
14.4.0 - 2019-03-07
Changed
Purge processes only not jet purged Data objects
Fixed
Allow references to missing Data objects in the output of finished Data objects, as we don’t have the control over what (and when) is deleted
14.3.0 - 2019-02-19
Added
Add
scheduled
field toData
objects to store the date when object was dispatched to the scheduling systemAdd
purge
field toData
model that indicates whetherData
object was processed bypurge
Fixed
Make Elasticsearch build arguments cache thread-safe and namespace cache keys to make sure they don’t interfere
Trigger the purge outside of the transaction, to make sure the Data object is commited in the database when purge worger grabs it
14.2.0 - 2019-01-28
Added
Add
input
Jinja filter to access input fields
14.1.0 - 2019-01-17
Added
Add
assertFilesExist
method toProcessTestCase
Add
clean_test_dir
management command that removes files created during testing
Fixed
Support registration of Python processes inherited from
process.Process
Skip docker image pull if image exists locally. This solves the issue where pull would fail if process uses an image that is only used locally.
14.0.1 - 2018-12-17
Fixed
Make sure that tmp dir exists in Docker executor
14.0.0 - 2018-12-17
Changed
BACKWARD INCOMPATIBLE: Run data purge in a separate worker to make sure that listener replies to the executor within 60 seconds
Use batcher for spawned processes in listener
Increase Docker’s memory limit for 100MB to make sure processes are not killed when using all available memory and tune Docker memory limits to avoid OOM.
Added
Raise an exception in Docker executor if container doesn’t start for 60 seconds
Set
TMPDIR
environment variable in Docker executor to.tmp
dir in data directory to prevent filling up container’s local storage
Fixed
Process SIGTERM signal in executor as expected - set the Data status to error and set the process_error field
Clear cached Django settings from the manager’s shared state on startup
13.3.0 - 2018-11-20
Changed
Switch channels_redis dependency to upstream version
Added
Python execution engine
Support multiple entity types
Support extending viewsets with custom filter methods
Add tags attribute to
ProcessTestCase.run_process
method which adds listed tag to the createdData
objectCopy
Data
objects tags from parent objects for spawnedData
objects andData
objects created by workflows
Fixed
Fix manager shutdown in the test runner. If an unrecoverable exception occurred while running a test, and never got caught (e.g. an unpicklable exception in a parallel test worker), the listener would not get terminated properly, leading to a hang.
Data and collection name API filters were fixed to work as expected (ngrams was switched to raw).
13.2.0 - 2018-10-23
Added
Use prioritized batcher in listener
13.1.0 - 2018-10-19
Added
Use batching for ES index builds
Fixed
Fix handling of M2M dependencies in ES indexer
13.0.0 - 2018-10-10
Changed
BACKWARD INCOMPATIBLE: Remove Data descriptors from Entity Elasticsearch index
Support searching by
slug
anddescriptor_data
in entity viewset text search
Added
Add tags to collections
12.0.0 - 2018-09-18
Changed
BACKWARD INCOMPATIBLE: Switch
Collection
andEntity
API viewsets to use ElasticsearchBACKWARD INCOMPATIBLE: Refactor
Relation
model, which includes:renaming
position
topartition
renaming
label
tocategory
and making it requiredadding
unit
making
collection
field requiredrequiring unique combination of
collection
andcategory
renaming partition’s
position
tolabel
adding (integer)
position
to partition (used for sorting)deleting
Relation
when the lastEntity
is removed
BACKWARD INCOMPATIBLE: Remove rarely used parameters of the
register
command--path
and--schemas
.Omit
current_user_permissions
field in serialization if only a subset of fields is requestedAllow slug to be null on update to enable slug auto-generation
Retire obsolete processes. We have added the
is_active
field to the Process model. The field is read-only on the API and can only be changed through Django ORM. Inactive processes can not be executed. Theregister
command was extended with the--retire
flag that removes old process versions which do not have associated data. Then it finds the processes that have been registered but do not exist in the code anymore, and:If they do not have data: removes them
If they have data: flags them not active (
is_active=False
)
Added
Add support for URLs in
basic:file:
fields in Django testsAdd
collections
andentities
fields to Data serializer, with optional hydration usinghydrate_collections
and/orhydrate_entities
Support importing large files from Google Drive in re-import
Add
python3-plumbum
package to resolwe/base:ubuntu-18.04 image
Fixed
Prevent mutation of
input_
parameter inProcessTestCase.run_process
Return 400 instead of 500 error when slug already exists
Add trailing colon to process category default
Increase stdout buffer size in the Docker executor
11.0.0 - 2018-08-13
Changed
BACKWARD INCOMPATIBLE: Remove option to list all objects on Storage API endpoint
Make the main executor non-blocking by using Python asyncio
Debug logs are not send from executors to the listener anymore to limit the amount of traffic on Redis
Added
Add size to Data serializer
Implement
ResolweSlugRelatedField
. As a result,DescriptorSchema
objects can only be referenced byslug
(instead ofid
)Add options to filter by
type
andscheduling_class
on Process API endpoint
Fixed
Inherit collections from
Entity
when addingData
object to it
10.1.0 - 2018-07-16
Changed
Lower the level of all
INFO
logs in elastic app toDEBUG
Added
Add load tracking to the listener with log messages on overload
Add job partition selection in the SLURM workload connector
Add
slug
Jinja filterSet
Data
status toERROR
if executor is killed by the scheduling system
Fixed
Include the manager in the documentation, make sure all references work and tidy the content up a bit
10.0.1 - 2018-07-07
Changed
Convert the listener to use asyncio
Switched to
channels_redis_persist
temporarily to mitigate connection storms
Fixed
Attempt to reconnect to Redis in the listener in case of connection errors
10.0.0 - 2018-06-19
Changed
BACKWARD INCOMPATIBLE: Drop support for Python 3.4 and 3.5
BACKWARD INCOMPATIBLE: Start using Channels 2.x
Added
Add the options to skip creating of fresh mapping after dropping ES indices with
elastic_purge
management commandAdd
dirname
andrelative_path
Jinja filters
9.0.0 - 2018-05-15
Changed
Make sorting by contributor case insensitive in Elasticsearch endpoints
Delete ES documents in post delete signal instead of pre delete one
Added
BACKWARD INCOMPATIBLE: Add on-register validation of default values in process and schemas
BACKWARD INCOMPATIBLE: Validate that field names in processes and schemas start with a letter and only contain alpha-numeric characters
Support Python 3.6
Add
range
parameter and related validation to fields of typebasic:integer:
,basic:decimal
,list:basic:integer:
andlist:basic:decimal
Support filtering and sorting by
process_type
parameter on Data API endpointAdd
dirname
Jinja filterAdd
relative_path
Jinja filter
Fixed
Add missing
list:basic:decimal
type to JSON schemaDon’t crash on empty
in
lookupFix {{ requirements.resources.* }} variables in processes to take in to the account overrides specified in Django settings
Create Elasticsearch mapping even if there is no document to push
8.0.0 - 2018-04-11
Changed
BACKWARD INCOMPATIBLE: Use Elasticsearch version 5.x
BACKWARD INCOMPATIBLE: Raise an error if an invalid query argument is used in Elasticsearch viewsets
BACKWARD INCOMPATIBLE: Switch
Data
API viewset to use ElasticsearchTerminate the executor if listener response with error message
verbosity
setting is no longer propagated to the executorOnly create Elasticsearch mappings on first push
Added
Add
sort
argument toassertFile
andassertFiles
methods inProcessTestCase
to sort file lines before asserting the contentAdd
process_slug
field toDataSerializer
Improve log messages in executor and workload connectors
Add
process_memory
andprocess_cores
fields toData
model andDataSerializer
Support lookup expressions (
lt
,lte
,gt
,gte
,in
,exact
) in ES viewsetsSupport for easier dynamic composition of type extensions
Add
elastic_mapping
management command
Fixed
Fix Elasticsearch index rebuilding after a dependant object is deleted
Send response to executor even if data object was already deleted
Correctly handle reverse m2m relations when processing ES index dependencies
7.0.0 - 2018-03-12
Changed
BACKWARD INCOMPATIBLE: Remove Ubuntu 17.04 base Docker image due to end of lifetime
BACKWARD INCOMPATIBLE: Remove support for Jinja in
DescriptorSchema
’s default valuesBACKWARD INCOMPATIBLE: Remove
CONTAINER_IMAGE
configuration option from the Docker executor; if no container image is specified for a process using the Docker executor, the same pre-defined default image is used (currently this isresolwe/base:ubuntu-16.04
)Add mechanism to change test database name from the environment, appending a
_test
suffix to it; this replaces the static name used before
Added
Add Ubuntu 17.10 and Ubuntu 18.04 base Docker images
Add database migration operations for process schema migrations
Add
delete_chunked
method toData
objects queryset which is needed due to Django’s extreme memory usage when deleting a large count ofData
objectsAdd
validate_process_types
utility function, which checks that all registered processes conform to their supertypesAdd
FLOW_CONTAINER_VALIDATE_IMAGE
setting which can be used to validate container image namesOnly pull Docker images at most once per process in
list_docker_images
Add
FLOW_PROCESS_MAX_CORES
Django setting to limit the number of CPU cores used by a process
Fixed
Make parallel test suite worker threads clean up after initialization failures
Add mechanism to override the manager’s control channel prefix from the environment
Fix deletion of a
Data
objects which belongs to more than oneEntity
Hydrate paths in
refs
ofbasic:file:
,list:basic:file:
,basic:dir:
andlist:basic:dir:
fields before processingData
object
6.1.0 - 2018-02-21
Changed
Remove runtime directory during general data purge instead of immediately after each process finishes
Only process the Data object (and its children) for which the dispatcher’s
communicate()
was triggeredPropagate logging messages from executors to the listener
Use process’ slug instead of data id when logging errors in listener
Improve log messages in dispatcher
Added
Add
descriptor_completed
field to theEntity
filterValidate manager semaphors after each test case, to ease debugging of tests which execute processes
Fixed
Don’t set Data object’s status to error if executor is run multiple times to mitigate the Celery issue of tasks being run multiple times
Make management commands respect the set verbosity level
6.0.1 - 2018-01-29
Fixed
Make manager more robust to ORM/database failures during data object processing
Rebuild the ElasticSearch index after permission is removed from an object
Trim
Data.process_error
,Data.process_warning
andData.process_info
fields before saving themMake sure values in
Data.process_error
,Data.process_warning
andData.process_info
cannot be overwrittenHandle missing
Data
objects inhydrate_input_references
functionMake executor fail early when executed twice on the same data directory
6.0.0 - 2018-01-17
Changed
BACKWARD INCOMPATIBLE:
FLOW_DOCKER_LIMIT_DEFAULTS
has been renamed toFLOW_PROCESS_RESOURCE_DEFAULTS
andFLOW_DOCKER_LIMIT_OVERRIDES
has been renamed toFLOW_PROCESS_RESOURCE_OVERRIDES
BACKWARD INCOMPATIBLE:
Process.PERSISTENCE_TEMP
is not used for execution priority anymoreBACKWARD INCOMPATIBLE: There is only one available manager class, which includes dispatch logic; custom manager support has been removed and their role subsumed into the new connector system
BACKWARD INCOMPATIBLE: Removed
FLOW_DOCKER_MAPPINGS
in favor of newFLOW_DOCKER_VOLUME_EXTRA_OPTIONS
andFLOW_DOCKER_EXTRA_VOLUMES
Parent relations are kept even after the parent is deleted and are deleted when the child is deleted
Dependency resolver in manager is sped up by use of parent relations
Validation of
Data
inputs is performed on save instead of on create
Added
Support for the SLURM workload manager
Support for dispatching
Data
objects to different managersSupport for passing secrets to processes in a controlled way using a newly defined
basic:secret
input typeis_testing
test helper function, which returnsTrue
when invoked in tests andFalse
otherwiseAdd
collecttools
Django command for collecting tools’ files in single location defined inFLOW_TOOLS_ROOT
Django setting which is used for mapping tools in executor whenDEBUG
is set toFalse
(but not in tests)
Fixed
Fix
Data
object preparation race condition incommunicate()
Set correct executor in flow manager
Make executors more robust to unhandled failures
Calculate
Data.size
by summingtotal_size
of all file-type outputsDon’t change slug explicitly defined by user - raise an error instead
Objects are locked while updated over API, so concurrent operations don’t override each other
Make manager more robust to unhandled failures during data object processing
Fix manager deadlock during tests
Fix ctypes cache clear during tests
Don’t raise
ChannelFull
error in manager’s communicate callDon’t trim predefined slugs in
ResolweSlugField
5.1.0 - 2017-12-12
Added
Database-side JSON projections for
Storage
modelsCompute total size (including refs size) for file-type outputs
Add
size
field toData
model and migrate all existing objects
Change
Change Test Runner’s test directory creation so it always creates a subdirectory in
FLOW_EXECUTOR
’sDATA_DIR
,UPLOAD_DIR
andRUNTIME_DIR
directories
Fixed
Do not report additional failure when testing a tagged process errors or fails
Fix Test Runner’s
changes-only
mode when used together with a Git repository in detachedHEAD
stateFix handling of tags and test labels together in Test Runner’s
changes-only
modeFix parallel test execution where more test processes than databases were created during tests
5.0.0 - 2017-11-28
Changed
BACKWARD INCOMPATIBLE: The
keep_data()
method inTransactionTestCase
is no longer supported. Use the--keep-data
option to the test runner instead.BACKWARD INCOMPATIBLE: Convert the manager to Django Channels
BACKWARD INCOMPATIBLE: Refactor executors into standalone programs
BACKWARD INCOMPATIBLE: Drop Python 2 support, require Python 3.4+
Move common test environment preparation to
TestCaseHelpers
mixin
Fixed
Fix parents/children filter on Data objects
Correctly handle removed processes in the changes-only mode of the Resolwe test runner
4.0.0 - 2017-10-25
Added
BACKWARD INCOMPATIBLE: Add option to build only subset of specified queryset in Elasticsearch index builder
--pull
option to thelist_docker_images
management commandTest profiling and process tagging
New test runner, which supports partial test suite execution based on changed files
Add
all
andany
Jinja filters
Changed
BACKWARD INCOMPATIBLE: Bump Django requirement to version 1.11.x
BACKWARD INCOMPATIBLE: Make
ProcessTestCase
non-transactionalPull Docker images after process registration is complete
Generalize Jinja filters to accept lists of
Data
objectsWhen new
Data
object is created, permissions are copied from collections and entity to which it belongs
Fixed
Close schema (YAML) files after
register
command stops using themClose schema files used for validating JSON schemas after they are no longer used
Close stdout used to retrieve process results in executor after the process is finished
Remove unrelated permissions occasionally listed among group permissions on
permissions
endpointFix
ResolwePermissionsMixin
to work correctly with multi-words model names, i.e.DescriptorSchema
Fix incorrect handling of offset/limit in Elasticsearch viewsets
3.1.0 - 2017-10-05
Added
resolwe/base
Docker image based on Ubuntu 17.04Support different dependency kinds between data objects
Fixed
Serialize
current_user_permissions
field in a way that is compatible with DRF 3.6.4+API requests on single object endpoints are allowed to all users if object has appropriate public permissions
3.0.1 - 2017-09-15
Fixed
Correctly relabel SELinux contexts on user/group files
3.0.0 - 2017-09-13
Added
Add filtering by id on
descriptor_schma
API endpointSupport assigning descriptor schema by id (if set value is of type int) on
Collection
,Data
andEntity
endpointsassertAlmostEqualGeneric
test case helper, which enables recursive comparison for almost equality of floats in nested containers
Changed
BACKWARD INCOMPATIBLE: Run Docker containers as non-root user
Fixed
Use per-process upload dir in tests to avoid race conditions
2.0.0 - 2017-08-24
Added
descriptor
jinja filter to get the descriptor (or part of it) in processesUbuntu 14.04/16.04 based Docker images for Resolwe
Add
list_docker_images
management command that lists all Docker images required by registered processes in either plain text or YAMLData status is set to
ERROR
and error message is appended toprocess_error
if value ofbasic:storage:
field is set to a file with invalid JSON
Changed
BACKWARD INCOMPATIBLE: Quote all unsafe strings when evaluating expressions in Bash execution engine
BACKWARD INCOMPATIBILE: Rename
permissions
attribute on API endpoints tocurrent_user_permissions
API
permissions
endpoint raises error if no owner is assigned to the object after applied changesowner
permission cannot be assigned to a groupObjects with public permissions are included in list API views for logged-in users
Owner permission is assigned to the contributor of the processes and descriptor schemas in the
register
management commandThe base image Dockerfile is renamed to Dockerfile.fedora-26
Fixed
Add
basic:url:link
field to the JSON schemaReturn more descriptive error if non-existing permission is sent to the
permissions
endpointHandle errors occurred while processing Elasticsearch indices and log them
Return 400 error with a descriptive message if permissions on API are assigned to a non-existing user/group
1.5.1 - 2017-07-20
Changed
Add more descriptive message if user has no permission to add
Data
object to the collection when the object is created
Fixed
Set contributor of
Data
object to public user if it is created by not authenticated userRemove remaining references to calling
pip
with--process-dependency-links
argument
1.5.0 - 2017-07-04
Added
Add Resolwe test framework
Add
with_custom_executor
andwith_resolwe_host
test decoratorsAdd
isort
linter to check order of importsSupport basic test case based on Django’s
TransactionTestCase
Support ES test case based on Django’s
TransactionTestCase
Support process test case based on Resolwe’s
TransactionTestCase
Add ability to set a custom command for the Docker executor via the
FLOW_DOCKER_COMMAND
setting.get_url
jinja filterWhen running
register
management command, permissions are automatically granted based on the permissions of previous latest version of the process or descriptor schema.Set
parent
relation in spawnedData
objects and workflowsRelations between entities
Resolwe toolkit Docker images
Archive file process
File upload processes
Resolwe process tests
Add
SET_ENV
setting to set environment variables in executorSupport ordering by version for descriptor schema
Add
NullExecutor
If
choices
are defined in JSON schema, value of field is validated with themAdd cpu core, memory and network resource limits
Add scheduling class for processes (
interactive
,batch
), which replaces the previously unused process priority fieldAdd
share_content
flag to the collection and entity permissions endpoint to also share the content of the coresponding objectAdd
delete_content
flag to the collection and entity destroy method on API to also delete the content of the coresponding object
Changed
Support running tests in parallel
Split
flow.models
module to multiple filesRemove ability to set a custom executor command for any executor via the
FLOW_EXECUTOR['COMMAND']
setting.Rename
RESOLWE_API_HOST
setting and environment variable in executor toRESOLWE_HOST_URL
Remove
keep_failed
function in tests.Rename
keep_all
function tokeep_data
.Manager is automatically run when new
Data
object is createdOutputs of
Data
objects with statusError
are not validatedSuperusers are no longer included in response in
permissions
endpoint of resourcesRemove
public_processes
field from theCollection
model as it is never usedPublic users can create new
Data
objects with processes and descriptor schemas on which they have appropriate permissionsAdd custom
ResolweSlugField
and use it instead ofdjango-autoslug
Fixed
SECURITY: Prevent normal users from creating new
Processes
over APIConfigure parallel tests
Isolate Elasticsearch indices for parallel tests
Fix Docker container name for parallel tests
Generate temporary names for upload files in tests
Fix permissions in Elasticsearch tests
Do not purge data in tests
Remove primary keys before using cached schemas’ in process tests
Set appropriate SELinux labels when mounting tools in Docker containers
Data
objects created by the workflow inherit its permissionsIf user doesn’t have permissions on the latest versions of processes and descriptor schemas, older ones are used or error is returned
Support
data:
andlist:data:
typesSet
Data
object status to error if worker cannot update the object in the databaseData
objects returned inCollectionViewset
andEntityViewset
are filtered by permissions of the user in requestPublic permissions are taken into account in elastic app
Treat
None
field value as if the field is missingCopy parent’s permissions to spawned
Data
objects
1.4.1 - 2017-01-27
Fixed
Update instructions on preparing a release to no longer build the wheel distribution which currently fails to install Resolwe’s dependency links
1.4.0 - 2017-01-26
Added
Auto-process style, type tree and category index
Support loading JSON from a file if the string passed to the
basic:json:
field is a file.list:basic:integer:
fieldData object’s checksum is automatically calculated on save
get_or_create
end point forData
objectsbasic:file:html:
field for HTML filesHelper function for comparing JSON fields in tests
Purge directories not belonging to any data objects
Ordering options to API endpoints
Workflow execution engine
data_by_slug
filter for jinja expression engineExport
RESOLWE_API_HOST
environment variable in executorAdd
check_installed()
test utility functionAdd support for configuring the network mode of Docker executor
Add
with_docker_executor
test utility decoratorSupport for Docker image requirements
Support version in descriptor schema YAML files
Add
Entity
model that allows grouping ofData
objectsIntroduce priority of Data objects
Data objects created with processes with temporary persistence are given high priority.
Add
resolwe.elastic
application, a framework for advanced indexing of Django models with ElasticSearch
Changed
Refactor linters, check PEP 8 and PEP 257
Split expression engines into expression engines and execution engines
Use Jinja2 instead of Django Template syntax
Expression engine must be declared in
requirements
Set Docker Compose’s project name to
resolwe
to avoid name clashesExpose
check_docker()
test utility functionUpdate versionfield to 0.5.0
Support Django 1.10 and update filters
Executor is no longer serialized
Put Data objects with high priority into
hipri
Celery queue.
Fixed
Fix pylint warnings (PEP 8)
Fix pydocstyle warnings (PEP 257)
Take last version of process for spawned objects
Use default values for descriptor fields that are not given
Improve handling of validation errors
Ignore file size in
assertFields
Order data objects in
CollectionViewSet
Fix tests for Django 1.10
Add quotes to paths in a test process test-save-file
1.3.1 - 2016-07-27
Added
Sphinx extension
autoprocess
for automatic process documentation
1.3.0 - 2016-07-27
Added
Ability to pass certain information to the process running in the container via environment variables (currently, user’s uid and gid)
Explicitly set working directory inside the container to the mapped directory of the current
Data
’s directoryAllow overriding any
FLOW_EXECUTOR
setting for testingSupport GET request on /api/<model>/<id>/permissons/ url
Add OWNER permissions
Validate JSON fields before saving
Data
objectAdd basic:dir field
RESOLWE_CUSTOM_TOOLS_PATHS
setting to support custom paths for tools directoriesAdd test coverage and track it with Codecov
Implement data purge
Add
process_fields.name
custom tamplate tagReturn contributor information together with objects
Added permissions filter to determine
Storage
permissions based on referencedData
object
Changed
Move filters to separate file and systemize them
Unify file loading in tests
Simplify
ProcessTestCase
by removing the logic for handling different uid/gid of the user running inside the Docker containerUpgrade to django-guardian 1.4.2
Rename
FLOW_EXECUTOR['DATA_PATH']
setting toFLOW_EXECUTOR['DATA_DIR']
Rename
FLOW_EXECUTOR['UPLOAD_PATH']
setting toFLOW_EXECUTOR['UPLOAD_DIR']
Rename
proc.data_path
system variable toproc.data_dir
Rename test project’s data and upload directories to
.test_data
and.test_upload
Serve permissions in new format
Rename
assertFiles
method inProcessTestCase
toassertFile
and add newassertFiles
method to checklist:basic:file
fieldMake
flow.tests.run_process
function also handle file pathsUse Travis CI to run the tests
Include all necessary files for running the tests in source distribution
Exclude tests from built/installed version of the package
Put packaging tests in a separate Tox testing environment
Put linters (pylint, pep8) into a separate Tox testing environment
Drop django-jenkins package since we no longer use Jenkins for CI
Move testing utilities from
resolwe.flow.tests
toresolwe.flow.utils.test
and fromresolwe.permissions.tests.base
toresolwe.permissions.utils.test
Add Tox testing environment for building documentation
Extend Reference documentation
Fixed
Spawn processors (add data to current collection)
Set collection name to avoid warnings in test output
Improve Python 3 compatibility
Fix setting descriptor schema on create
1.2.1 - 2016-05-15
Added
Add docker-compose configuration for PostgreSQL
Processes can be created on API
Enable spawned processes
Changed
Move logic from
Collection
model to theBaseCollection
abstract model and make it its parentRemove all logic for handling
flow_collection
Change default database user and port in test project’s settings
Keep track of upload files created during tests and purge them afterwards
Fixed
Test processes location agnostic
Test ignore timezone support
1.2.0 - 2016-05-06
Changed
Rename
assertFileExist
toassertFileExists
Drop
--process-dependency-links
from Tox’s pip configurationImprove documentation on preparing a new release
Added
Ability to use a custom executor command by specifying the
FLOW_EXECUTOR['COMMAND']
settingMake workload manager configurable in settings
Fixed
Make Resolwe work with Python 3 again
Fix tests
Render data name again after inputs are resolved
Ensure Tox installs the package from sdist
Pass all Resolwe’s environment variables to Tox’s testing environment
Ensure tests gracefully handle unavailability of Docker
1.1.0 - 2016-04-18
Changed
Rename process_register manage.py command to register
Reference process by slug when creating new Data object
Run manager when new Data object is created through API
Include full DescriptorSchema object when hydrating Data and Collection objects
Add djangorestframework-filters package instead of django-filters
Added
Tox tests for ensuring high-quality Python packaging
Timezone support in executors
Generating slugs with django-autoslug package
Auto-generate Data name on creation based on template defined in Process
Added endpoint for adding/removeing Data objects to/from Collection
Fixed
Pass all Resolwe’s environment variables to Tox’s testing environment
Include all source files and supplementary package data in sdist
Make Celery engine work
Add all permissions to creator of flow_collection Colection
Set DescriptorSchema on creating Data objects and Collections
Loading DescriptorSchema in tests
Handle Exceptions if input field doesn’t match input schema
Trigger ORM signals on Data status updates
Don’t set status od Data object to error status if return code of tool is 0
1.0.0 - 2016-03-31
Changed
Renamed Project to Collection
Register processes from packages and custom paths
Removed support for Python 3.3
Added
Permissions
API for flow
Docker executor
Expression engine support
Celery engine
Purge command
Framework for testing processors
Processor finders
Support for Django 1.9
Support for Python 3.5
Initial migrations
Introductory documentation
0.9.0 - 2015-04-09
Added
Initial release.