Actions annotations #11
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Content is the same
- The python_requires option in setup is not very helpful because setup will attempt to install the other dependencies prior to the python version requirement being checked. This is not helpful for developers which need quick feedback when trying to build. Instead, added a explicit check. - Added a mechanism to launch.py to run covfee commands in the VS code debugger - Removed some relative imports
- Implemented a AnnotationTypeDropdown based on the antd dropdown. Notes: there is a good likelihood that this can be trimmed and used as a direct instance given the experience with it. Also, this needs to be reusable, so we may simply fallback into using "Dropdown" directly. - Tried to set the styling for the dropdown, making a layout with a sidebar on the left and the video on the right. However, it is currently not working as it seems the video is being forced to render ocupying all available space. This may be forced somewhere in a higher level. - Added another task instance in the journey, for simple testing.
- The proposed AnnotationTypeDropdown became unnecessary because it did not add any new functionality that the base type Dropdown did not have, so it was removed. - Instantiated Dropdown now for both action selection and participant selection, while managing their state in Redux - Added custom styling and trying to have a functional sidebar with the video playback on the right - Added interactive gallery as a fullscreen overlay - The gallery image is the image reference for annotators to identify participants. - A button was added to display the gallery - The gallery is displayed as a modal overlay (fullscreen, steal keyboard input) - Added a simple logic to set the participant to annotate based on a click on the gallery image. This is easy given the gallery image is a regular grid. - Note: the image was made svg because there seem to be rules in the project in which only svg images are picked up. Unclear how to change atm
# Conflicts: # covfee/client/tasks/index.ts # covfee/shared/spec/task.ts # covfee/shared/task_dataclasses.py
- The "ContinuousAnnotation" task had been created as an example of how to retrieve the current annotations data. - Said logic was incorporated into the WIP frontend ActionAnnotations - This seems to replicate the example outcome. - Formatting was applied. There seems to be a configuration change.
- The ContinousAnnotationTask was one example of how to access the custom API, but without a proper frontend - The ActionAnnotationTask was the WIP implementation of the frontend - Now the logic of both elements is merged into ContinousAnnotationTask
- Implemented the POST/UPDATE functionality for an annotation buffer - On the update annotation function of the rest api, the updates to "created_at" and "updated_at" are being ignored now, because setting a str type leads to a crash - Introduced "participant" as part of the specs and annotation data point - Redefined specs to include a participant entry. - The UI is now populated by annotations data fetched by the server, instead of being filled up by the specs - Renamed the annotation.name into annotation.category given that name loses meaning given a annotation is chracterized by its type (eg. Speaking, Laughing, nose, etc.), defined as category, and the participant associated with the annotation. - The database is now also initialized with participant definition data - Now the ContinuousAnnotationTask React component is driven solely by the currently selected annotation index, which points to an AnnotationData element, which contains both the annotation category and participant name - Added safety logic for uninitialized data (not yet fetched), for fetched but not valid, and for a selection index outside of the array length. - The annotation options are now also populated and defined per participant, as found in the data.
- The annotation data array was extended to have a needs_upload flag. This was necessary to create another hook for posting data to server which would be guaranteed to take effect only after state changes to the annotation buffer fully take effect. - Added event listeners to keyboard. By default the key "s" - Improved Dropdown styling
- Implemented a basic css styling as a function of state in order to show a flashscreen, i.e., a little box changing color when they key is pressed during the annotation process. - The previous implementation was abusing useCallback, i.e., was used in many places where it was not really necesary. They were removed - Code was moved around to have logical groups of functions
- The functional component is composed of an SVG image component with minimum styling to make it occupy max width while preserving aspect ratio, and; a div component which is empty and simply styling as a bounding box overlay whose geometry is dynamically from the SVG geometry estimated and the selected index - Whenever the user clicks on the image, it leads to a view selection change based on where in the image the click was. - It is assumed that the views are regularly distributed in the image - Note: a ResizeObserver reference was used to monitor the size of the SVG when rendered, as the rendered size of the SVG was needed to dynamically compute the geometry of the bounding box - Added state to the parent component to manage the selected view index which is shared with the CamViewSelection component through Props
…ee into actions-annotations
- Rearranged the UI such that it reads like instructions (Step 1 to N) that the annotator has to follow in a loop in order to complete the entire task. - Added a feature in which a participant can be tagged as not found in any camera view. - When the annotator mark that the participant can't be found, their respective annotation data will be set to an array of 1 element - We keep track of how many data_json entries have valid data, to keep track of progress on the task. - Once all the data_json elements have data, the completion code is shown, together with a prolific academic link to bring the user to Prolific using their completion code in the link - Using the data_json entries, we also add a checked icon to the respective participant or action annotation to further convey progress and the action annotation and/or participants pending - The flashscreen has been moved to below the video and the status is also written, so the annotator understands what the flashscreen means. - The UI feedback related to pressing the annotation key (S) is now active even outside an annotation process. This is to let the annotator "train" without starting an annotation. - Improved the style of the left sidebar. - Improved the style of the right sidebar, together with the addition of instructions so the user knows what to do.
…ee into actions-annotations
- Expanded the Annotator DB table to include a column for the study ID
- Created a backref from TaskInstance to Annotation so we can iterate
through the annotations linked to such instance accordingly
- Created functionality to "Cleanup" or "Reset" the annotation data
associated to TaskInstances/Nodes associated to a journey
- Using the Prolific Academic API, we now query whether certain
annotators are returned, so we can decide whether to "Release" the journey
so it is assignable to another annotator
- Removed the unique constraint from Annotator, because now we treat
different prolific academic studies (Through study_id) differently, so
the same person can annotate for different studies (even if returned in one)
but they can't annotate twice or annotate after a returned state within
the same study
- Deleted files specific to TU Delft deployment
print("Resetting annotation data for all annotations")
…definition - Fixed a bug indexing the nodes assigned to a given HIT in the admin panel - Hiding the Graph for CONFLAB, which is distracting - Fixed a bug which was preventing the creation of multiple HITs, as there was a constraint that per row of the HIT the project_id + project_name had to be unique, which did not make sense. - Now dynamically associating ids to hits in the same way it is done for journeys, through a counter in dev mode and through a secret in deployment
- Currently covfee allows to build projects based on a spec file, that is mostly a python file that the study manager uses to create a CovfeeApp specification. However, in practice, studies sometimes have the requirement that new hits are needed because of multiple reasons: perhaps more annotators are required, or more data needs to be annotated, or a specific journey was poorly done. So we need functionality to add new hits. - This new functionality was created based on two principles: 1) HITs and Journeys can now be uniquely identified by the study manager thorugh sequential runs of covfee make; 2) If HITs/Journeys with new ids are detected, these get instantiated and committed to the database, and thus extending the study. - The loader and launcher were refactored, with the responsibility of creating and updated orm objects delegated to the Launcher, exclusively, whereas Loader is only about parsing spec files - Added functionality to confirm database changes and creating backups, whenever not working in dev mode.
- This allows to select which video url to use per participant being selected.
This ensures that there are no mismatches between the participant_id and its data
- Added the prolific_study_id as a new column in the ORM of the JourneySpec - Now journeys are ignored if their study id does not correspond to the one of the incoming annotator candidate - Bugfix: In the logic to assign journeys to incoming annotators, we will also filter out those which have 100% progress and consider them as FINISHED because there may be a bug in Covfee and the state is not being changed or the annotators do not click on the corresponding button
Fix video per per participant
- Fixed an issue that when the video failed to load and the user would try to annotate, it would create a data_json array which was non null, but empty. This would then become a "valid" annotation from the point of view of progress and task completion. This should now be prevented. - Progress is now calculated considering empty arrays as not valid, thus allowing to send again the url to annotators. - Stopped using the selectedAnnotationIndex from redux, and instead created a useState hook for the continuous annotation task because the custom dispatch appears to break the functionality when the FINISHED state is sent. - Exporting the prolific_study_id into the results json files
This reverts commit 651b185.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Here is the PR with all the changes that were done for the new conflab annotation. The highlights here are