Back to [Projects List](../../README.md#ProjectsList) # Time-Series Segmentation / Annotation Module ## Key Investigators - Rebecca Hisey (Queen's University) - Tamas Ungi (Queen's University) - Andras Lasso (Queen's University) - Andres Diaz-Pinto - Tina Kapur # Project Description Labelling data for training deep neural networks is a tedious and time-consuming task, especially for segmentation problems. Many of the existing tools available within Slicer are optimized for working with static 3D volumes such as MRI and CT. Annotation of video-based sequences such as ultrasound and RGB video have unique challenges that are not seen when working with static volumes. The primary goal of this project is to assess the requirements of researchers working with time-series or video data to design a Slicer module that will be convenient to use for a wide variety of users. We will mostly focus on the task of creating segmentation labels, but are open to the discussion of annotation for classification and detection problems as well. ## Objective 1. Objective A. Establish the requirements, challenges and current workflow of researchers working with time-series or sequence data 2. Objective B. Design a module to be implemented in Slicer that will provide the most benefit and convenience for satisfying the requirements above 4. Objective C. If we have time we would like to begin the implementation of the module ## Approach and Plan 1. We would like to have discussions with as many people who work with video and time-series data to assess their requirements and their current workflow for labelling data, along with any specific challenges relating to using currently available tools. 2. Compile a list of features that would be most useful and convenient to satisfy the requirements from objective A. 3. If there is time we will begin to design a user-interface for the module and potentially begin some preliminary implementation. ## Progress and Next Steps Have had many great discussions this week to assess the needs of various different projects. It seems that we can separate the task of segmenting time-series data into 2 distinct categories: 1. Segmentation of regions within the video frames - The task that we typically think of when we hear the word segmentation in reference to medical imaging - Requires tools for defining relatively small regions of interest - There is potential here for integration of online learning, but should treat the entire time-series as one object, rather than splitting into individual frames 2. Segmentation of the time-series as a whole into separate phases - More closely resembles a classification problem - Must be able to accommodate time-series with data obtained from mulitple modalities - Potential need for automatic segmentation of time-series into phases - Many helpful suggestions given by Steve Pieper about how to improve my current module for labelling this type of data (e.g. linking the range sliders to the sequence slider, automatic navigation to a phase within the series, use of a single button to add a phase to the segmentation at the current time in the series) For both of these tasks I will work to update them so that they can accommodate the use of AI to assist in the annotation process, whether that involves running the network locally on the computer, integrated into slicer, or on a server such as used by MONAI projects. # Illustrations Current module that I've developed for labelling time-series data for classification purposes: https://user-images.githubusercontent.com/22460517/124283562-0be32400-db1a-11eb-9046-36f435ba8cac.mp4 # Background and References The source code for this module can be found here: https://github.com/SlicerIGT/aigt/tree/master/DeepLearnLive