Left Ventricle Full Quantification Challenge MICCAI 2019

An extension of Left Ventricle Full Quantification Challenge MICCAI 2018

In conjunction with the STACOM 2019 Workshop

Challenge Description

This challenge is an extension of Left Ventricle Full Quantification Challenge MICCAI 2018 (LVQuan18), the main difference is that this challenge (LVQuan19) will provide original data without preprocessing for training and testing phases, which is more clinical than the data providing by LVQuan18. The challenge addresses the analysis of left ventricles (LV) in patients undergoing cardiac MRI.


To assess the heart's function, the LV's function, morphology and temporal dynamics is of clinical interest. This analysis in clinical routine is time consuming and prone to error and inter-observer variability. In this challenge the extraction of the LV's cavity and myocardium and subsequently the regression of regional wall thicknesses, LV dimensions and the classification of the phase of the cardiac cycle are to be performed. These are common and significant parameters to assess the LV function.


The aim of this challenge is to learn effective machine learning models that can estimate a set of clinical significant LV indices (regional wall thicknesses, cavity dimensions, area of cavity and myocardium, cardiac phase) directly from MR images. No intermediate segmentation is required in the whole procedure.


  • The training dataset is available via E-mail: lvquan19@outlook.com
  • DATA DESCRIPTION

    Features of Training and test dataset will be descripted below.

    Training Dataset


    A training dataset with processed SAX MR sequences of 56 subjects from clinical environment is used for model learning and validation. For each subject, 20 frames are included for the whole cardiac cycle. All ground truth values of the above-mentioned LV indices are provided for every single frame. With the training dataset, we encourage the participants to report the performance of their algorithms with N-fold cross validation, where the following configuration are used :

    N=3, the number of each folds are (18, 19, 19);
    N=5, the number of each folds are (11, 11, 11, 11, 12).

    In such a way, we can have initial impression of the algorithms’ stability. The training dataset is available via
    E-mail: lvquan19@outlook.com (in order to count the number of participants)


    Test Dataset


    A test dataset with processed SAX MR sequences of 30 subjects will be provided to participants according to the challenge schedule for algorithms assessment and ranking. The processing procedure is the same as that in the training dataset. For each subject, only the SAX image sequences of 20 frames are provided, while their ground truth values are not.

    Important dates

    Objective

    Developing reliable and accurate automatic method for full quantification of cardiac LV for each frame in the short axis view cardiac MR sequences. The quantifications include the following LV indices.


    lv_quan19

    Name Description
    A1, A2 cavity area and myocardium area, in mm^2.
    D1~D3 dimensions of cavity of three directions (IS-AL, I-A, and IL-AS) in mm.
    RWT1~RWT6 regional wall thickness in mm, staring from the anterior-septal segment in counter clockwise direction, i.e., IS, I, IL, AL, A, AS.
    Phase binary variable for cardiac phase, 1 means systolic phase and 0 means diastolic.


    For more details about left ventricle quantification, the following two references are recommended:


    Evaluation Criterion

    Metrics


    • Mean Absolute Error (MAE) and Pearson correlation coefficient (PCC) are used to assess the performance of the algorithms for estimation of areas, dimensions and regional wall thicknesses. For the 30 subjects in the test dataset, there are a total of 600 images. For any LV indices in {A1, A2, D1~D3, RWT1~RWT6}, the MAE and PCC can be computed as:





      where , is the groud truth value and is the estimated value, and are their mean values.


    • Error Rate (ER) is used to assess the performance of the algorithms for cardiac phase identification. It can be computed as:



      where is the indication function, and are the ground truth value and the estimated value of cardiac phase, respectively.


    Ranking


    MAE and ER are used for the final ranking of all participants.


    1. Initialization:

    2. 1. For each participant, compute for and .

    3. 2. Compute the mean MAE values for areas, dimensions, and regional wall thicknesses, and obtain , , .

    4. 3. Build four ranking lists (, , , ) of all participants according to , , and in ascending order, respectively. Statistical test is conducted for areas, dimensions, and regional wall thicknesses to test the significance of difference between two participants. If no significance exists, the two participants will have the same rank in the list.

    5. 4. The final ranking list is computed by sorting the summation of the four ranking lists above in ascending order.

    Paper Submission

    Workshop paper


    All participants are encouraged to submit a full workshop paper describing their algorithms and results. Manuscript up to 8 pages should follow the template of main conferences’ paper and be submitted via the STACOM submission system.


    One-page abstract


    Participants who cannot finish the task on time are encouraged to complete the task before 6 September 2019, with the one-page abstract submitted to the organizer via email (lvquan19@outlook.com). The results on test data should be sent to the organizer for performance evaluation before 1 September 2019.


    Challenge ranking


    Successful participants with workshop paper will be ranked according the assessment criterion.


    Full ranking


    A full ranking list including all the participants who completed the LV quantification task, with either workshop paper or one-page abstract, will also be announced during the workshop.

    Result Submission

    Details about the final result submission.

    How


    Send algorithm output on the test dataset to organizers via email (lvquan19@outlook.com).


    Format


    Results of test dataset can be submitted in a MATLAB data file (.mat file), with the following protocols.

    Pay attention! Don't forget to convert the results to real physical values(with rwt,dims in mm, and areas in mm^2).


  • physical value of rwt= rwt*pix_spacing*80
  • physical value of dims= dims*pix_spacing*80
  • physical value of areas= areas*(pix_spacing*80)^2

    1. File Name: LVQUAN_XXX.mat. XXX is the initials of the participant’s name.


    2. MATALB data: the submitted results should be in the same format as the ground truth in the training dataset. Four variables should be contained in the LVQUAN_XXX.mat file:

      • areas_hat:
        • array of size of 2×600.
        • areas(1,:) is for cavity area and areas(2,:) is for myocardium area.
        • For the 20 frames of subject s, the results should be areas(:, (s-1)×20:s×20).

      • dims_hat:
        • array of size of 3×600.
        • dims(1,:) is for direction of IS-AL, dims(2,:) is for direction of I-A, and dims(3,:) is for direction of IL-AS.
        • For the 20 frames of subject s, the results should be dims(:, (s-1)×20:s×20).

      • rwt_hat:
        • array of size of 6×600.
        • areas(1,:)~areas(6,:) are for segments IS, I, IL, AL, A and AS, accordingly.
        • For the 20 frames of subject s, the results should be rwt(:, (s-1)×20:s×20).

      • lv_phase_hat:
        • array of size of 1×600.
        • For the 20 frames of subject s, the results should be lv_phase(:, (s-1)×20:s×20).

    Award and Publication

    • Multiple awards will be selected according to the ranking list of all participants.

    • Selected challenge papers will be published with the STACOM proceedings in Lecture Notes in Computer Science, Springer.

    • A potential publication in MedIA or IEEE TMI which summarizes the benchmarked algorithms will be completed and all participants share the credit.

    • Excellent works will be recommended to special issue (Advances in Artificial Intelligence for Cardiac Imaging) in journal of Computerized Medical Imaging and Graphics: Please click here for more details.

    Organizer

    • Guanyu Yang (yang.list@seu.edu.cn), Lab of Image Science and Technology, School of Computer Science, Southeast University, Nanjing, China

    • Tiancong Hua (hua1009464760@outlook.com), Lab of Image Science and Technology, School of Computer Science, Southeast University, Nanjing, China

    • Wufeng Xue (xwolfs@hotmail.com), School of Biomedical Engineering, Shenzhen University, Shenzhen, China

    • Shuo Li (slishuo@gmail.com), Western University, ON, Canada

    Contact

    • For any questions about the challenge, feel free to contact: E-mail: lvquan19@outlook.com