Automatic Tool Annotation for Surgical Workflow Analysis

Surgical tool detection is attracting increasing attention from the medical image processing community [1]. The goal generally is not to precisely locate tools in images, but rather to indicate which tools are being used by the surgeon at each instant. The main motivation for annotating tool usage is to design efficient solutions for surgical workflow analysis. Analyzing the surgical workflow has potential applications in report generation, surgical training and even real-time decision support.

Cataract Surgery

Most existing tool annotation algorithms focus on laparoscopic surgeries [1]. However, the most common surgical procedure in the world is cataract surgery: 19 million cataract surgeries are performed annually [2]. A cataract is a clouding of the lens, inside the eye, which leads to a decrease in vision. The purpose of the surgery is to remove this lens and replace it with an artificial lens. The entire procedure can be performed with small incisions only. The natural lens is indeed broken into small pieces with the help of a high-frequency ultrasound device before leaving the eye. As for the artificial lens, it is rolled up inside an injector before entering the eye. The surgery typically lasts 15 minutes. During the surgery, the patient’s eye is under a microscope and the output of the microscope is video-recorded.

Purpose of this Challenge

This challenge was organized to evaluate existing and new tool annotation algorithms in the specific context of cataract surgery. It relies on more than nine hours of videos, from 50 cataract surgeries, in which the presence of 21 surgical tools was manually annotated by two experts. We hope that this competition will stimulate research on automatic tool annotation for this procedure.

Note that this competition is not associated with a conference. It will be open for eight months (from April 1, 2017 to November 30, 2017). At the end of this period, the designers of the top ten algorithms will be invited to contribute to a joint journal paper describing and summarizing the methods used and the results found in the challenge. The paper will be submitted to a high-impact journal of the field.

How to Participate?

  1. Teams must register to participate in the challenge.
  2. After registration, the dataset can be downloaded. The content of this dataset is described on this page.
  3. Tool annotation results can be submitted. The format of submissions is described on this page.
  4. The submitted results will be processed and will be published on the 'results' page.


[1] Bouget D, Allan M, Stoyanov D, Jannin P. Vision-based and marker-less surgical tool detection and tracking: a review of the literature. Med Image Anal. 2017 Jan;35:633-654.

[2] Trikha S, Turnbull AM, Morris RJ, Anderson DF, Hossain P. The journey to femtosecond laser-assisted cataract surgery: new beginnings or false dawn? Eye. 2013; 27:461–473.