Aarhus University Seal

Predicting embryo viability based on self-supervised alignment of time-lapse videos

Research output: Contribution to journal/Conference contribution in journal/Contribution to newspaperJournal articleResearchpeer-review


  • Mikkel Fly Kragh, Vitrolife A/S
  • ,
  • Jens Rimestad, Vitrolife A/S, Denmark
  • Jacob Theilgaard Lassen, Vitrolife A/S, Denmark
  • Jørgen Berntsen, Vitrolife A/S, Denmark
  • Henrik Karstoft

With self-supervised learning, both labeled and unlabeled data can be used for representation learning and model pretraining. This is particularly relevant when automating the selection of a patient's fertilized eggs (embryos) during a fertility treatment, in which only the embryos that were transferred to the female uterus may have labels of pregnancy. In this paper, we apply a self-supervised video alignment method known as temporal cycle-consistency (TCC) on 38176 time-lapse videos of developing embryos, of which 14550 were labeled. We show how TCC can be used to extract temporal similarities between embryo videos and use these for predicting pregnancy likelihood. Our temporal similarity method outperforms the time alignment measurement (TAM) with an area under the receiver operating characteristic (AUC) of 0.64 vs. 0.56. Compared to existing embryo evaluation models, it places in between a pure temporal and a spatio-temporal model that both require manual annotations. Furthermore, we use TCC for transfer learning in a semi-supervised fashion and show significant performance improvements compared to standard supervised learning, when only a small subset of the dataset is labeled. Specifically, two variants of transfer learning both achieve an AUC of 0.66 compared to 0.63 for supervised learning when 16% of the dataset is labeled.

Original languageEnglish
JournalIEEE Transactions on Medical Imaging
Pages (from-to)465-475
Number of pages11
Publication statusPublished - Feb 2022

    Research areas

  • Annotations, Clustering, Embryo, Embryo Selection, In vitro fertilization, Manuals, Pregnancy, Self-supervised learning, Supervised learning, Task analysis, Temporal Cycle-Consistency, Videos, Temporal cycle-consistency, Embryo selection, self-supervised learning, temporal cycle-consistency, in vitro fertilization, embryo selection

See relations at Aarhus University Citationformats

ID: 225444523