Multi-Dimensional Gated Recurrent Units
for the Segmentation of Biomedical 3D-Data

Simon Andermatt, Simon Pezold, and Philippe Cattin

Medical Image Analysis Center


2nd Workshop on Deep Learning in Medical Image Analysis in Conjunction with MICCAI 2016

RNN (Example)

text to text: translation

RNN (Example)

unfolded

RNN (Example)

more likely setup

Vanishing Gradient Problem


TanHLSTMGRU
  • only one activation
  • 3 gates, 1 state, 1 activation
  • 2 gates, 1 state and activation

http://colah.github.io/posts/2015-08-Understanding-LSTMs/

MD-RNN Recurrent Connections (2D)

Direct predecessor only

MD-RNN Recurrent Connections (2D)

Including neighborhood of predecessor (convolution)

Convolutional Gated Recurrent Unit (C-GRU)

MD-GRU Layer: 1 C-GRU for each direction & dimension

$r^j = \sigma \left( \sum\limits_i^I (x^i\color{red}*\color{black}w_r^{i,j}) + \sum\limits_k^J ( h_{t-1}^k\color{red}*\color{black}u_r^{k,j} ) \color{red}+b^j_r\color{black}\right),$ $z^j = \sigma \left( \sum\limits_i^I (x^i\color{red}*\color{black}w_z^{i,j}) + \sum\limits_k^J ( h_{t-1}^k\color{red}*\color{black}u_z^{k,j} ) \color{red}+b^j_z\color{black}\right),$ $\tilde{h}^j_t = \phi \left( \sum\limits_i^I (x^i\color{red}*\color{black}w^{i,j}) + \color{red}r^j \odot\color{black} \sum\limits_k^J ( h_{t-1}^k \color{red}*\color{black} u^{k,j} ) \color{red}+b^j\color{black}\right),$ $h^j_{t} = z^j\odot h^j_{t-1} + (1-z^j)\odot\tilde{h}^j_{t}.$

Network



  • Caffe 1.0 rc3
  • Custom layers using CuDNN v5

MrBrains13 challenge

MD-GRU / MD-LSTM [1]

[1] Stollenga, M.F., Byeon, W., Liwicki, M., Schmidhuber, J.: Parallel Multi- Dimensional LSTM, With Application to Fast Biomedical Volumetric Image Segmentation. Advances in Neural Information Processing Systems 28, pp. 2998–3006. (2015)

MD-GRU / MD-LSTM

MD-GRU / MD-LSTM

MD-LSTM: 12.8 s / MD-GRU: 9.1 s per iteration
(volume of 192 × 192 × 14)

MD-GRU Challenge Results

MD-GRU Challenge Results

[1] Stollenga, M.F., Byeon, W., Liwicki, M., Schmidhuber, J.: Parallel Multi- Dimensional LSTM, With Application to Fast Biomedical Volumetric Image Segmentation. Advances in Neural Information Processing Systems 28, pp. 2998–3006. (2015)

[2] Çiçek, Ö., Abdulkadir, A., Lienkamp, S.S., Brox, T., Ronneberger, O.: 3d U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation. arXiv:1606.06650 [cs] (Jun 2016)

[3] Chen, Hao, et al. VoxResNet: Deep Voxelwise Residual Networks for Volumetric Brain Segmentation. arXiv preprint arXiv:1608.05895 (2016).

Acknowledgements