Automatic Segmentation of Brain Structures
using Multi-Dimensional
Recurrent Neural Networks

Simon Andermatt

Medical Image Analysis Center (MIAC)

### Contents * Voxelwise classification of MrBrains * Recurrent neural networks (RNN) * Adaptation to 3 dimensions * MD-GRU on MrBrains13

MrBrains13 challenge

### RNN * Neural network handling sequences * For each point in sequence: * Input: - Input x at time t - Output of time t-1 (recurrent) * Learned weights w and u (same for each t) * Output: - Output at time t

RNN (typical application)

text to text: translation

RNN (typical application)

unfolded

RNN (typical application)

more likely setup

MD-RNN predecessor information (2D)

Direct predecessor only
(multiplication)

MD-RNN predecessor information (2D)

Including neighborhood of predecessor (convolution)

Inside the black box:

Convolutional Gated Recurrent Unit (C-GRU)

MD-GRU Layer: 1 C-GRU for each direction & dimension

$r^j = \sigma \left( \sum\limits_i^I (x^i*w_r^{i,j}) + \sum\limits_k^J ( h_{t-1}^k*u_r^{k,j} ) +b^j_r\right),$

$z^j = \sigma \left( \sum\limits_i^I (x^i*w_z^{i,j}) + \sum\limits_k^J ( h_{t-1}^k*u_z^{k,j} ) +b^j_z\right),$ $\tilde{h}^j_t = \phi \left( \sum\limits_i^I (x^i*w^{i,j}) + r^j \odot \sum\limits_k^J ( h_{t-1}^k * u^{k,j} ) +b^j\right),$ $h^j_{t} = z^j\odot h^j_{t-1} + (1-z^j)\odot\tilde{h}^j_{t}.$

Network