AUTO-ENCODERS

With pattern $p$ and classifier $T_l$ a reconstructor module $A_l$ (another back-prop network) receives $y^{p,l}$ as an input. The combination of $T_l$ and $A_l$ functions as an auto-encoder. The auto-encoder is trained to emit the reconstruction $h^{p,l}$ of $T_l$'s external input $x^{p,l}$, thus forcing $y^{p,l}$ to tell something about $x^{p,l}$. $D_l$ is defined as

$$D_l = \frac{1}{2} \sum_p \Vert h^{p,l} - x^{p,l} \Vert^2.$$ (7)