Next: Intuitive explanation of equation Up: Exponential error decay Previous: Gradients of the error

### Error path integral

Suppose we have a fully connected net whose non-input unit indices range from 1 to . Let us focus on local error flow from output unit to arbitrary unit (later we will see that the analysis immediately extends to global error flow). The error occurring at at time step is propagated back in time'' for time steps, to an arbitrary unit at time . This scales the error by the following factor:
 (1)

In order to solve the above equation, we will expand it by unrolling over time (as done for example in deriving BPTT). In particular, for let denote the index of a generic non input unit in the replica of the network at time . Moreover, let and . We obtain:
 (2)

(proof by induction). It can be immediately shown that if the local error vanishes, then the global error vanishes too. To see this compute

where denotes the set of output units.

Next: Intuitive explanation of equation Up: Exponential error decay Previous: Gradients of the error
Juergen Schmidhuber 2003-02-19