... BPTT).¹

It should be noted that there is a fixed-size storage algorithm (a hybrid between BPTT and RTRL) with $R_{time} = O(m^2)$ (better than with RTRL, worse than with BPTT) and $R_{space} = O(m^2)$ (much better than with BPTT, same as with RTRL) ([7][4]).

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

... patterns²

Certain biological evidence is consistent with the idea of fast weight changes (`dynamic links', see [6]). In [1], for instance, it is shown that the effective connectivity between certain neurons may change drastically within a few 10 msec.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

... sequence³

The active weight changing capabilities represent a similarity to the system described in [5], which is based on two separate modules - one for learning to control fast weight changes of the other one. Unlike with this previous approach, however, the system described herein does not require two separate modules. It can learn to manipulate its own weights.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

Back to Recurrent Neural Networks page