next up previous
Next: REDUNDANCY REDUCTION FOR SEQUENCES Up: THE DIFFICULT TASK Previous: THE DIFFICULT TASK

SIMULATION

With the modified automaton, it turns out that conventional recurrent networks fail to learn the correct classifications within $10^7$ training sequences (various topologies were tested). Two related reasons are:

But the ``real'' search space ought to be small, because most possible symbol combinations can never occur. The modification of the automaton did not cause a change in entropy. How can an adaptive system find this out? The next section gives an answer.



Juergen Schmidhuber 2003-02-19