With the modified automaton, it turns out that conventional recurrent networks fail to learn the correct classifications within training sequences (various topologies were tested). Two related reasons are:
But the ``real'' search space ought to be small, because most possible symbol combinations can never occur. The modification of the automaton did not cause a change in entropy. How can an adaptive system find this out? The next section gives an answer.