ARCHITECTURE 1

Figure 2 shows a static subgoal generator $S$ (a feed-forward back-prop net, e.g. [Werbos, 1974]). With problem $p$, the input vector of $S$ is $s^p \circ g^p$.

The output of $S$ is

$$s^p(1) \circ s^p(2) \circ \ldots \circ s^p(n).$$

$n+1$ copies of $E$ need to be connected to $S$ such that the input of the $k$-th copy of $E$ is equal to $s^p(k-1) \circ s^p(k)$. The output of the $k$-th copy of $E$ is $eval(s^p(k-1), s^p(k))$.

Figure 2: An adaptive non-recurrent subgoal generator emitting two subgoals. Three copies of the differentiable evaluation module are required to compute the proper gradient. Check out Schmidhuber's Habilitation thesis for pictures.

Back to Subgoal learning - Hierarchical Learning
Pages with Subgoal learning pictures