next up previous
Next: EXAMPLE 3: Discovering Factorial Up: EXAMPLE 2: Text Compression Previous: SIMULATIONS

ON-LINE METHODS / LIMITATIONS

A disadvantage of the offline technique above is that it is offline: the predictor does not adapt to the specific text file it sees. This limitation is not essential, however. It is straight-forward to construct an online variant of the approach. With the online variant, the predictor continues to learn during compression. The online variant proceeds like this: Both the sender and the receiver start with exactly the same initial predictor. Whenever the sender sees a new character, it encodes it using its current predictor. The code is sent to the receiver who decodes it. Both the sender and the receiver use exactly the same learning protocol to modify their weights. This implies that the modified weights need not be sent from the sender to the receiver and do not have to be taken into account to compute the average compression ratio. Of course, the online method promises higher compression ratios than the offline method.

The main disadvantage of both online and offline variants, however, is their computational complexity. The current offline implementation is clearly slower than conventional standard techniques, by about three orders of magnitude (but no attempt was made to optimize the code with respect to speed). And the complexity of the online method is even worse (the exact slow-down factor depends on the precise nature of the learning protocol, of course). For this reason, especially the promising online variants can be recommended only if special neural net hardware is available. Note, however, that there are many commercial data compression applications which rely on specialized electronic chips.

Among the methods presented in this paper, perhaps the most interesting way of using neural predictors for redundancy reduction is the one described in the next section.


next up previous
Next: EXAMPLE 3: Discovering Factorial Up: EXAMPLE 2: Text Compression Previous: SIMULATIONS
Juergen Schmidhuber 2003-02-19