The fundamental limitations are closely related to those first identified by Gödel [10]: Any formal system that encompasses arithmetics is either flawed or allows for unprovable but true statements. Hence even a Gödel machine with unlimited computational resources necessarily must ignore those self-improvements whose effectiveness it cannot prove. Compare Blum's speed-up theorem [3,4] based on certain incomputable predicates.
Similarly, a realistic Gödel machine with limited resources cannot profit from self-improvements whose usefulness it cannot prove within its time and space constraints.