Daan Wierstra

Research Description

My research has focused on reinforcement learning in non-Markovian environments, long-term dependency sequence processing using internal state, and the application thereof to robotics. My interests include spiking neuron models, genetic algorithms and the theoretical properties of evolution strategies, policy gradient based learning methods, and, ultimately, proper, well-founded research into artificial general intelligence.

Publications

• Daan Wierstra, Alexander Foerster, Jan Peters and Juergen Schmidhuber (2009). Recurrent Policy Gradients Logic Journal of the IGPL, in press.
• Sun Yi, Daan Wierstra, Tom Schaul and Juergen Schmidhuber (2009). Stochastic Search using the Natural Gradient. Proceedings of the 26th International Conference on Machine Learning (ICML-09), Montreal.,
• Sun Yi, Daan Wierstra, Tom Schaul and Juergen Schmidhuber (2009). Efficient Natural Evolution Strategies. Genetic and Evolutionary Computation Conference (GECCO-09), Montreal.
• Justin Bayer, Daan Wierstra, Julian Togelius and Juergen Schmidhuber (2009), Evolving memory cell structures for sequence learning. Proceedings of the 19th International Conference on Artificial Neural Networks (ICANN-09), Cyprus.
• Julian Togelius, Tom Schaul, Daan Wierstra, Christian Igel, Faustino Gomez and Juergen Schmidhuber (2009). Ontogenetic and Phylogenetic Reinforcement Learning. Kuenstliche Intelligenz, in press.
• Niels van Hoorn and Julian Togelius and Daan Wierstra and Juergen Schmidhuber. Robust player imitation using multiobjective evolution. Proceedings of the Congress on Evolutionary Computation (CEC-09), Trondheim.
• Hermann Mayer, Faustino Gomez, Daan Wierstra, Istvan Nagy, Alois Knoll, and Juergen Schmidhuber (2008). A System for Robotic Heart Surgery that Learns to Tie Knots using Recurrent Neural Networks. Advanced Robotics, 22(13-14), pp 1521-1537.
• Daan Wierstra, Tom Schaul, Jan Peters and Juergen Schmidhuber (2008). Episodic Reinforcement Learning by Logistic Reward-Weighted Regression. Proceedings of the International Conference on Artificial Neural Networks (ICANN-2008, Prague).
• Daan Wierstra, Tom Schaul, Jan Peters and Juergen Schmidhuber (2008). Fitness Expectation Maximization. Proceedings of Parallel Problem Solving from Nature (PPSN-2008, Dortmund).
• Daan Wierstra, Tom Schaul, Jan Peters and Juergen Schmidhuber (2008). Natural Evolution Strategies. Proceedings of IEEE Congress on Evolutionary Computation (CEC-2008, Hongkong).
• Juergen Schmidhuber, Daan Wierstra, Matteo Gagliolo, and Faustino Gomez (2006). Training Recurrent Neural Networks by Evolino. Neural Computation, 19(3): 757-779, 2007
• Daan Wierstra and Juergen Schmidhuber (2007). Policy Gradient Critics. In Proceedings of the European Conference on Machine Learning (ECML-07, Berlin).
• Daan Wierstra, Alexander Foerster, Jan Peters and Juergen Schmidhuber (2007). Solving Deep Memory POMDPs with Recurrent Policy Gradients. In Proceedings of the International Conference on Neural Networks (ICANN-07, Porto).
• Hermann Mayer, Faustino Gomez, Daan Wierstra, Istvan Nagy, Alois Knoll, and Juergen Schmidhuber (2006). A System for Robotic Heart Surgery that Learns to Tie Knots Using Recurrent Neural Networks. In Proceedings of the International Conference on Intelligent Robotics and Systems (IROS-06, Beijing).
• Juergen Schmidhuber, Matteo Gagliolo, Daan Wierstra, and Faustino Gomez (2006). Evolino for Recurrent Support Vector Machines. In Proceedings of the European Symposium on Artificial Neural Networks (ESANN-06, Bruge).
• Daan Wierstra, Faustino Gomez, and Juergen Schmidhuber (2005). Modeling Systems with Internal State using Evolino. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-05, Washington, D.C.). Winner of Best Paper Award in Learning Classifier Systems and Other Genetics-Based Machine Learning.
• Juergen Schmidhuber, Daan Wierstra, and Faustino Gomez (2005). Evolino: Hybrid Neuroevolution / Optimal Linear Search for Sequence Learning. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI-05, Edinburgh).
• Daan Wierstra and Marco Wiering (2004). Utile Distinction Hidden Markov Models. In Proceedings of the International Conference on Machine Learning (ICML-04, Banff, Canada).

Master's Thesis

• Daan Wierstra (2004). Statistical Models for Non-Markovian Control Tasks, Utrecht University, 2004.