Copyright © 2007 The Institute of Electronics, Information and Communication Engineers
Regular Section -- Papers -- Algorithm Theory |
Incremental Leaning and Model Selection for Radial Basis Function Network through Sleep
1 The authors are with the Graduate School of Information Science and Technology, Hokkaido University, Sapporo-shi, 060–0814 Japan. E-mail: yamauchi{at}complex.eng.hokudai.ac.jp
 |
Abstract |
|---|
The model selection for neural networks is an essential procedure to get not only high levels of generalization but also a compact data model. Especially in terms of getting the compact model, neural networks usually outperform other kinds of machine learning methods. Generally, models are selected by trial and error testing using whole learning samples given in advance. In many cases, however, it is difficult and time consuming to prepare whole learning samples in advance. To overcome these inconveniences, we propose a hybrid on-line learning system for a radial basis function (RBF) network that repeats quick learning of novel instances by rote during on-line periods (awake phases) and repeats pseudo rehearsal for model selection during out-of-service periods (sleep phases). We call this system Incremental Learning with Sleep (ILS). During sleep phases, the system basically stops the learning of novel instances, and during awake phases, the system responds quickly. We also extended the system so as to shorten the periodic sleep periods. Experimental results showed the system selects more compact data models than those selected by other machine learning systems.
Key Words: incremental learning, sleep learning, model selection, RBF
Manuscript received December 21, 2005. Manuscript revised August 22, 2006.