IJCNN 2011 Tutorial

Autonomous Machine Learning

Asim Roy
Arizona State University
www.lifeboat.com/ex/bios.asim.roy

1. The NSF report on the limitations of our state-of-the-art learning algorithms

Autonomous machine learning has become a priority in the science and engineering of learning. In July 2007, NSF had a workshop on the "Future Challenges for the Science and Engineering of Learning." Here is the summary of the "Open Questions in Both Biological and Machine Learning" from the workshop (http://www.cnl.salk.edu/Media/NSFWorkshopReport.v4.pdf).

"Biological learners have the ability to learn autonomously, in an ever changing and uncertain world. This property includes the ability to generate their own supervision, select the most informative training samples, produce their own loss function, and evaluate their own performance. More importantly, it appears that biological learners can effectively produce appropriate internal representations for composable percepts -- a kind of organizational scaffold - - as part of the learning process. By contrast, virtually all current approaches to machine learning typically require a human supervisor to design the learning architecture, select the training examples, design the form of the representation of the training examples, choose the learning algorithm, set the learning parameters, decide when to stop learning, and choose the way in which the performance of the learning algorithm is evaluated. This strong dependence on human supervision is greatly retarding the development and ubiquitous deployment of autonomous artificial learning systems. Although we are beginning to understand some of the learning systems used by brains, many aspects of autonomous learning have not yet been identified."

This obviously opens the door to developing a new generation of learning algorithms. And IJCNN could become the focal point for research collaboration on this new breed of learning algorithms.

The objective of this tutorial is to present some new ideas regarding brain-like learning to the IJCNN 2011 participants, ideas that can lead to the development of truly autonomous learning methods. Completely autonomous learning is extremely important from the point of view of robotics and computational intelligence. For example, we cannot develop autonomous robots of any type, those that can learn on their own, with learning algorithms that need constant human baby-sitting and human intervention. For autonomous robots, we have to have tweak-free learning algorithms that can design and train computational structures (e.g. neural networks) on their own without any kind of external assistance.

2. Goals:

The tutorial will broadly introduce some new ideas about learning and some new types of learning methods developed over the last few years. Participants will learn about a set of principles for designing and constructing autonomous learning algorithms. There will also be a demonstration of these new autonomous learning algorithms on a variety of problems.

INNS also has a Special Interest Group (SIG) that focuses exclusively on autonomous machine learning (AML SIG). An important objective of the AML SIG is to organize a research group to work on this new breed of learning algorithms. From IJCNNs point of view, this tutorial would be an attempt to grow a set of researchers focused on autonomous learning, which in turn will lead to future IJCNN sessions in this area.

3. Intended audience:

People doing research on learning algorithms, which is the vast majority of IJCNN participants, should be interested in this tutorial. This should be of special interest to students and those in the industry doing research in the area of neural networks and machine learning. They are the ones who will be working on the next generation of learning algorithms that don't depend on human supervision and intervention.

I expect an active exchange of ideas as we venture into this new frontier of research.

4. Tutorial Outline

a) What properties NSF wants in future learning algorithms – on some general properties of biological learning
b) An overview of hypersphere nets without connection weights; similarities to RBF nets;
c) On approximate rule extraction from hypersphere nets
d) Demonstration of an autonomous learning system for pattern classification and discussion of its basic features
e) Problem decomposition and class-specific feature selection; finding the best feature set for each class based on separation maximization principle
f) Feature selection and generalization (minimum error, minimum description length)
g) A new hypersphere classification algorithm; iterative generation of hyperspheres
h) Next generation learning algorithms – no parameters to set, no optimization method used, self-selection of training examples
i) Conclusions

5. A note about the Autonomous Machine Learning (AML) SIG:

INNS formed the Autonomous Machine Learning Special Interest Group (AML SIG) for long term collaboration in this area within the robotics and neural network community. The AML SIG is already bringing a lot of focus to autonomous learning within the broader neural network community through special sessions and panel discussions. AML SIG has a special track within IJCNN with 7 special sessions and 2 panel discussions.

AML SIG now has over 130 members worldwide and everyone is encouraged to join. Here's the link to the AML SIG website: http://autonomoussystems.org/default.html . Email Asim Roy at asim.roy@asu.edu to join the AML SIG.

Biography – Asim Roy

http://lifeboat.com/ex/bios.asim.roy

Asim Roy is a Professor of Information Systems at Arizona State University. He received his M.S. in Operations Research from Case Western Reserve University, Cleveland, Ohio, and Ph.D. in Operations Research from University of Texas at Austin. He has been a Visiting Scholar at Stanford University, visiting Prof. David Rumelhart in the Psychology Department, and a Visiting Scientist at the Robotics and Intelligent Systems Group at Oak Ridge National Laboratory, Oak Ridge, Tennessee.

His research interests are in brain-like learning, neural networks, machine learning, data mining, intelligent systems and nonlinear multiple objective optimization. His research has been published in Management Science, Mathematical Programming, Neural Networks, Neural Computation, various IEEE Transactions (Neural Networks, Fuzzy Systems, Systems, Man and Cybernetics) and other journals. He has been invited to many national and international conferences for plenary talks and for tutorials, workshops and short courses on his new learning theory and methods. He is listed in Who's Who in America, Who's Who in the World, Who's Who in American Education and Who's Who in Industry and Finance, among others.

Physorg.com recently wrote a story on his new brain theory postulating that parts of the brain control other parts and thus control theoretic architectures can be used to design brain-like systems. Here's the link to the story: http://www.physorg.com/news146319784.html