Amnon H. Eden
Master thesis, Department of Computer Science, Tel Aviv University
Publication year: 1994
This research focused on learning ‘natural concepts’: creating an agent capable of recognising noisy categories such as ‘chair’ and ‘bird’, which have no¬†crisply defined boundaries but which humans can easily identify without¬†explicit, formal definitions.
Early machine learning models attempted to employ GOFAI (good old-fashioned AI) techniques, which represent concepts in symbolic logic and use formal reasoning to learn natural concepts, without much success. Instead, we believe, an effective agent may may build upon lessons learned from the human ability to perform classification tasks quickly and effectively despite moderate noise. A more successful learning agent must therefore borrow from the interdisciplinary literature on concept formation and models of representation from cognitive psychology and the philosophy of mind corpora.
This paper begins with a comparative study of of existing supervised learning methods, such as decision trees, neural networks, bayesian methods, and semantic networks, and their effectiveness of learning natural concepts. We examine concept acquisition and concept formation models in cognitive psychology, focusing on prototype-based and examplar-based learning. We continue with lessons learned from the philosophical examination of the nature of natural concepts, including notions of family resemblance and polymorphism, which our learning agent must account for.
We describe an instance-based learning model which builds upon the lessons learned about the human performance in natural concept learning tasks, implemented in C++. We conduct an empirical study comparing the performance of our learning model with existing machine learning algorithms in two learning tasks:
  • Wine recognition
    This task required learning to classify 158 samples of wines grown by 3 different producers of the same area of Italy in Genoa, Italy using data obtained from the UCI Machine Learning Repository describing the wines’ visual properties (such as colour intensity and hue) and their chemical composition (such as alcohol, phenols, and Malic acid), originating from the Institute of Pharmaceutical and Food Analysis and Technologies in Genoa.
    Empirical results show that our algorithms achieved 0.99 success rate in the task, which improves over the success rate achieved by the best instance-based machine learning model available at the time (IB1), which was 0.94.
  • Character recognition
    This task required classifying 20,000 pixel images of distorted letters in script, italic, serif, and Gothic uppercase typefaces generated from different commercial fonts. The data was generated and shared by Peter W. Frey of the Department of Psychology, Northwestern University and David J. Slate of the Pattern Recognition Group, Odesta Corporation.
    Empirical results show that our algorithms achieved 0.75, 0.8 and 0.82 success rates, which improve significantly on the adaptive classifier system developed by Frey & Slate (1991) whose success rate was 0.28, 0.51 and 0.82 and required the generation of hundred of thousands rules.