Connectionist Knowledge Representation and Reasoning Tutorial at KI 2005, Koblenz, Germany, September 2005 Barbara Hammer and Pascal Hitzler

Topic

Intelligent systems based on symbolic knowledge processing on the one hand, and on artificial neural networks (also called connectionist systems) on the other, differ substantially. They are both standard approaches to artificial intelligence and it would be very desirable to combine the robust neural networking machinery with symbolic knowledge representation and reasoning paradigms like logic programming in such a way that the strengths of either paradigm will be retained. While significant progress has recently been made on knowledge representation and reasoning using neural networks on the one side and direct processing of symbolic and structured data with neural methods on the other side, the integration of neural computation and expressive logics such as first order logic is still in its early stages of development. In this tutorial, we will present past achievements and present state of the art in neural-symbolic integration. We believe in the potential of neural-symbolic integration for machine learning applications and for furthering fundamental insights into the working of the human mind. The course thus shall display the importance of neural-symbolic integration as a research area and stimulate discussions and a strengthening of the field.

Intended Audience

The tutorial is aimed at researchers and PhD students interested in getting to know the recently evolving field of neural-symbolic integration research. It is designed to suit both the practitioner interested in a new machine learning paradigm and its application to areas such as bioinformatics, as well as the scientist interested in understanding the knowledge processing abilities of the human mind from a formal perspective.

Contents: Overview

We will present past and present achievements in neural-symbolic integration. Starting from successful achievements for propositional logic and integration of symbolic information into recursive neural systems as well as principled limitations and problems, we will work towards discussing state-of-the-art research on the integration of first-order logic programming and connectionism, based on recent research publications by the lecturers. We believe in the potential of neural-symbolic integration for machine learning applications and for furthering fundamental insights into the working of the human mind. The course thus shall display the importance of neural-symbolic integration as a research area and stimulate discussions and a strengthening of the field.

Lecture slides

Introduction (ppt, 614 KB)
Part 1 (ppt, 1.311 KB)
Part 2 (zipped ps, 3.501 KB)

Further Reading

Part 1

• Complete list of references for part 1.
• FNN and rules: Duch,Setiono,Zurada,Computational intelligence methods for understanding of data, Proc. of the IEEE 92(5):771- 805, 2004
• Structure kernels: Gärtner, Lloyd, Flach, Kernels and distances for structured data, Machine Learning, 57, 2004. (new overview is forthcoming)
• RNNs: Hammer, Steil, Perspectives on learning with recurrent networks, in: Verleysen, ESANN'2002, D-side publications, 357-368, 2002
• RNNs and rules: Jacobsson, Rule extraction from recurrent neural networks: a taxonomy and review, Neural Computation, 17:1223-1263, 2005
• Recursive representations: Hammer, Perspectives on Learning Symbolic Data with Connectionistic Systems, in: Kühn, Menzel, Menzel, Ratsch, Richter, Stamatescu, Adaptivity and Learning, 141-160, Springer, 2003.
• Recursive networks: Frasconi, Gori, Sperduti, A General Framework for Adaptive Processing of Data Structures, IEEE Transactions on Neural Networks, 9(5):768-786,1998
• Neural networks and structures: Hammer, Jain, Neural methods for non-standard data, in: Verleysen, ESANN'2004, D-side publications, 281-292, 2004

Part 2

• Survey with plenty of references: Sebastian Bader and Pascal Hitzler, Dimensions of neural-symbolic integration - a structured survey. Submitted. [ps (paper)][zipped ps (paper)].
• Sebastian Bader, Artur S. d'Avila Garcez and Pascal Hitzler, Computing First-Order Logic Programs by Fibring Artificial Neural Networks. In:I. Russell, Z. Markov (Eds.): Proceedings of the Eighteenth International Florida Artificial Intelligence Research Symposium Conference, Clearwater Beach, Florida, USA. AAAI Press 2005, May 2005, pp. 314-319.
• Sebastian Bader and Pascal Hitzler, Logic Programs, Iterated Function Systems, and Recurrent Radial Basis Function Networks, Journal of Applied Logic, 2004.
• Sebastian Bader, Pascal Hitzler, and Steffen Hölldobler, The Integration of Connectionism and First-Order Knowledge Representation and Reasoning as a Challenge for Artificial Intelligence, Information 8(5), 2005, to appear.
• Sebastian Bader, Pascal Hitzler and Andreas Witzel, Integrating First Order Logic Programs and Connectionist Systems - A Constructive Approach, In: Proceedings of the IJCAI-05 Workshop on Neural-Symbolic Learning and Reasoning, NeSy'05, Edinburgh, UK, August 2005.
• A. S. d'Avila Garcez, Krysia Broda and Dov M. Gabbay, Symbolic knowledge extraction from trained neural networks: A sound approach, Artificial Intelligence, 2001.
• A. S. d'Avila Garcez and G. Zaverucha, The Connectionist Inductive Learning and Logic Programming System, Applied Intelligence, 1999.
• Pascal Hitzler, Sebastian Bader and Artur S. d'Avila Garcez, Ontology leaning as a use case for neural-symbolic integration, In: Proceedings of the IJCAI-05 Workshop on Neural-Symbolic Learning and Reasoning, NeSy'05, Edinburgh, UK, August 2005.
• Pascal Hitzler, Steffen Hölldobler and Anthony K. Seda. Logic Programs and Connectionist Networks. Journal of Applied Logic, 2004.
• Steffen Hölldobler and Yvonne Kalinke, Towards a Massively Parallel Computational Model for Logic Programming, ECAI, 1994.
• Steffen Hölldobler, Yvonne Kalinke and Hans-Peter Störr, Approximating the Semantics of Logic Programs by Recurrent Neural Networks, Applied Intelligence, 1999.
• Jens Lehmann, Sebastian Bader and Pascal Hitzler, Extracting reduced logic programs from artificial neural networks, In: Proceedings of the IJCAI-05 Workshop on Neural-Symbolic Learning and Reasoning, NeSy'05, Edinburgh, UK, August 2005.
• Anthony Seda and Maire Lane, On Approximation in the Integration of Connectionist and Logic-Based Systems, In: Proceedings of the Third International Conference on Information, Tokyo, 2004

Presenters Information

Prof. Dr. Barbara Hammer has accepted a call for a professorship for Theoretical Computer Science at Clausthal University of Technology in late 2005. Beforehand, she has been the leader of a young researchers group with research topic 'Learning with Neural Methods on Structured Data' at the University of Osnabrück which has been funded by the MWK Niedersachsen. Her research interests focus on possibilities to directly process and represent structures by means of connectionist methods, the development and exact mathematical investigation of involved models, as well as applications ranging from bioinformatics up to industrial cooperations. Her research record includes more than 70 publications (including two books) covering diverse topics regarding mathematics of neural networks, processing of structured data, structural bias of learning, compositionality, learning metrics, self-organization etc. She is a member of the editorial board of Neurocomputing, a member of the program committee of the ESANN conference, and she has organized several special sessions and a special issue connected to the topic of the tutorial jointly with colleagues. Besides, she serves as reviewer for various international journals and conferences. Since 1999, she has tought courses connected to Neural Networks, Machine Learning, Softcomputing, and other topics of Computer Science at the Universities of Osnabrueck and Clausthal.

Dr. Pascal Hitzler is project leader and researcher at the Institute for Applied Informatics and Formal Description Methods (AIFB) at the University of Karlsruhe in Germany. His research record lists over 50 publications in such diverse areas as neural-symbolic integration, knowledge representation and reasoning, semantic web, lattice and domain theory, denotational semantics, and set-theoretic topology. He is co-organizer of the IJCAI-05 workshop on Neural-symbolic Learning and Reasoning (NeSy'05). At ESSLLI'2005, he will give a tutorial on Integrating logic programs and connectionist systems. He has taught seminars and lectures in the field of the tutorial at TU Dresden since 2002. He serves as a reviewer for international journals, conferences, and research project applications. He has also been an organizer of international enhancement programmes for highly skilled students in Mathematics and Computer Science, and has served as an editor for several books in this area.