Introduction to Neural Networks

by Jeannette Lawrence
5th edition, January, 1993 ISBN# 1-883157-00-5
Copyright 1988-1998 California Scientific, Nevada City, CA 95959. All rights reserved.

This book introduces you to neural networks, expert systems, and fuzzy systems. This work represents six years of research and experience in the field of artificial intelligence, particularly neural networks.

Why Read this Book?
Many researchers, in an attempt to assemble a detailed working model of the mind, have contributed to the field of neural network research. The material available today reflects the complexity of their research, and consequently is not easy to understand without a degree in mathematics, biology, or computer science. Expert systems, sometimes glamorized as an incredibly intelligent and complex technology, are actually not so difficult to understand. However, easy-to-read introductory material on expert systems is also surprisingly scarce.

We try to alleviate some of the complexity surrounding neural networks and expert systems by simplifying highly technical explanations and relating them to common knowledge whenever possible. We use consistent terminology based on biology, and we include a complete glossary of terms. We use a neural network drawing convention which is a conglomerate of those used by several of the foremost researchers.

This book provides background on the subjects of fuzzy logic, computer science, artificial neural network theory, and biology. You don't have to read about them in order to make use of neural networks or experts systems, but they're available if you're interested. You will learn to identify when and why these two forms of computer intelligence obtain the best results. You may find that a combination of technologies provides much more problem solving power than any one alone. Whether you're curious about how the mind operates, how an expert system works, or how to apply these new technologies, this book is designed to answer your questions in a simple, straightforward manner. We introduce new terminology only as needed, explain it thoroughly, use simple diagrams generously, and provide several appendices of supplemental information. We provide detailed examples of several working neural network applications as well as rules of thumb for the designer. This will give you a good background for working with neural networks and expert systems.

"Introduction to Neural Networks (the book included with the program) could and should be used for a college-level introductory course on neural networks. It is clear, concise and does not talk down to the reader."
- James G. Yearwood, Personal Computing Magazine

Introduction to Neural Networks Table of Contents

1. Computer Intelligence
   1. Introduction
   2. What is Computer Intelligence?
   3. What are Neural Networks?
   4. Neural Network Applications
   5. Designing a Neural Network
   6. Rebirth of Neural Network Research
   7. What are Expert Systems?
   8. What are Fuzzy Systems?
   9. What are Fuzzy Sets?
2. Computing Methods for Simulating Intelligence
   1. Introduction
   2. Conventional Computing Methods
   3. Neural Networks vs. Conventional Computing
   4. Example: Learning the Alphabet
   5. Why Use a Neural Network?
   6. Faster Neural Networks
   7. Parallel Processing Computers
   8. Expert Systems vs. Neural Networks
   9. Why Use an Expert System?
   10. Using both Neural Networks and Expert Systems
3. Expert Systems
   1. Rules, Facts, and Inference
   2. How Does an Expert System Work?
   3. Overview of Expert System Design
   4. Project Evaluation and Planning
   5. Knowledge Engineering
   6. Structured Design
   7. Interfaces
   8. Debugging and Delivery
   9. Liability with Expert Systems
4. Probability in Expert Systems
   1. Probability
   2. Bayesian Analysis
   3. Dempster-Shafer Theory
   4. Certainty Factors
   5. Decision Matrices
5. Fuzzy Logic
   1. Introduction
   2. Background and History
   3. Fuzzy Set Theory
   4. Fuzzy Logic
   5. Fuzziness in Neural Networks
   6. Fuzziness in Expert Systems
6. Neural Network Theory
   1. Introduction
   2. A Little Biology
   3. Neurons
   4. Layers
   5. Connections
   6. Learning Methods
   7. Drawing Conventions and Terminology
   8. Activation and Transfer Functions
   9. Learning Rules
7. Neural Network Models
   1. Introduction
   2. Development of Neural Network Theories
   3. Classifying Neural Networks
   4. Feedback Networks
   5. Computational Energy in Feedback Networks
   6. Feed Forward Networks
   7. Supervised Learning
   8. Unsupervised Learning
   9. Cooperation and Competition
   10. Characteristics of Models
   11. Feedback Models
   12. Feed Forward Models
   13. Advantages and Disadvantages of Various Models
8. Popular Feed Forward Models
   1. Introduction
   2. Back Propagation
   3. Probabilistic Neural Networks
   4. Basis Functions
9. Brains, Learning, and Thought
   1. Introduction
   2. Nerve Cell Biology
   3. Learning at the Neuron Level
   4. The Brain
   5. Behavior, Learning, and Thought Processes
10. Neural Network Design Process
    1. Introduction
    2. The Design Process
    3. Building a Network
    4. Training, Testing, and Running
    5. What Neural Networks Can and Cannot Do
    6. The Five-Step Design Process
    7. Example 1: Private Plane Flight Status
    8. Example 2: Football Winners
    9. Example 3: A Financial Predictor
11. Data Preparation
    1. Introduction
    2. It's the Data that Counts
    3. Non-Distributed vs. Distributed Information
    4. Thinking Like a Neural Network
    5. Continuous and Binary Data
    6. Actual Values vs. Changes in Values
    7. A Data Transformation Example
    8. Collecting a Training Set
    9. Image Data
    10. Waveforms
    11. Designing for Certainty Checking
    12. Interpreting the Network Output
12. Advanced Design Topics
    1. Introduction
    2. Training Events
    3. Equal Rights for Training and Testing
    4. The Lean, Mean Neural Machine
    5. Hidden Neurons: How Many is Best?
    6. Hidden Layers: More is Not Better
    7. Noise Can Be Good
    8. Keeping the Network Up to Date
    9. How to Prevent Neural Network Drop Out
    10. Experimental Territory
    11. Confusing Data Representations
    12. Focusing the Network on the Important Data
    13. Understanding the Neural Network's Knowledge
13. Real Estate Appraisal Application
    1. Introduction
    2. The Problem
    3. The Data
    4. Training Experiments
    5. What the Trained Network Can Tell Us
    6. Conclusion
14. Financial Forecasting Application
    1. Introduction
    2. Design Philosophy for Financial Forecasting
    3. The Data
    4. Training Experiments
    5. Conclusion

Appendices

1. Glossary
2. Linear Algebra
   1. Introduction
   2. Vectors
   3. Graphical Representation
   4. Math with Vectors
   5. Multiplying a Vector Times a Scalar
   6. Vector Addition
   7. Vector Multiplication: The Dot Product
   8. Vector Multiplication: The Outer Product
   9. Orthogonality
   10. Matrices
   11. Matrix Addition
   12. Multiplication of a Matrix by a Scalar
   13. Matrix Multiplication
   14. Vector-Matrix Multiplication
3. Back Propagation Mathematics
   1. Introduction
   2. Mean Square Error
   3. Back Propagation Mathematics
   4. Gradient Descent
   5. The Chain Rule
4. Hopfield, CAM, and BAM Mathematics
   1. Hopfield Mathematics
   2. A Content Addressable Memory Application (CAM)
   3. Pattern Association with the Hopfield Model
   4. Optimization Applications of the Hopfield Model
   5. A Bi-directional Associative Memory (BAM)
5. Neuron Transfer Functions
   1. Introduction
   2. Linear Transfer Function
   3. Linear Threshold Function
   4. Step Transfer Function
   5. Sigmoid Transfer Function
   6. Gaussian Transfer Function
6. Bibliography
   1. Artificial and Biological Neural Networks
   2. Expert Systems
   3. Fuzzy Logic
   4. Mathematics and Symbolic Logic

Index