Neural Networks

Neural Networks

n  Advantages

n  prediction accuracy is generally high

n  robust, works when training examples contain errors

n  output may be discrete, real-valued, or a vector of several discrete or real-valued attributes

n  fast evaluation of the learned target function

n  Criticism

n  long training time

n  difficult to understand the learned function (weights)

n  not easy to incorporate domain knowledge

A  Neuron

Network Training

n  The ultimate objective of training

n  obtain a set of weights that makes almost all the tuples in the training data classified correctly

n  Steps

n  Initialize weights with random values

n  Feed the input tuples into the network one by one

n  For each unit

n  Compute the net input to the unit as a linear combination of all the inputs to the unit

n  Compute the output value using the activation function

n  Compute the error

n  Update the weights and the bias

Multi-Layer Perceptron

Network Pruning and Rule Extraction

n  Network pruning

n  Fully connected network will be hard to articulate

n  N input nodes, h hidden nodes and m output nodes lead to h(m+N) weights

n  Pruning: Remove some of the links without affecting classification accuracy of the network

n  Extracting rules from a trained network

n  Discretize activation values; replace individual activation value by the cluster average maintaining the network accuracy

n  Enumerate the output from the discretized activation values to find rules between activation value and output

n  Find the relationship between the input and activation value

n  Combine the above two to have rules relating the output to input

Association-Based Classification

n  Several methods for association-based classification

n  ARCS: Quantitative association mining and clustering of association rules (Lent et al’97)

n  It beats C4.5 in (mainly) scalability and also accuracy

n  Associative classification: (Liu et al’98) 

n  It mines high support and high confidence rules in the form of “cond_set => y”, where y is a class label

n  CAEP (Classification by aggregating emerging patterns) (Dong et al’99)

n  Emerging patterns (EPs): the itemsets whose support increases significantly from one class to another

n  Mine Eps based on minimum support and growth rate

Other Classification Methods

n  k-nearest neighbor classifier

n  case-based reasoning

n  Genetic algorithm

n  Rough set approach

n  Fuzzy set approaches