Neural Networks (ANN) and brainjs

Neural network abstract 3d rendering
Standard

The post is for fundamental understanding on how to get started with Neural networks and build applications using NodeJS. I have not discussed any self developed example for now. I would share the same very soon.

Briefly I would introduce ANN i.e Artificial Neural Networks : The introduction of the concepts talk fundamental based using logic gates in design but nowadays we just simulate the logic in our web applications using libraries to get the same results.

What is ANN ?

An Artificial Neural Network (ANN) is an information processing paradigm that is inspired by the way biological nervous systems, such as the brain, process information. The key element of this paradigm is the novel structure of the information processing system. It is composed of a large number of highly interconnected processing elements (neurones) working in unison to solve specific problems. ANNs, like people, learn by example. An ANN is configured for a specific application, such as pattern recognition or data classification, through a learning process. Learning in biological systems involves adjustments to the synaptic connections that exist between the neurones. This is true of ANNs as well.

Why use ANN ?

Neural networks, with their remarkable ability to derive meaning from complicated or imprecise data, can be used to extract patterns and detect trends that are too complex to be noticed by either humans or other computer techniques. A trained neural network can be thought of as an “expert” in the category of information it has been given to analyse. This expert can then be used to provide projections given new situations of interest and answer “what if” questions.
Other advantages include:

  1. Adaptive learning: An ability to learn how to do tasks based on the data given for training or initial experience.
  2. Self-Organisation: An ANN can create its own organisation or representation of the information it receives during learning time.
  3. Real Time Operation: ANN computations may be carried out in parallel, and special hardware devices are being designed and manufactured which take advantage of this capability.
  4. Fault Tolerance via Redundant Information Coding: Partial destruction of a network leads to the corresponding degradation of performance. However, some network capabilities may be retained even with major network damage.

Difference between Algo based computing systems and ANN :

Neural networks take a different approach to problem solving than that of conventional computers. Conventional computers use an algorithmic approach i.e. the computer follows a set of instructions in order to solve a problem. Unless the specific steps that the computer needs to follow are known the computer cannot solve the problem. That restricts the problem solving capability of conventional computers to problems that we already understand and know how to solve. But computers would be so much more useful if they could do things that we don’t exactly know how to do.

Neural networks process information in a similar way the human brain does. The network is composed of a large number of highly interconnected processing elements(neurones) working in parallel to solve a specific problem. Neural networks learn by example. They cannot be programmed to perform a specific task. The examples must be selected carefully otherwise useful time is wasted or even worse the network might be functioning incorrectly. The disadvantage is that because the network finds out how to solve the problem by itself, its operation can be unpredictable.

On the other hand, conventional computers use a cognitive approach to problem solving; the way the problem is to solved must be known and stated in small unambiguous instructions. These instructions are then converted to a high level language program and then into machine code that the computer can understand. These machines are totally predictable; if anything goes wrong is due to a software or hardware fault.

Neural networks and conventional algorithmic computers are not in competition but complement each other. There are tasks are more suited to an algorithmic approach like arithmetic operations and tasks that are more suited to neural networks. Even more, a large number of tasks, require systems that use a combination of the two approaches (normally a conventional computer is used to supervise the neural network) in order to perform at maximum efficiency.

Architecture of ANN :

They are divided into :

(1) Feed Forward Networks OR Bottoms up OR Top down Networks :

  • The signal travels from Input to the output.
  • There is no feedback loops i.e O/P doesn’t affect the same layer.
  • Use extensively in pattern recognition.

(2) Feedback Networks :

  • The signal can flow from O/P to I/P as well.
  • They are powerful and can be extremely complicated.
  • There state is constantly changing. The state is dynamic.
  • Point of Equilibrium is only when the change is detected in input.

(3) Network Layers

The commonest type of artificial neural network consists of three groups, or layers, of units: a layer of “input” units is connected to a layer of “hidden” units, which is connected to a layer of “output” units.

  • The activity of the input units represents the raw information that is fed into the network.
  • The activity of each hidden unit is determined by the activities of the input units and the weights on the connections between the input and the hidden units.
  • The behaviour of the output units depends on the activity of the hidden units and the weights between the hidden and output units.

This simple type of network is interesting because the hidden units are free to construct their own representations of the input. The weights between the input and hidden units determine when each hidden unit is active, and so by modifying these weights, a hidden unit can choose what it represents.

(4) Perceptrons

  • They follow the MCP model i.e Neurons with Weighted Objects.
  • Every Input has specific tasks that it has to perform following a model weight based approach.

How to Train ANN ? 

This is the most important portion that one need to think before applying ANN to real world web applications. The bigger view can be divided into :

(1) Supervised Learning 

  • It incorporates an external teacher, so that each output unit is told what its desired response to input signals ought to be.
  • During the learning process global information may be required.
  • Paradigms of supervised learning include error-correction learning, reinforcement learning and stochastic learning.

Constraints :

  • Problem of error convergence, i.e the minimisation of error between the desired and computed unit values. The aim is to determine a set of weights which minimises the error.
  • One well-known method, which is common to many learning paradigms is the least mean square (LMS) convergence.

(2) Unsupervised Learning 

  • It uses no external teacher and is based upon only local information.
  • It is also referred to as self-organisation, in the sense that it self-organises data presented to the network and detects their emergent collective properties.
  • Paradigms of unsupervised learning are Hebbian lerning and competitive learning.
  • We say that a neural network learns off-line if the learning phase and the operation phase are distinct. A neural network learns on-line if it learns and operates at the same time. Usually, supervised learning is performed off-line, whereas usupervised learning is performed on-line.

Other level of learning methods specific to ANN can be  :

(1) Associative Mapping 

  • The network learns to produce a particular pattern on the set of input units whenever another particular pattern is applied on the set of input units.

Divided further into :

  • Auto-association: an input pattern is associated with itself and the states of input and output units coincide. This is used to provide pattern completition, ie to produce a pattern whenever a portion of it or a distorted pattern is presented. In the second case, the network actually stores pairs of patterns building an association between two sets of patterns.

  • Hetero-association: is related to two recall mechanisms:

    Nearest-neighbour recall, where the output pattern produced corresponds to the input pattern stored, which is closest to the pattern presented, and

    Interpolative recall, where the output pattern is a similarity dependent interpolation of the patterns stored corresponding to the pattern presented. Yet another paradigm, which is a variant associative mapping is classification, ie when there is a fixed set of categories into which the input patterns are to be classified.

(2) Regularity Detection 

  • The units learn to respond to particular properties of the input patterns. Whereas in asssociative mapping the network stores the relationships among patterns, in regularity detection the response of each unit has a particular ‘meaning’.
  • This type of learning mechanism is essential for feature discovery and knowledge representation.
  • Every neural network posseses knowledge which is contained in the values of the connections weights. Modifying the knowledge stored in the network as a function of experience implies a learning rule for changing the values of the weights.

(3) Fixed Networks Weights are able to change.

  • Weights are not able to change.
  • The values are set prior to problem to solve.

(4) Adaptive Networks 

  • Weights are able to change.

The other important concepts related to learning are :

  • Transfer Function

ANN depends upon weights, Input and Output that are defined in units. So the units are defined as  :

  1. Linear Units : O/P activity is proportional to the total weighted output.
  2. Threshold Units : the output is set at one of two levels, depending on whether the total input is greater than or less than some threshold value.
  3. Sigmoid Units : the output varies continuously but not linearly as the input changes. Sigmoid units bear a greater resemblance to real neurones than do linear or threshold units, but all three must be considered rough approximations.

Note : We must choose these units correctly, Also check if the unit of one system can influence another system.

  • Back Propagation Algorithm

In order to train a neural network to perform some task :

  • We must adjust the weights of each unit in such a way that the error between the desired output and the actual output is reduced.
  • This process requires that the neural network compute the error derivative of the weights (EW).
  • In other words, it must calculate how the error changes as each weight is increased or decreased slightly.
  • The back propagation algorithm is the most widely used method for determining the EW.

The back-propagation algorithm is easiest to understand if all the units in the network are linear. The algorithm computes :

  • Each EW by first computing the EA, the rate at which the error changes as the activity level of a unit is changed.
  • For output units, the EA is simply the difference between the actual and the desired output.
  • To compute the EA for a hidden unit in the layer just before the output layer, we first identify all the weights between that hidden unit and the output units to which it is connected.
  • We then multiply those weights by the EAs of those output units and add the products. This sum equals the EA for the chosen hidden unit.
  • After calculating all the EAs in the hidden layer just before the output layer, we can compute in like fashion the EAs for other layers, moving from layer to layer in a direction opposite to the way activities propagate through the network. This is what gives back propagation its name.
  • Once the EA has been computed for a unit, it is straight forward to compute the EW for each incoming connection of the unit. The EW is the product of the EA and the activity through the incoming connection.

ANN and NodeJS

We use a specific library – Brain.js that can be found here.

Installation 


//If you have express-generator installed

$express --css stylus -ejs Appname

$cd Appname

$npm install

$npm install brain

Configuring and Defining the functions

Note : I have tried to explain the code in comments.

//You can define these in the app.js or use export() to create a separate code.


var brain = require(./lib/brain');
// Defining a variable to call the library.
var net = new brain.NeuralNetwork();
// Call the Input for the Library
//This can be dynamic from the web application
// Call the train() to train your ANN
net.train([{input: [0, 0], output: [0]},
{input: [0, 1], output: [1]},
{input: [1, 0], output: [1]},
{input: [1, 1], output: [0]}]);
// Defining the Output for the ANN.
var output = net.run([1, 0]); // [0.987]

How to use the same using in a web application :

INPUT : You can define the Input from any source. I would recommend :

  1. Take the input from the input as JSON from a process. This will make it dynamic.
  2. Take the input from user on the web GUI.

OUTPUT : You can define the output here as :

  1. Set of values fetched from JSON from a process. This will make it dynamic.
  2. Define them manually as set of values.

TRAIN :  Each training pattern should have an input and an output, both of which can be either an array of numbers from 0 to 1 or a hash of numbers from 0 to 1.


.train([

{input: {}, output: {}}

],
{
errorThresh: 0.005,// error threshold to reach
iterations: 20000, // maximum training iterations
log: true, // console.log() progress periodically
logPeriod: 10, // number of iterations between logging
learningRate: 0.3// learning rate}
);

The Output of Train()


{
error: error_value_in_numerics;  //0.0004 i.e less then or equal to one
iterations: number_of_iterations_in_value_numerics; //206
}

OFFLINE TRAIN 

It is very painful to compute online for learning and training process. One alternative can be using JSON as the offline data training solution.

var json = net.toJSON();
net.fromJSON(json);
//

DEFINE LEARNING RATE & HIDDEN LAYERS

var net = new NeuralNetwork({
hiddenLayers: [4],
learningRate: 0.6 // global learning rate, useful when training using streams
});
//Hidden Layers can be defined as hiddenLayers[X,Y]. The X and Y are values of hidden layers in two layers.

All good to go! Thanks for reading.

Share Button