Synaptic weight

In a computational neural network, a vector or set of inputs ${\displaystyle {\textbf {x}}}$ and outputs ${\displaystyle {\textbf {y}}}$, or pre- and post-synaptic neurons respectively, are interconnected with synaptic weights represented by the matrix ${\displaystyle w}$, where for a linear neuron

${\displaystyle y_{j}=\sum _{i}w_{ij}x_{i}~~{\textrm {or}}~~{\textbf {y}}=w{\textbf {x}}}$.

where the rows of the synaptic matrix represent the vector of synaptic weights for the output indexed by ${\displaystyle j}$.

The synaptic weight is changed by using a learning rule, the most basic of which is Hebb's rule, which is usually stated in biological terms as

Neurons that fire together, wire together.

Computationally, this means that if a large signal from one of the input neurons results in a large signal from one of the output neurons, then the synaptic weight between those two neurons will increase. The rule is unstable, however, and is typically modified using such variations as Oja's rule, radial basis functions or the backpropagation algorithm.

For biological networks, the effect of synaptic weights is not as simple as for linear neurons or Hebbian learning. However, biophysical models such as BCM theory have seen some success in mathematically describing these networks.

In the mammalian central nervous system, signal transmission is carried out by interconnected networks of nerve cells, or neurons. For the basic pyramidal neuron, the input signal is carried by the axon, which releases neurotransmitter chemicals into the synapse which is picked up by the dendrites of the next neuron, which can then generate an action potential which is analogous to the output signal in the computational case.

The synaptic weight in this process is determined by several variable factors:

• How well the input signal propagates through the axon (see myelination),
• The amount of neurotransmitter released into the synapse and the amount that can be absorbed in the following cell (determined by the number of AMPA and NMDA receptors on the cell membrane and the amount of intracellular calcium and other ions),
• The number of such connections made by the axon to the dendrites,
• How well the signal propagates and integrates in the postsynaptic cell.

The changes in synaptic weight that occur is known as synaptic plasticity, and the process behind long-term changes (long-term potentiation and depression) is still poorly understood. Hebb's original learning rule was originally applied to biological systems, but has had to undergo many modifications as a number of theoretical and experimental problems came to light.

• Neural network
• Synaptic plasticity
• Hebbian theory