Warren Sturgis McCulloch

Neuroscience

He studied the excitation of the brain by strychnine neuronography, which was a method to map brain connections. Applying strychnine in one point of the brain causes excitations in different points of the brain.^[11] Bailey, Bonin, and McCulloch conducted a series of studies in the 1940s that identified connections in the brains of macaque and chimpanzee that are consistent with modern understanding of VOF.^[12][13]

Mathematical logic

In 1919 he began to work mainly on mathematical logic, and by 1923 he attempted to make a logic of transitive verbs. His goal in psychology is to invent a "psychon" or "least psychic event" that are binary atomic events with necessary causes, such that they can be combined to create complex logical propositions concerning their antecedents. He noticed in 1929 that these may correspond to the all-or-nothing firings of neurons in the brain.^[3]

In the 1943 paper, they described how memories can be formed by a neural network with loops in it, or alterable synapses. These then encodes for sentences like "There was some x such that x was a ψ" or ${\displaystyle (\exists x)(\psi x)}$, and showed that looped neural networks can encode all first-order logic with equality and conversely, any looped neural networks is equivalent to a sentence in first-order logic with equality, thus showing that they are equivalent in logical expressiveness.^[3]

The 1943 paper describes neural networks operating over time, and logical universals -- "there exists" and "for all"—for spatial objects, such as geometric figures, was further developed in their 1947 paper.^[14]

He worked with Manuel Blum in studying how a neural network can be "logically stable", that is, can implement a boolean function even if the activation thresholds of individual neurons are varied.^[15]: 64 They were inspired by the problem of how the brain can perform the same functions, such as breathing, under influence of caffeine or alcohol, which shifts the activation threshold over the entire brain.^[3]

He worked on triadic relations, an extension of the calculus of relations to handle relations that relates 3 objects, such as "A gives B to C" or "A perceives B to be C". He was convinced that such a logic is necessary for understanding brain activity.^[10][16]

How we know universals

In the 1947 paper How we know universals, they studied the problem of recognizing objects despite changes in representation. For example, recognizing a square under different viewing angles and lighting conditions, or recognizing a phoneme under different loudness and tones. That is, recognizing objects invariant under the action of some symmetry group. This problem was partly inspired by a practical problem in designing a machine for the blind to read (recounted in Wiener's Cybernetics, see before).^[17]

The paper proposed two solutions. The first is in computing an invariant by averaging over the symmetry group. Let the symmetry group be ${\displaystyle G}$ and the object to be recognized be ${\displaystyle x}$. Let a neural network implement a function ${\displaystyle T}$. Then, the group-invariant representation would be ${\displaystyle {\frac {1}{|G|}}\sum _{g\in G}T(gx)}$, the group-action average. The second solution is in a negative feedback circuit that drives a canonical representation. Consider the problem of recognizing whether an object is a square. The circuit moves the eye so that the "center of gravity of brightness" of the object is moved to the middle of the visual field. This then effectively converts each object into a canonical representation, which can then be compared with a representation in the brain.^[18][19]

Neural network modelling

In the 1943 paper McCulloch and Pitts attempted to demonstrate that a Turing machine program could be implemented in a finite network of formal neurons (in the event, the Turing Machine contains their model of the brain, but the converse is not true^[20]), that the neuron was the base logic unit of the brain. In the 1947 paper they offered approaches to designing "nervous nets" to recognize visual inputs despite changes in orientation or size.

From 1952 McCulloch worked at the Research Laboratory of Electronics at MIT, working primarily on neural network modelling. His team examined the visual system of the frog in consideration of McCulloch's 1947 paper, discovering that the eye provides the brain with information that is already, to a degree, organized and interpreted, instead of simply transmitting an image.

With Roberto Moreno-Díaz, he studied a formalized problem of memory. Given that neural networks can story memory by a pattern of oscillations in a circle, they studied the number of possible oscillation patterns that can be sustained by some neural network with ${\displaystyle N}$ neurons. This came out to be ${\displaystyle K(N)={\binom {2^{N}}{k}}\sum _{k=1}^{2^{N}-1}k!}$ (Schnabel, 1966).^[21] Also, they proved a universality theorem, in that for each ${\displaystyle N}$, there exists a neural network (possibly with more than ${\displaystyle N}$ neurons) with ${\displaystyle \log _{2}K(N)}$ binary inputs, such that, for any oscillation pattern realizable by some neural network with ${\displaystyle N}$ neurons, there exists a binary input for this universal network such that it exhibits the same pattern.^[22][23][24]

Control

McCulloch considered the problem of contradictory information and motives, which he called a "heterarchy" of motives, meaning that the motives are not linearly ordered, but can be ordered like ${\displaystyle A>B>C>A}$B>C>A}">.^[25] He posited the concept of "poker chip" reticular formations as to how the brain deals with contradictory information in a democratic, somatotopical neural network. Specifically, how the brain can commit the animal to a single course of action when the situation is ambiguous.

They designed a prototypic example neural network "RETIC", with "12 anastomatically coupled modules stacked in columnar array", which can switch between unambiguous stable modes based on ambiguous inputs.^[26][16]

His principle of "Redundancy of Potential Command"^[26] was developed by von Foerster and Pask in their study of self-organization^[27] and by Pask in his Conversation Theory and Interactions of Actors Theory.^[28]