In their book “Perceptrons” (1969), Minsky and Papert demonstrate that a simplified version of Rosenblatt’s perceptron can not perform certain natural binary classification tasks, unless it uses an unmanageably large number of input predicates. It is easy to show that with sufficiently many input predicates, a perceptron (even on this type) can perform any classification with perfect accuracy (see page 3 of the notes below). The contribution of Minsky and Papert is to show that meaningful restrictions on the type of input predicates hamper the expressive ability of the perceptron to such a degree that it is unable to e.g. distinguish connected from disconnected figures, or classify according to whether the number of active pixels is odd or even. The former has a simple picture proof, whereas the crucial ingredient for the latter is the action of a permutation group on the retina (i.e. the input array) of the perceptron.
Talk
Here are my notes from a recent talk I gave on the book at the Berlin machine learning seminar:
Style
The book is an engaging and instructive read – not only is it peppered with the author’s opinions and ideas, but it includes also enlightening comments on how the presented ideas originated, and why other ideas that occurred to the authors didn’t work out. The book still bears the marks of it’s making, so to speak.
Controversy
The publication of the first edition in 1969 is popularly credited with bringing research on perceptrons and connectionism in general to a grinding halt. The book is held to be unjust, moreover. The “perceptrons”, which Minsky and Papert prove to be so limited in expressive power, were in fact only a very simplified version of what practitioners then regarded as a perceptron. A typical perceptron (unlike those of Minsky and Papert) might include more layers, feedback loops, or even be coupled with another perceptron. All these variations are described in Rosenblatt’s book “Principles of Neurodynamics” (1962). This is put very well (and colourfully!) by Block in his review of the book (1970):
Thus, although the authors state (p. 4, lines 12-14) “we have agreed to use the name ‘perceptron’ in recognition of the pioneer work of Frank Rosenblatt.”, they study a severely limited class of machines from a viewpoint quite alien to Rosenblatt’s. As a result, the title of the book, although generous in intent, is seriously misleading to the naive reader who wants to find out something about the general class of Perceptrons.
In summary then, Minsky and Papert use the word perceptron to denote a restricted subset of the general class of Perceptrons. They show that these simple machines are limited in their capabilities. This approach is reminiscent of the möhel who throws the baby into the furnace, hands the father the foreskin and says, “Here it is; but it will never amount to much.”
Despite these serious criticisms, it should be noted that Block (himself a trained mathematician) was full of praise for the “mathematical virtuosity” exhibited by Minsky and Papert in their book.
As to whether the book alone stopped research into perceptrons is hard to judge, particularly given it’s impact is confounded by the tragic death of Rosenblatt (at age 41) only two years later.
As a former member of the Cornell Cognitive Systems Research Program crew, I see the tumult as a mash-up between an entirely deterministic philosophy (Minsky) and a probabilistic one (Rosenblatt). To me, Minsky’s approach smacks of arrogance: “never mind the brain research, I know a better way to do it”; Rosenblatt’s was, to the contrary, more of a scientific-method one. He started with the body of knowledge, facts, and assumptions made about brain structure and function by investigators, philosophers, and psychologists through all history; reduced these to manageable size (considering the capabilities of computing machinery in that day) by simplifying them, and then experimented to ascertain whether the models thereby constructed exhibited brain-like behavior. That the Mark I Perceptron (which we attempted to name Brunnhilde, vetoed by Frank) could offer any behavior even approaching the consciousness of a planarian (flatworm) was, in light of these simplifications, truly remarkable.
Minsky teamed up with Harry Harrison to write a science-fiction novel about a brain-injured human who was fitted with a prosthetic brain. It came across as propaganda, rather than readable literature. What I have read of Minsky and Papert’s book also seems this way to me — a manifesto, rather than a scientific investigation. Their thesis therein seems to me to exhibit a childish glee, a snideness and smugness that gives the lie to their assumption of the mantle of impartiality and scientific detachment.