"The steps between my original suggestion of the chess playing machine, Mr. Shannon's move to realize it in the metal, the use of computing machines to plan the necessities of war, and the colossal state machine of Pere Dubarle, are in short clear and terrifying... The mechanical control of man cannot succeed unless we know man's built-in purposes, and why we want to control him." Norbert Wiener, The Human Use of Human Beings, 1950.
I THINK THEREFORE I AM
Norbert Wiener's popular legacy is the word "cybernetics", which he crafted in his 1948 book "Cybernetics," subtitled "Control and Communication in the Animal and Machine." Cybernetics, according to Wiener, described a new way of looking at life. Where once scientists imagined the universe as a giant clockwork where everything was set according to a pattern, Wiener postulated that the universe was a massively disorganized unpatterned place. Whatever order existed, Wiener thought, came from the exchange of information -- messages, coding, decoding -- between everything from the smallest atomic particle to galactic clusters. Information created order in a disorderly universe. The human brain was a message processor at its core.
Wiener, by analogy, postulated that the human brain and the mechanical brain of the newly invented digital computer were similar. For Wiener, the modern computer was the closest thing to a mechanical brain ever invented. When Wiener invoked this theory, in 1947, the Cold War had not yet fully unveiled itself. The Berlin Blockade of 1948, the Soviet detonation of a nuclear weapon in 1949 -- these had yet to become history -- and little mention in "Cybernetics" appears concerning the potential dangers and temptations Wiener's cybernetics introduced.
By 1950, when Wiener published a second book called "The Human Use of Human Beings", his work took an explicitly political and social tone. He wrote the "Human of Use Human Beings" for non-mathematicians; unlike "Cybernetics" there were no mathematical equations covering the pages. Instead, Wiener emphasized a fear. If information is the currency of life, controlling the shape of things, then wasn't it possible, in theory, to send out messages which would effectively control the way people perceive the world? Critics of Wiener's "Cybernetics" raised this issue, reading his work as the means to the creation of a theoretical machine, a "machine to govern." Wiener felt such an idea wasn't ludicrous, writing in the first edition of "The Human Use of Human Beings" that such a machine "is quite possibly being planned by a secret military project for the purposes of combat and domination." Then, mysteriously, that edition of "The Human Use of Human Beings" disappeared.
All later editions of the book, after the first edition from which these quotes come, use a vastly different text, so different that no two pages are alike. The later editions are much less concerned with secret projects to control human behavior. Instead, Wiener focuses on the theory of cybernetics in relation to Newtonian physics and game theory. The passage concerning the "machine to govern" loses its tone of imminent doom. "The [machine to govern] is not frightening because of any danger it may achieve of autonomous control over humanity.