Thursday, May 12, 2016

Lucifer's Technologies: "From the Brow of Zeus"

Google recently paid tribute to a man whose name many people do not know, but whose legacy dominates every aspect of our lives all the same.

The author of the epoch-making paper, "A Mathematical Theory of Communication," Claude Shannon is the father of Information Theory and indeed, the architect of the Digital Age itself.

YouTube, Snapchat, Facebook, Netflix, Instagram, iTunes -- all these services that dominate the flow of words, images, and sounds -- would be nothing but strange and random neologisms were it not for a single paper that laid out the entire science of translating all varieties of information into binary code.
Shannon showed how the once-vague notion of information could be defined and quantified with absolute precision. He demonstrated the essential unity of all information media, pointing out that text, telephone signals, radio waves, pictures, film and every other mode of communication could be encoded in the universal language of binary digits...Shannon laid forth the idea that once information became digital, it could be transmitted without error. 
And where was Shannon employed when he wrote this world-changing paper?

Do you even have to ask? At Bell Laboratories, of course.

You may not know his name but experts in the field of Information Technology do. And they are all awed by him.
Solomon W. Golomb, former president of the IEEE Information Theory Society, thinks Shannon’s influence cannot be overstated. “It’s like saying how much influence the inventor of the alphabet has had on literature,” Golomb remarked
One researcher goes so far as to claim that Shannon is more important even than Albert Einstein:
“Einstein looms large, and rightly so. But we’re not living in the relativity age, we’re living in the information age....It’s Shannon whose fingerprints are on every electronic device we own, every computer screen we gaze into, every means of digital communication.  
"He’s one of these people who so transform the world that, after the transformation, the old world is forgotten.” 

Sci-Fi legend Isaac Asimov paid tribute to Shannon by presenting him like a character from one of his novels:
Claude Shannon: Born on the planet Earth (Sol III) in the year 1916 A.D. Generally regarded as the father of the information age, he formulated the notion of channel capacity in 1948 A.D. Within several decades, mathematicians and engineers had devised practical ways to communicate reliably at data rates within one per cent of the Shannon limit.
Bell Labs' colleague John Pierce, credited with naming the transistor, notes how unique and unprecedented Shannon's paper was:
“It was like a bolt out of the blue ... I don’t know of any other theory that came in a complete form like that, with very few antecedents or history.”
Indeed, Shannon's work was so radical that a professor and electrical engineer at MIT, would have to reach to mythology for an analog: 
“He created a whole field from scratch, from the brow of Zeus.”  
Fascinating choice of words. And yet how many of you have heard of Claude Shannon, or heard of him before the Google tribute?

Information Theory was a revolution, an entirely new vision in computer science. Yet, Shannon's ideas were made for an age beyond that in which they first appeared.

The technology didn't actually exist to turn Shannon's vision into reality when the paper was written. It was only as computer speeds increased-- thanks in large part to the miniaturization that transistors and microchips made possible-- would the full power of Shannon's ideas became apparent. 

Indeed, some said he was too far ahead of the curve in 1948:
 “A lot of practical people around the labs thought it was an interesting theory but not very useful,” said Edgar Gilbert, who went to Bell Labs (to work alongside Shannon).
Not for long.


So how did such a remarkable man come to work for Bell Labs? 

Turns out Shannon met the estimable Vannevar Bush at MIT and became his lab assistant. Shannon was a quick study†:
With the encouragement of (Bush), Shannon decided to follow up his master’s degree with a doctorate in mathematics -- a task that he completed in a mere year and a half. Not long after receiving this degree in the spring of 1940, he joined Bell Labs. 
At Bell, Shannon played the crypto game. Spycraft and codebreaking. Like the group at Shelter Island, here was a man who could be trusted with a secret. 
Shannon's work on cryptography was even more closely related to his later publications on communication theory. At the close of the war, he prepared a classified memorandum for Bell Telephone Labs entitled "A Mathematical Theory of Cryptography..."  

Indeed, digitization and espionage are linked more closely than most would have once expected:
(Shannon) is also credited with the introduction of sampling theory (which) was essential in enabling telecommunications to move from analog to digital transmissions systems in the 1960s and later.
Shannon explained that his codebreaking work helped birth his work on Information Theory:
Shannon said that his wartime insights into communication theory and cryptography developed simultaneously and that "they were so close together you couldn’t separate them."  
Given how apparently disenchanted Shannon would become with the results of his own theory, and how he would eventually walk away from the field he created (to the great worry of his friends and colleagues), one wonders how much can be read into that statement.

But Shannon didn't seem to follow through with Information Theory in quite the way you'd expect from the creator of such a profound new science. 

Instead he seemed to spend his time immersed in increasingly bizarre and obscure pursuits; eccentric experiments in Information Theory, poetry, juggling robots, magnetic mice, gambling techniques. He quit Bell Labs to teach at MIT where he would cowrite several papers, but he largely retreated from publishing by the end of the 50's.

Strangely, Shannon didn't seem to appreciate just how influential his work would be. 
In a 1956 paper entitled “The Bandwagon,” in the journal Transactions on Information Theory, he declared that information theory was being greatly oversold. “It has perhaps ballooned to an importance beyond its actual accomplishments,” he wrote.
He would later claim that he continued to work in Information Theory but didn't find any of his work worth putting out. He stopped attending meetings. He refused invitations to speak out of a terror he could not live up to his reputation. 
A very strange career arc, to be sure.
A flash of inspiration, an earth-shaking foundational theory appearing out of the ether, and then the air slowly going out of the balloon.
 Zeus seemed to have taken back his thunder.


So how often does a theory seem to come out of nowhere, frighteningly complete and ready-to-wear in science or technology? I can't think of another example. It's certainly not the case in any of the technology we've looked at in this series; not even close.  

As always, we see vague references to precedent in Shannon's previous papers or his cryptography work (often by Shannon himself) or to Boolean logic, but the prevailing opinion seems to be that his 1948 paper was in fact novel, unique and unprecedented.

Even the transistor was anticipated, if only in theory (despite the fact that Wolfgang Pauli thought semiconductors were a pipe-dream). There were diodes and rectifiers, even if these antecedents are wildly oversold as true transistor prototypes. But Claude Shannon's theory truly seemed to fall from the sky.

What should be remembered is that codebreaking is every bit a kind of reverse-engineering; it's like standing at a door without a key and learning how to pick the lock.  But first you need to find where the lock is.

To do so you need to learn to think the way your opponent thinks, understand the mysteries that confront you from the inside out.

There were a lot of excellent codebreakers at work during World War II, including the staff at Bletchley Park, who broke the German's Enigma Code*. But somehow they didn't go on to single-handedly create the digital age. Maybe this is another case of "it's not what you know, it's who you know." And what kind of code you're given access to, perhaps.

The timing here is also worth noting: Shannon was publishing his theory at the same time his Bell Labs' colleagues were first showing the transistor to the world.

Very interesting timing indeed.

Shannon's ability to break codes is how he got on Bell's radar in the first place. Given the facts in hand we have today about Shannon, perhaps there was a particular kind of code that Bell needed to...translate
Maybe Shannon's work and Shockley's group's work weren't altogether unrelated.

Because just as Claude Shannon got his position at Bell through his longtime association with Vannevar Bush, William Shockley had worked under Vannevar Bush during WWII on the anti-submarine program. 

Ah. Maybe that explains how Shockley was able to leapfrog over his more qualified partners on the Transistor team and get the lion's share of the glory. Through his association with Vannevar Bush. The same Vannevar Bush who just happened to join AT&T's Board of Directors. In 1947

Timing is everything.

Now it's all starting to come together. One world-changing technology within the space of year from one firm is one thing, but two? And still more to come? Maybe we need to look at those pesky MJ-12 documents again.
NOTE: Read this interview with Shannon- there seems to be a mystery over an important paper which predates his work on Information Theory- it seems it was dated 1940, and Shannon insists it should have been dated 1946. Shannon and the interviewer chalk the discrepancy up to a typo, but that seems a very strange mistake for an important scientific paper.  There was no record of the paper in the IEEE archives either.

An attempt at backdating that was later dropped? Might this "mistake" shed light on the "sketchy" Lilienfeld patents? The interview also goes into greater detail vis a vis Shannon's relationship with Vannevar Bush and his National Security work, as well as his very deep ties to Bell Labs.

† One thing to bear in mind is that the concept of contact technology- ancient or modern- does nothing to diminish the genius of the scientists and engineers involved. Quite the contrary. These are individuals who are seen fit to essentially enter into communion with greater intelligences; gods, daimons, djinn, angels, whatever term you choose.

I can't imagine that a higher intelligence would pick anyone but the absolute best-and-brightest to bestow this information upon, even if those individuals remain in the shadows or if others may not necessarily recognize their abilities.

* Shannon and British computer pioneer Alan Turing were good friends and Turing spent time at Bell Labs during the war and Shannon spent time in England as well. One important detail to remember is that Turing's work on the Enigma code was about foiling the Nazi submarine attacks on British shipping. 
Remember this.