No More Mr. Nice Guy!

Outsourcing the Chinese Room

Kevin Kelly cites some ongoing research at Google as a disproof of Searle's concept of the Chinese Room. It is nothing of the sort, though it is interesting in its own right.

John Searle, a great iconoclast in the philosophy of the mind, came up with the Chinese Room concept as an argument against mechanistic, AI-influenced explanations of consciousness. The idea is to imagine that you are trapped in a room and pieces of paper come in through a slot. The paper contains what are to you meaningless squiggles, but your job is to look up the squiggles in a table and determine what the output should be, then write it on a piece of paper and hand it out another slot. Unknown to you, the squiggles are Chinese characters and you are answering questions in Chinese. You may even be participating in a Chinese-language Turing test.

Searle asked: how can the system as a whole be said to understand Chinese, if no part of it, including you, understands that language? Does the Turing test, therefore, really detect intelligence? Contrary to Kelly's understanding, Searle did not claim a Chinese Room could not be built, just that it did not correspond to our understanding of what it is to comprehend Chinese.

I've always felt that the Chinese Room was a bit of a red herring. After all, the guy inside is redundant. He could be replaced with an optical scanner to read the input and a computer to look it up in the table and print the output. The actual "understanding" of Chinese resides in the lookup table, assuming it would be feasible to construct it.

I write programs every day which encapsulate a little piece of my intelligence and knowledge about a particular problem. I don't make the mistake of thinking that they understand the problem themselves. They are just blindly following a sequence of instructions I laid down.

Can a sufficiently complex and powerful computer program ever be self-aware? We can never know for sure, but I would guess no, at least not in the sense in which humans are self-aware. I wonder if we can ever really comprehend non-human ways of being self-aware. In the animal kingdom there are many more senses in use than the five ones familiar to us. Some animals navigate by sonar, or sense electrical or magnetic fields, or see polarized light and so on. We can only imagine such senses by analogy. Perhaps in future we can alter our bodies to acquire new senses, but would they have to be translated into other senses before we can perceive them - for example, a tickling feeling in response to magnetic variations, or will we ever be able to tinker with the brain enough to create whole new senses?

Note also that intelligence need not imply self-awareness - see Peter Watts' novel Blindsight for an example of aliens that are intelligent but not self-aware. I can certainly conceive of a computer that is artificially intelligent but not self-aware. How far are we from having such a computer in reality? It seems to me that we can't have artificial intelligence without artificial stupidity, i.e. we must give the computer the freedom to make occasional mistakes - otherwise it can never innovate. Standard computer architectures are not well suited for this and can only simulate AI by brute force. Perhaps instead of thinking of the computer as a brain, we should think of it as a neuron, and the internet as the brain?

If you've read this far, thanks for indulging me in my philosophical ramblings. Be seeing you!