One of the world's best artificial intelligence computers is only as smart as a four year old human child, a new study suggests.
Researchers at the University of Illinois at Chicago IQ-tested one of the best available artificial intelligence systems to see how intelligent it really is.
It turned out that the computer is about as smart as the average 4-year-old, researchers said.
The UIC team put ConceptNet 4, an artificial intelligence system developed at Massachusetts Institute of Technology (MIT), through the verbal portions of the Weschsler Preschool and Primary Scale of Intelligence Test, a standard IQ assessment for young children.
They found ConceptNet 4 has the average IQ of a young child. But unlike most children, the machine's scores were very uneven across different portions of the test.
"If a child had scores that varied this much, it might be a symptom that something was wrong," said Robert Sloan, professor and head of computer science at UIC, and lead author on the study.
Sloan said ConceptNet 4 did very well on a test of vocabulary and on a test of its ability to recognise similarities.
"But ConceptNet 4 did dramatically worse than average on comprehension - the 'why' questions," he said.
One of the hardest problems in building an artificial intelligence, Sloan said, is devising a computer programme that can make sound and prudent judgement based on a simple perception of the situation or facts–the dictionary definition of commonsense.
Commonsense has eluded artificial intelligence engineers because it requires both a very large collection of facts and what Sloan calls implicit facts–things so obvious that we don't know we know them. A computer may know the temperature at which water freezes, but we know that ice is cold.
"All of us know a huge number of things. As babies, we crawled around and yanked on things and learned that things fall. We yanked on other things and learned that dogs and cats don't appreciate having their tails pulled," said Sloan.
"We are still very far from programmes with commonsense - artificial intelligence that can answer comprehension questions with the skill of a child of eight," said Sloan.
Researchers at the University of Illinois at Chicago IQ-tested one of the best available artificial intelligence systems to see how intelligent it really is.
It turned out that the computer is about as smart as the average 4-year-old, researchers said.
The UIC team put ConceptNet 4, an artificial intelligence system developed at Massachusetts Institute of Technology (MIT), through the verbal portions of the Weschsler Preschool and Primary Scale of Intelligence Test, a standard IQ assessment for young children.
They found ConceptNet 4 has the average IQ of a young child. But unlike most children, the machine's scores were very uneven across different portions of the test.
"If a child had scores that varied this much, it might be a symptom that something was wrong," said Robert Sloan, professor and head of computer science at UIC, and lead author on the study.
Sloan said ConceptNet 4 did very well on a test of vocabulary and on a test of its ability to recognise similarities.
"But ConceptNet 4 did dramatically worse than average on comprehension - the 'why' questions," he said.
One of the hardest problems in building an artificial intelligence, Sloan said, is devising a computer programme that can make sound and prudent judgement based on a simple perception of the situation or facts–the dictionary definition of commonsense.
Commonsense has eluded artificial intelligence engineers because it requires both a very large collection of facts and what Sloan calls implicit facts–things so obvious that we don't know we know them. A computer may know the temperature at which water freezes, but we know that ice is cold.
"All of us know a huge number of things. As babies, we crawled around and yanked on things and learned that things fall. We yanked on other things and learned that dogs and cats don't appreciate having their tails pulled," said Sloan.
"We are still very far from programmes with commonsense - artificial intelligence that can answer comprehension questions with the skill of a child of eight," said Sloan.
No comments:
Post a Comment