“What is so impressive is that PaLM has never seen parallel sentences between Bengali and English,” Pichai said at that event. “It was never explicitly taught to answer questions or translate. The model brought together all its capabilities to answer questions correctly in Bengali, and we can extend the technique to more languages and other complex tasks.”
Jason Post, a Google spokesperson, told BuzzFeed News that the company had never claimed that it did not train PaLM in Bengali. “While the PaLM model was trained to complete basic sentences in a wide variety of languages (including English and Bengali), it was not trained to know how to 1) translate between languages, 2) answer questions in a question-and-answer format, or 3) translate information between languages while answering questions,” Post said in a statement. “He learned these emerging capabilities on his own, and that’s an impressive achievement.”
Emily M. Bender, a professor and researcher at the University of Washington who wrote a Twitter thread about him 60 minutes segment, took issue with Manyika’s comments. The program’s ability to translate “all Bengali,” Bender told BuzzFeed News, is a “claim without scope or substance.”
“What does ‘all Bengali’ really mean?” Bender tweeted. “How was this tested?” He also wrote that Manyika’s statement ignored or obscured the fact that there are Bangla texts in the training data.
Bender tweeted that the term “’emergent properties’ seems to be the respectable way of saying AGI,” which stands for artificial general intelligence, a hypothetical technology that can learn on its own and perform tasks better than humans. “It’s still nonsense,” he said.
Mitchell was equally blunt on Twitter. “Maintaining the belief in ‘magical’ properties and expanding it to millions (thanks for nothing @60Minutes!) serves Google’s public relations goals,” Mitchell tweeted. “Unfortunately, it is misinformation.”
Several other people in the tech space also publicly criticized CBS and Google:
