Research shows machine reading minds in Mandarin

A team of researchers in China was able to decipher what five patients were thinking in Mandarin, a breakthrough for tonal languages.

Chinese researchers say they have created a "mind-reading" machine that can turn thoughts into spoken Mandarin, according to the South China Morning Post. The process of generating speech from brain signals is also known as speech synthesis.

Earlier studies have achieved a level of speech synthesis for non-tonal languages like English and Japanese, but tonal languages like Mandarin pose unique challenges. Two billion people speak tonal languages, which represent more than 60% of languages spoken around the world.

Scientists say their research could help Chinese-speaking patients with communication disorders, the South China Morning Post reported. It could also be used to improve the efficiency of brain-computer interface technology, which allows people to control computers with their brains​.

Mandarin, like other tonal languages, use pitch changes in syllables to convey different meanings. For example, the Mandarin syllable "ma" can mean "mother," "hemp," "horse," or "scold" – all depending on the tone used​.

The team, led by Fudan University's Huashan Hospital in Shanghai in collaboration with Tianjin University and ShanghaiTech University, used electrocorticography, a type of electrophysiological monitoring that records brain signals directly from its surface.

Researchers implanted electrodes in the brains of five patients undergoing brain tumor surgery and asked them to produce one of eight syllables with different tones during the medical operation.

Electrode implants recorded the activity and the model used in research then decoded it thanks to its ability to simultaneously pick on both the base syllable and the tone from the brain signals.

The study was published in Science Advances journal.