Adult Verbal Comprehension Performance is Better from Human Speakers than Social Robots, but only for Easy Questions

Abstract

The present study examined whether any differences existed in verbal comprehension performance when information was communicated through either a human or a robot speaker, and whether the nonverbal communication of either a “high social” robot (i.e., frequent gestures, direct head gaze) or a “low social” robot (i.e., no gestures, indirect head gaze) influenced comprehension. This study also assessed whether question difficulty moderated comprehension performance. A sample of 46 adult participants (23 human + high social robot; 23 human + low social robot) were given verbal comprehension questions from both a human and a robot. The results showed that, as question difficulty increased, performance elicited by the robot speaker reached parity with that of the human speaker. Conversely, for easy questions, performance generated from the robot speaker was significantly inferior to that from the human speaker. The robot social level of behaviour had no measurable impact on the results. These results suggest that the traditional use of social robots for simple communications could be extended to more complex domains. Practical implications for education, healthcare, and marketing are further discussed.

Appendix A: Verbal Comprehension Questions (from AssessmentDay Ltd.)

[Correct answers are bolded. The full set of questions and rationale for solutions can be found at AssessmentDay Ltd.]

• 1. Organisational development is defined as a “deliberately planned effort to increase an organisation’s relevance and viability”. The concept of organisational development was conceived by German-American psychologist Kurt Lewin, who was among the first to apply social psychology, group dynamics and social research to the world of work. Since its beginnings, organisational development has become a popular and widespread method for implementing organisational change. The key principles underlying organisational development philosophy are primarily humanistic, focused around improving organisational performance by empowering its employees.

  Where do the key principles of organisational development come from?

  • a. Pragmatism

  • b. Egalitarianism

  • c. Humanism

  • d. Objectivism

• 2. Researchers from the international agricultural research partnership have suggested that bananas could soon become a staple food source for millions of people in developing countries due to climate change. Rising temperatures and volatile weather makes growing existing critical foods, such as potatoes, maize, and wheat, difficult, leading to lower crop yields of these foods. However, bananas are much more adept at surviving in tropical climates frequently racked by high temperatures and weather volatility. Although, it may not be a viable food staple for all developing countries, it may become more critical for the economies of many developing nations in future years.

  Which crop is not stated to have difficulty growing in increasing temperatures?

  • a. Potato

  • b. Barley

  • c. Maize

  • d. Wheat

  • 3. The most expensive brand of coffee today on the open market is known as Kopi Luwak, or civet coffee, from Indonesia. What makes this brand of coffee unique is that it is made from the coffee beans eaten by the Asian palm civet. The civet eats the coffee berry for its fleshy pulp, along with the coffee bean inside, which is indigestible. However, while passing through the digestive system of the civet, proteolytic enzymes seep into the bean, making peptides shorter and increasing the number of free amino acids, improving the taste of the coffee bean. Similarly, due to the expensive and time-consuming process of searching for civet droppings, Kopi Luwak can be sold for up to $3000 per kilogram.

    Why does civet digestion supposedly improve the coffee beans taste?

    • a. Lengthens coffee bean peptides

    • b. Shortens coffee bean peptides

    • c. Removes coffee bean peptides

    • d. Increases number of coffee bean peptides.

  • 4. Rare earth elements, also known as rare earth metals, are a group of seventeen chemical elements in the periodic table. Contrary to their name, rare earth elements are relatively plentiful in the Earth’s crust, with cerium as abundant as copper. However, these elements are rarely found in large enough deposits to make mining financially viable, hence the name rare earth. Currently, over 90% of global rare earth element supply is met by the people’s republic of China, despite only holding 23% of the world’s proven reserves of these metals. China has been steadily decreasing its quotas of rare earth element exports due to environmental concerns, depleted reserves and difficulties with industry regulation, which may lead to shortages of rare earth elements in future years.

    What is not a reason for lower rare earth element exports in China?

    • a. Difficulties with regulation

    • b. Environmental issues

    • c. Reserve depletion

    • d. International competition

  • 5. Europa is the sixth closest moon of the planet Jupiter, and one of the largest moons in the solar system. Europa is slightly smaller than the Earth’s moon and has a tenuous atmosphere composed primarily of oxygen. The surface of Europa is composed of water and is one of the smoothest in the solar system. Scientists suggest that it is possible Europa has an ocean of liquid water beneath its surface, making it a strong candidate for extra-terrestrial life. It is also predicted that heat energy from tidal flexing may cause oceans to remain liquid and drives geological activity similar to plate tectonics on Earth.

    What makes Europa a potential home to extra-terrestrial life?

    • a. Ocean below its surface

    • b. Smooth ice water surface

    • c. Atmosphere comprised of oxygen

    • d. Geological activity

  • 6. The Mojave Desert occupies portions of California, Nevada, Utah and Arizona in the United States. The desert is named after the Mohave tribe of Native Americans, and it displays typical basin and range topography. The Mojave Desert receives less than three hundred millimetres of rain a year and is generally between 2000 and 5000 feet in elevation. The desert experiences temperature extremes at both ends of the spectrum depending on the season. During winter, temperatures may drop to around −18 °C at higher elevations. During summer, the temperature may rise to over 49 °C, making it the hottest place in the United States.

    How much rain falls in the Mojave Desert per year?

    • a. 300 mm

    • b. 30 mm

    • c. 330 mm

    • d. Cannot say

  • 7. The Japanese asset price bubble collapse lead to the economic period known as “the lost decade” in japan from 1991 to 2000. During the 1970s and 1980s Japan’s economic growth was particularly strong; fuelling speculation that Japan would rise, effectively, to superpower status. However, abnormalities within the Japanese economic system fuelled massive amounts of speculation from Japanese firms. High property prices combined with low interest rates lead to large amounts of borrowing and heavy investment in domestic and foreign stock. However, the finance ministry realised this was an asset bubble, and subsequently raised interest rates, which in turn caused a crash in the stock market, leading to the bursting of the Japanese asset price bubble.

    What economic conditions caused the large amounts of borrowing in Japan?

    • a. High interest rates, low property prices

    • b. High interest rates, high property prices

    • c. Low Interest rates, low property prices.

    • d. Low interest rates, high property prices

  • 8. Carbon emissions’ trading is a form of emissions trading that targets the gas, carbon dioxide. Emissions trading works by setting a limit on the emissions produced by emitters, such as organisations, whom, therefore, have to buy carbon credits to permit them to emit particular volumes of carbon dioxide, motivating them to decrease their carbon footprint. Carbon emissions trading was the first gas involved in emissions trading and it makes up the bulk of emissions trading, with sulphur dioxide and nitrogen oxides also traded. This incentive to reduce carbon emissions may aid environmental efforts to meet obligations specified by the Kyoto Protocol: the agreement to “stabilise greenhouse gas concentrations at a level that would prevent dangerous anthropogenic interference with the climate system.”

    What were the first gas emission permits to be traded?

    • a. Carbon dioxide

    • b. Sulphur dioxide

    • c. Nitrogen oxides

    • d. Chlorofluorocarbons

  • 9. A random number generator is a computational or physical device designed to generate a sequence of numbers that lacks a pattern. Several methods for random number generation exist, but many fall short of achieving true randomness based on the use of statistical tests for randomness. Random number generators have been used for hundreds of years through devices such as dice, shuffling playing cards and roulette wheels, and have therefore been applied to games of chance. Similarly, random number generators have been used in situations in which fairness is approximated by randomization, such as selecting jurors and military draft lotteries.

    What does the passage state is an application of a random number generator?

    • a. Games of chance

    • b. Statistical sampling

    • c. Survey sampling

    • d. Divination

  • 10. The Seawise Giant was an ultra-large crude oil carrier super tanker, and was subsequently the longest ship ever built. She possessed the greatest deadweight tonnage ever recorded and when fully laden, her weight was 657,019 tonnes, making her the heaviest ship of any kind, and the largest self-propelled man-made object ever built. She was last used as a floating storage and offloading unit off the coast of Qatar. The vessel was sold to an Indian ship breaker and renamed Mont for her final journey in December 2009. The ship was then sailed to, and intentionally beached at, Gujarat India for demolition.

    What was not a record held by the Seawise Giant?

    • a. Heaviest ever ship

    • b. Longest ever ship

    • c. Tallest ever ship

    • d. Largest self-propelled man-made object