In recent years, artificial intelligence (AI) has been widely used in various fields, solving many problems for humans. However, this hot technology is not omnipotent. A recent study found that AI has not mastered some skills that people can easily accomplish. For example, most AI models cannot understand familiar clocks and calendars.
The University of Edinburgh in the United Kingdom stated in a press release that their research team found that even the most advanced AI models cannot reliably explain the position of clock hands or correctly answer questions about dates on calendars.
The team mentioned that unlike simply identifying shapes, understanding clocks and calendars requires a combination of spatial awareness, background, and basic mathematics – which remains challenging for AI.
The team tested an AI system that processes text and images – called the Multimodal Large Language Model – to see if it could answer time-related questions by looking at images of clocks or calendars.
They tested various clock designs, including clocks with Roman numerals, clocks with and without second hands, and clocks with dials of different colors.
The results showed that the AI system accurately judged the position of the clock hands less than a quarter of the time. Errors were more likely to occur when the clock had Roman numerals or formatted hands.
The team indicated that even when the second hand of the clock was removed, there was no improvement in the performance of the AI system, suggesting that these systems have deep-rooted problems in detecting the second hand and interpreting angles.
Regarding calendars, the researchers asked the AI model to answer a series of calendar-related questions, such as identifying holidays and calculating past and future dates.
They found that even the best-performing AI model had a error rate of one-fifth of the time in date calculations. For example, when asked, “Which day is the 153rd day of this year?” the AI model had a high error rate.
The lead author of the study, researcher Rohit Saxena, said, “Most people can distinguish time and use calendars from a young age. Our research highlights the significant gap in AI’s performance on basic human skills.”
Another researcher at the university, Aryo Gema, mentioned, “Today’s AI research often emphasizes complex reasoning tasks, but ironically, many systems still struggle with relatively simple everyday tasks.”
Saxena also stated that if people want to successfully integrate AI into time-sensitive applications in the real world, such as scheduling and automation, these challenges must be addressed.
The research findings were published on the online preprint database arXiv. The researchers also presented the paper at the International Conference on Learning Representations held on April 28th.