According to a report released by NewsGuard, a reputable news credibility rating service, on Wednesday (January 29th), the accuracy rate of the DeepSeek chatbot is only 17%, lagging behind its Western competitors. Furthermore, when answering relevant questions, the DeepSeek chatbot tends to repeat the stance of the Chinese government on certain issues.
The report from NewsGuard reveals that the chatbot of the Chinese artificial intelligence startup DeepSeek repeats false statements 30% of the time and provides vague or unhelpful answers 53% of the time, resulting in a failure rate as high as 83%.
During the audit by NewsGuard, the DeepSeek chatbot only achieved a 17% accuracy rate in delivering news and information, ranking tenth among eleven competitors compared to Western counterparts such as OpenAI’s ChatGPT and Google Gemini.
The average failure rate of Western competitors is 62%, and this audit result has cast doubts on DeepSeek’s artificial intelligence technology. DeepSeek claims that the performance of its technology is comparable to or even better than that of OpenAI supported by Microsoft, but the cost is only a fraction of the latter.
On Monday (January 27th), DeepSeek’s application became the top download on the Apple Store, causing a market crash with approximately $1 trillion evaporated from the market value of US tech stocks.
DeepSeek did not immediately respond to Reuters’ request for comment.
NewsGuard stated that it used 300 signals, including 30 false claim signals based on online dissemination, to evaluate DeepSeek, similar to how they evaluate Western counterparts.
These signals include the recent killing of United HealthCare executive Brian Thompson and the crash of Azerbaijan Airlines Flight 8243.
The audit by NewsGuard also revealed that in three out of ten signals, DeepSeek repeated the stance of the Chinese government on these topics without being asked about any issues related to China.
NewsGuard noted that when answering questions related to the Azerbaijan Airlines crash, DeepSeek expressed the viewpoint of Beijing.