LLMs Miscount Letters and Numbers

The Problem with LLMs

ChatGPT and other large language models miscount letters and numbers. For example, ChatGPT couldn’t accurately count the number of letters in the word ‘strawberry’, miscounting the number of ‘R’s. This issue is not isolated, as seen in the case of carb counting, where an individual asked AI to count carbs 27,000 times and received different answers each time.

The Technical Limitations

These models are trained on vast amounts of text data, which can include errors and inconsistencies. Confident mistakes are a common problem of large language models used in AI chatbots. The training data can be flawed, leading to the models learning and repeating incorrect information.

The Broader Industry Context

The limitations of LLMs are not unique to OpenAI’s ChatGPT. Other companies, such as Google, are also working on improving their language models. Google Translate, for example, has introduced a feature to practice pronunciation, demonstrating the ongoing efforts to enhance AI capabilities. However, the issue of confident mistakes remains a challenge across the industry. Google’s efforts to improve its language models are part of a larger trend, as companies like Meta and Microsoft are also investing in AI research and development. The market for AI-powered language models is growing rapidly, with applications in areas such as customer service, language translation, and content generation.

The History of LLMs

Large language models have been developed over the years, with significant advancements in recent times. Despite these advancements, the problem of miscounting letters and numbers persists. Prior launches of LLMs have also been met with similar challenges, highlighting the need for continued improvement. The first LLMs were developed in the early 2010s, and since then, there have been numerous updates and improvements. However, the issue of confident mistakes has remained a persistent problem, with each new generation of LLMs introducing new challenges and limitations.

The Technical Mechanics

The technical mechanics behind LLMs involve complex algorithms and neural networks. The models are trained on vast amounts of data, which enables them to generate human-like text. However, this training data can be flawed, leading to the models learning and repeating incorrect information. The design choice of using vast amounts of text data is intended to improve the models’ language understanding capabilities, but it also introduces the risk of confident mistakes. The neural networks used in LLMs are typically trained using a technique called deep learning, which involves multiple layers of interconnected nodes. This approach allows the models to learn complex patterns and relationships in the data, but it also makes them more prone to overfitting and confident mistakes.

The Downstream Implications

The struggles of LLMs with basic facts have significant implications for their use in everyday applications. The development of more accurate and reliable AI models is crucial for realizing the full potential of these technologies. As AI becomes increasingly integrated into various aspects of life, the need for accurate and consistent models becomes more pressing. For example, in healthcare, AI models are being used to analyze medical images and diagnose diseases. However, if these models are prone to confident mistakes, it could lead to incorrect diagnoses and potentially harm patients. Similarly, in finance, AI models are being used to analyze market trends and make investment decisions. If these models are flawed, it could lead to significant financial losses.

What’s Next

Developers must prioritize accuracy and consistency in their models. The community’s reaction to the carb-counting article and the strawberry R-count issue indicates a growing awareness of the limitations of LLMs. As the industry continues to evolve, it is essential to address these challenges and develop more reliable AI models. One potential solution is to use more diverse and high-quality training data, which could help to reduce the risk of confident mistakes. Additionally, developers could use techniques such as data augmentation and transfer learning to improve the performance of their models.

Updates

• 2026-05-13 — OnlyFans’ First-Gen Creators Are Retiring—and Some Are Begging You to Forget They Exist (source)