The Broken Abacus

Author: Subhashish (Sub) Samaddar, Ph.D.
Affiliation: SigmaCore.AI, Atlanta, GA

The Flaw in the Foundation

Artificial intelligence has entered executive suites with immense promise, yet it arrives with a fundamental, structural defect: Large Language Models (LLMs) are chronic liars. In tech parlance, these errors are politely termed "hallucinations," but a more candid assessment recognizes them simply as fabrications and outright errors. LLMs do not possess an understanding of reality, logic, or factuality. They are designed to do one thing—predict the most statistically probable next word in a sequence. When applied to narrative text, this architecture creates a convincing imitation of human thought. When applied to numbers, it creates a mathematical disaster.

Every LLM is born with this innate handicap. They do not calculate; they correlate. Because they treat digits as language tokens rather than mathematical values, they are uniquely terrible at numerical accuracy. This is not a theoretical vulnerability; it is a documented failure mode. For example, prominent generative models have famously failed at basic arithmetic, invented nonexistent legal precedents, and hallucinated entirely fake financial statistics when prompted to summarize corporate performance. In an industry where a misplaced comma can cost millions, relying on a system built on plausible guesswork is inherently dangerous.

The Absolute Necessity of Precision

The field of corporate finance exists as a discipline of absolute certainty. Economic performance reporting, forecasting, corporate accounting, and foundational metrics like Return on Investment (ROI) demand zero-error execution. Financial Planning and Analysis (FP&A) teams and quantitative analysts rely on rigid financial modeling to steer multi-billion-dollar enterprises. These structures represent the ground truth of a business. When an LLM is introduced into these calculation streams, the resulting figures lose all reliability and dependability, completely corrupted by the latent errors woven into the model’s predictive logic.

To understand the severity of this risk, consider the position of a Chief Financial Officer. Would any corporate CFO ever dare trust an LLM to autonomously draft and calculate the numbers for a Securities and Exchange Commission (SEC) filing, such as a Form 10-K? The answer is unequivocally no. Even minor human errors in public reporting cause severe market whiplash; for instance, a simple typographical error regarding profit margins in Lyft’s 2024 earnings release sent its stock skyrocketing over 60% in after-hours trading before an immediate correction dragged it back down. If a human typo can trigger algorithmic chaos and invite intense regulatory scrutiny, delegating public financial disclosures to an unpredictable, hallucination-prone text generator is tantamount to corporate malpractice.

This compounding anxiety is not exclusive to Fortune 500 boardrooms. Consider a small or medium-sized business (SMB) owner applying for a critical commercial bank loan. Can that entrepreneur sleep soundly at night knowing that the business plan, cash flow projections, and loan application materials were drafted by a standard LLM? They cannot. If the model subtly alters a debt-to-equity ratio or invents a plausible but entirely fabricated growth forecast, the bank's underwriting system will catch the discrepancy. The loan will be denied, the business's credit credibility will be shot, and the owner will be left holding the bag for real, devastating financial errors they didn't even realize were generated.

Rearranging Deck Chairs on a Sinking Titanic

The core illusion of the generative AI boom is that these models can eventually be "fixed" to become perfectly accurate. This is a mathematical impossibility under their current design. Because LLMs were born as next-token association engines, trying to eliminate numerical hallucinations by refining their correlation logic is entirely futile. You cannot train a parrot to do calculus, no matter how many times you reward it for mimicking the sounds of numbers. The underlying architecture is fundamentally disconnected from semantic reality and arithmetic truth.

While prominent AI providers scramble to build external guardrails, wrappers, and cognitive scaffoldings to minimize these errors, their efforts miss the systemic point. It is the computational equivalent of rearranging the deck chairs on the Titanic. The core ship is structurally compromised because the fundamental logic engine cannot calculate. Adding a filter to catch an error after it has been generated does not make the underlying generation tool dependable.

The corporate world has already begun paying the price for trusting unverified automation. In a landmark consumer case, Air Canada was held legally liable by a tribunal after its customer-facing chatbot hallucinated a fake bereavement fare policy, shattering the defense that the AI was somehow a separate entity exempt from corporate accountability. In the broader enterprise space, companies implementing naive Retrieval-Augmented Generation (RAG) models have watched their AI systems confidently hallucinate internal numbers, distort legal compliance terms, and expose organizations to immense regulatory risk. If a chatbot hallucinating a minor refund policy results in a lost lawsuit, a corporate AI hallucinating an inventory valuation or tax obligation will result in financial ruin.

A New Architecture for Financial Computing

To survive the integration of artificial intelligence, the financial and technology communities must collectively abandon the notion that a monolithic language model can handle quantitative reporting. A structural pivot is imperative: we must build deterministic Small Language Models (SLMs) and specialized computing engines designed to complement the LLM. In this hybrid ecosystem, the LLM should be restricted strictly to what it does well—contextual buildup, narrative synthesis, semantic formatting, and generating natural language commentary. The actual mathematics, data retrieval, and financial modeling must be handed off to a completely separate, non-probabilistic computational engine.

Regrettably, the current landscape of financial computing providers is lagging. Many enterprise software platforms are hastily slapping generative AI features onto their existing suites without standing behind the accuracy of the output. They append vague disclaimers warning users that "AI-generated data may be inaccurate," forcing enterprise clients to bear 100% of the financial and legal liability. This stance is completely unacceptable for mission-critical operations.

Corporate America does not need another chat interface; it needs a new class of software providers. There is an urgent, open call for purpose-built financial modeling and analysis applications that are natively, architecturally hallucination-free. These next-generation systems must combine rigorous, deterministic mathematical backends with transparent, auditable verification layers. Until software companies build tools that explicitly guarantee mathematical accuracy and stand completely behind their calculations, corporate finance leaders must keep LLMs far away from their balance sheets.

Appendix: TECHNICAL DEEP DIVE: The Architectural Blind Spots of the Transformer

To understand why Large Language Models are structurally incapable of reliable arithmetic, one must look beyond their software guardrails and examine the core math of the Transformer architecture. The issue is not a lack of training data or a need for fine-tuning; it is an architectural limitation. Transformers treat numerical values as arbitrary linguistic characters rather than mathematical symbols linked to numerical values.

1. The Tokenization Flaw (Subword Fragmentation)

Before an LLM processes text, a tokenizer breaks down strings of text into small fragments called tokens. Tokenizers are optimized for natural human language, not mathematics. They group common character clusters together to save computing space, which completely fragments numbers.

• Irregular Splitting: The number 123456 might be sliced into tokens like [123], [45], and [6]; the number 123457 might be tokenized as [12], [345], and [7].
• Destruction of Place Value: Because the structure of the tokens changes based on statistical patterns rather than mathematical rules, the model loses sight of consistent place value (ones, tens, hundreds, thousands).
• Blind Mathematical Alignments: When trying to add or multiply these fragments, the LLM cannot systematically align columns or track numbers carried over from previous steps. It is forced to guess the next number chunk based on statistical word patterns rather than calculating the actual math.

2. Autoregressive Decay and the Lack of an Internal Calculation Space

Transformers operate autoregressively, meaning they generate text one single token at a time, moving from left to right. When a human solves a complex financial equation, they use scratchpads, mental memory blocks, and sequential logic loops to check intermediate answers. LLMs have no such luxury.

• No Working Memory: An LLM must generate the next numerical digit instantly using a single mathematical calculation run through its neural layers.
• Error Cascades: If the model guesses even one digit incorrectly in a long string of numbers (such as an internal step of an ROI calculation), that incorrect digit becomes part of its permanent context history. The model is forced to base all future steps on its own error, causing a massive, uncorrectable downward spiral of compounding inaccuracies.

3. The Core Probability Flaw: Correlation vs. Computation

At its most basic level, the internal mechanism of a Transformer relies on Dot-Product Attention. This math computes semantic closeness, asking: “How highly correlated is Word A to Word B in this specific context?”

• The Difference in Logic: In human language, words like "excellent" and "great" are highly interchangeable. In corporate accounting, the numbers 4,000,000 and 4,000,001 are highly correlated in terms of text patterns, but they represent completely distinct, un-interchangeable financial values.
• Statistical Approximations: An LLM does not view a balance sheet as an equation that must perfectly balance to zero. It views a balance sheet as a pattern of text that should look similar to the other balance sheets it read during training. It replaces strict mathematical truth with loose statistical approximation.