Who controls the past now controls the future
Who controls the present now controls the past.
Rage Against the Machine, “Testify”
There is irony in starting this post with a Rage Against the Machine (RATM) quote. They’re not just a band known for relentless protest against oppression, corporate greed, police brutality, war, propaganda, and systemic injustice; they’re a call to action, urging listeners to question authority, challenge the status quo, and fight for change.
Quoting RATM in a post aimed at helping corporations use AI more effectively would draw the ire of Zack de la Rocha. However, my motivation is to question the blind adoption of a technology without doing the hard work that would improve outcomes for the people, in this case, the users and customers.
The Status Quo
In my career, I’ve spent a lot of time focused on improving the customer experience, whether working on self-service tools like chatbots or finding ways to help internal employees better serve customers. In both cases, the goal is to provide the right answer at the right time to help customers get the most out of our products and services.
A common challenge in customer service, especially in call centers, is finding the answer to satisfy a customer’s need. Often, companies have large collections of documents managed by a content management system (CMS). Different yet overlapping documents may exist across products or business lines or intended for different audiences, such as internal or customer-facing documentation. Errors and updates may inconsistently cascade throughout the documentation, resulting in missing, incorrect, or conflicting information. As a result, the CMS may return the wrong results, causing slower response times, conflicting answers, higher call volumes, and frustration by both employees and customers.
The RAG
With the availability of large language models (LLMs) and the introduction of retrieval-augmented generation (RAG), organizations saw a possible solution to improve the search process. RAG offered a way to retrieve a specific answer to a question, rather than returning a document in a search result that would need to be read to find the answer. It was also seen as a way to overcome the challenges of poor documentation.
Companies quickly discovered, however, that this wasn’t the case. Whether it was due to missing, incorrect, or conflicting information1,2, the adage of “garbage in, garbage out” applied and resulted in poor performance.
Additionally, the documentation format made it difficult to ingest and maintain the complete context, even if the documentation itself was correct. Key information in images, tables, and complex formatting is often lost or misinterpreted in a RAG system.3
Ultimately, these challenges caused these systems to be delayed or prevented them from being deployed at scale.
The Machine
The “machine” represents the continuation of the status quo. In this case, it symbolizes the tendency to think every problem can be solved with the latest technology. It involves our inclination to blindly trust often-overhyped technology to solve problems without fixing any underlying issues. It also includes grinding through poor implementations and subpar results rather than dedicating resources to evaluating other less-hyped or less-technical solutions.
I’ve worked on AI products where the underlying data was so bad that it was impossible to make reliable, actionable predictions. But rather than direct teams to fix the underlying data, the machine allows for only one direction–forward. The only acceptable actions were to continue to endlessly iterate and hope for a different result, to lower the bar for success, or look for a problem that we could solve with the bad data, even if it doesn’t align with the original or important business goals.
It’s happening again with RAG and generative AI. The machine is seeking technical tweaks to improve the results rather than fixing the underlying documentation or converting it to a more RAG-friendly format. Change the chunking strategy. Fix the embeddings. Adjust the prompt. Select a different algorithm. Use reinforcement learning with human feedback. 4,5,6
Of course, these are great recommendations when the underlying problem isn’t the data or the documentation. But they’re ineffective when the problem is the source. No amount of tweaking will fix fundamentally bad documentation.
The Resistance
If the machine perpetuates the existing pattern, the resistance challenges that pattern, takes a different approach, and focuses on the business goal, not the technology.
At its core, RAG is designed to make searching better. It includes additional benefits such as synthesizing coherent, context-aware responses, but in service of returning the most relevant, useable answers to a user.
To take advantage of RAG’s secondary benefits, companies should not rush into the technical solution pushed by the machine and instead spend time fixing the documentation. As controversial as that recommendation seems, it’s the most fundamental task likely to improve search results.
In a world where generative AI initiatives benefit from seemingly unlimited funding, requesting resources to update documentation will likely draw questions.
Why can’t we solve this with technology? Have you heard about this new feature that might solve the issue? Can we bring in a new vendor?
It is up to leaders to take a stand and convince the business to take a different path. It requires focusing on the business problem, identifying the root cause, and recommending the best way to address it.
If the root cause of poor search results is the documentation, addressing issues in the documentation is the best course of action and may show immediate improvements with the existing search solution. It’s possible the improvements will be dramatic enough to not require RAG at all, at least right away. But it leaves the door open to implement a RAG-based solution on top of better documentation to take advantage of RAG’s additional benefits.
The machine will persist.
But leaders must resist.
Conclusion
The allure of new technology is powerful. The promise of AI and RAG as instant solutions to deeply rooted problems is tempting, especially in corporate environments eager for innovation. But when the fundamental issue is bad documentation, no AI model—no matter how sophisticated—can fully compensate for it.
The real battle isn’t against the technology itself but against the mindset that technology alone is always the answer. True leadership means recognizing when the most effective solution isn’t the most exciting or trendy one, but the one that addresses the root cause.
Fixing documentation isn’t glamorous. It won’t make headlines like an AI-powered chatbot or a multimillion-dollar LLM investment. But it’s the foundation of good search, good RAG, and ultimately, good customer experience. Without it, RAG becomes just another iteration of the machine—grinding forward without progress.
So before blindly deploying the latest AI tool, ask this question: Are we solving the problem, or are we just feeding the machine?
- https://www.valprovia.com/en/blog/top-7-challenges-with-retrieval-augmented-generation
↩︎ - https://www.aimon.ai/posts/top_problems_with_rag_systems_and_ways_to_mitigate_them
↩︎ - https://arxiv.org/abs/2401.05856
↩︎ - https://snorkel.ai/blog/retrieval-augmented-generation-rag-failure-modes-and-how-to-fix-them/
↩︎ - https://medium.com/towards-data-science/12-rag-pain-points-and-proposed-solutions-43709939a28c
↩︎ - https://doi.org/10.48550/arXiv.2501.13264
↩︎
