Context is Key
While it may seem that providing more context would lead to better results, the opposite can often be the case. A large and unfocused context can be detrimental to the outcome. Providing models with large context might not work as expected:
| Context Poisoning When LLM hallucinates & generates factually incorrect information that gets incorporated into the context leading to a cascade of errors. |
Context Confusion This is where unnecessary and irrelevant information in the context affects the response. |
| Context Distraction Here the large volume of information overwhelms the models training causing it to lose focus on the users primary query. |
Context Clash When different parts of the context disagree with each other, leading to an incoherent response. |
How can we provide LLMs with relevant context without falling into these traps? "Retrieval-Augmented Generation" (RAG) is a powerful method that involves selectively adding relevant information to the context at the time of the query.
| Context Sandbox Isolate context in dedicated threads preventing it from interfering with other parts of the models processing. |
Context Summary Provide a condensed version of the context to the model for processing. |
| Context Priming Removing unneeded or redundant information from the context before it is fed to the LLM. |
Context Offloading Store information outside the immediate context of the LLM and using tools to access it only when needed. |
| Tool Loadout Select only the most relevant tools for the task at hand. This concept is similar to how players in games like Counter-Strike select their "loadout" before a match begins. Just as a player chooses the most relevant weapons, grenades, and armor for the upcoming round based on their strategy, map and the opposing team, we should select only the most relevant tools for the task at hand. |
|