As organizations increasingly explore the capabilities of large language models (LLMs) and Retrieval-Augmented Generation (RAG), the line between applied AI and true innovation becomes more critical—especially when considering eligibility for Scientific Research and Experimental Development (SR&ED) incentives.
This post explores the underlying technologies of LLMs and RAG, their practical applications, and how to distinguish between routine development and SR&ED-eligible innovation.
Understanding the Technology: LLMs and RAG
Large Language Models (LLMs) like OpenAI’s GPT, Meta’s LLaMA, and DeepSeek’s R1 represent a leap in natural language processing. Trained on massive datasets, these models can perform a wide range of tasks, from summarization and translation to code generation and semantic search. Their architecture typically involves:
Tokenization: Splitting text into smaller pieces (tokens).
Embeddings: Turning tokens into numeric representations.
Attention: Deciding which tokens matter most in context.
Generation: Predicting the next best token to produce coherent responses.
So how do LLMs actually work?
They start by breaking your input into small pieces called tokens, then convert each one into a unique numerical fingerprint. These fingerprints live in a mathematical space where similar meanings cluster together.
The model then uses attention to weigh which tokens matter most in context, and finally generates a response based on those relationships.
What’s remarkable is that the model isn’t “understanding” language, it’s predicting what comes next. But with enough data and structure, prediction starts to look a lot like reasoning.
However, LLMs have limitations: they hallucinate, lack real-time knowledge, and struggle with domain-specific tasks.
Retrieval-Augmented Generation (RAG) addresses these gaps by combining LLMs with external data sources. RAG systems retrieve relevant documents in real time and feed them into the LLM to generate more accurate, context-aware responses. This architecture typically includes:
- Indexing: Splitting and storing documents in a vector database.
- Retrieval: Fetching relevant chunks based on a query.
- Generation: Using the LLM to produce a response grounded in retrieved data.
Where SR&ED Can Be Found in RAG and LLM Work
Not all work with LLMs and RAG qualify for SR&ED. The key lies in whether the work involves overcoming technological uncertainties and advancing the state of the art.
Examples of Non-SR&ED Work:
- Using off-the-shelf tools like FAISS or Pinecone without modifying their core mechanisms.
- Tuning parameters (e.g., chunk size or overlap) through trial and error.
- Integrating third-party services without addressing underlying technical limitations.
Examples of potentially eligible SR&ED work:
- Fine-tuning an LLM on your own company data when standard methods don’t give good results: Quick-win or did it require problem solving to make it work?
- Integrating more sources (emails, PDFs, chat logs) to your RAG system: Did you have to design a new approach to keep results accurate and relevant?
- Fine-tuning which documents show up first in retrieval: Was it just a change to search ranking, or did you have to design/test new scoring methods to improve relevance?
Examples of SR&ED-Eligible Work:
- Novel Vector Databases: Developing new storage architectures or embedding systems that outperform existing solutions.
- Advanced Retrieval Algorithms: Creating new methods for semantic search, ranking, or clustering that go beyond current capabilities.
- Domain-Specific Innovation: Building custom models or pipelines for high-accuracy domains like healthcare, where standard tools fall short.
Key Questions to Identify SR&ED Potential
To assess whether your RAG or LLM project may qualify for SR&ED, ask:
- Did we face a clear technological limitation?
- Was the limitation at the level of retrieval, storage, or generation?
- Did we develop new methods or significantly improve existing ones?
- Were the outcomes uncertain at the outset?
Conclusion
LLMs and RAG are powerful tools, but true innovation lies in how we adapt, extend, and challenge their capabilities. For organizations investing in this space, understanding where SR&ED applies can unlock not just technical breakthroughs—but also valuable funding opportunities.
Perhaps building your own RAG system is an option, or it is already in the works. Either way, we can support with the guidance and advice to maximize any potential funding outcomes through SR&ED.
