⏱ 14 min read📝 Assignment insideNo coding required
Welcome to Unit 4. In Units 1–3 you built prompt engineering skills and applied them to real work. Unit 4 goes deeper into how enterprise AI actually works — the architectures and techniques that power the AI systems being built at TCS, Infosys, Wipro, and HCL right now.
This module covers RAG — the most important AI concept in enterprise deployments in 2025–2026. You do not need to build a RAG system to understand this module. You need to understand what it is, why companies use it, and how to talk about it intelligently — which is what every placement interview at an AI-forward Indian IT company will test you on.
Why RAG matters right now
$40B
RAG market projected by 2035, growing from $1.96B in 2025 — Grand View Research
330K+
employees at Infosys, TCS, and Wipro now using Microsoft 365 Copilot — a RAG-powered tool — as of June 2026
85%
of enterprises projected to adopt hybrid RAG systems combining vector and graph databases by 2026
RAG is an AI architecture that combines retrieval-based methods with generative models to produce more relevant and accurate outputs. Instead of relying solely on the pre-trained knowledge stored in a large language model, RAG enables the model to fetch relevant documents or data from an external knowledge base in real-time. This one idea — giving the AI access to your data, not just its training data — is what makes enterprise AI actually useful.
The problem RAG solves
You already know from Module 1.1 that LLMs have a knowledge cutoff — they only know what was in their training data. This creates a fundamental problem for enterprise use:
Without RAG — what goes wrong
Ask about your company's internal HR policy → model guesses or hallucinates
Ask about a client project → model has no knowledge of it
Ask about last week's product release notes → training data is months old
Ask about your bank's specific loan scheme → model describes generic loans
Ask about your codebase's architecture → model cannot access your repository
With RAG — what becomes possible
Ask about your HR policy → model retrieves the actual document and answers from it
Ask about a client project → model retrieves the project wiki and summarises accurately
Ask about release notes → model retrieves the latest file and answers specifically
Ask about your loan scheme → model retrieves your product manual and explains it correctly
Ask about your codebase → model retrieves relevant code files and explains them
The core insight: A standard LLM knows what the world knew up to its training cutoff. A RAG system knows what the world knew up to its training cutoff plus whatever documents you give it access to. That addition — your company's documents, your client's data, your product manuals — is what makes AI genuinely useful for enterprise work.
How RAG works — the five steps
ANALOGY — THE OPEN-BOOK EXAM
Think of a standard LLM as a closed-book exam — the model answers from memory only, and if the answer was not in what it studied, it may guess (hallucinate). RAG is an open-book exam — before answering, the model is allowed to look up the most relevant pages in your textbook. It still needs intelligence to interpret and synthesise what it finds, but it is no longer limited to what it memorised. The textbook is your company's data.
1
User asks a question
A user types a query — this could be through a chatbot interface, a search box, or an internal tool. The query is captured and passed to the RAG system.
User: "What is the leave encashment policy for Infosys employees in Bengaluru?"
2
Query is converted to a vector (embedding)
The query is passed through an embedding model that converts it into a numerical representation — a vector — that captures the semantic meaning of the question. Similar concepts will have similar vectors even if the exact words differ.
"Leave encashment policy Bengaluru" → [0.82, -0.14, 0.67, ...] (a list of numbers representing the meaning)
3
Vector database is searched for similar content
The system searches a vector database — which contains your company's documents, also converted to vectors — to find the most semantically similar chunks of text. This is not a keyword search; it finds conceptually related content even if the words are different.
The retrieved document chunks are added to the LLM's context window alongside the original question. This is the "augmented" part — the prompt now contains both the question and the relevant source material.
Prompt to LLM: "Answer this question using only the provided documents: [Question] [HR Policy Section 7.3] [Circular 2025-HR-42] [FAQ pages 12-14]"
5
LLM generates a grounded answer
The LLM reads both the question and the retrieved documents and generates an answer grounded in the actual source material — not from its training data. Many systems also include citations so the user can verify the source.
Response: "According to HR Policy Section 7.3 and Circular 2025-HR-42, Infosys employees in Bengaluru can encash up to 30 days of earned leave per year at their basic salary rate, subject to a minimum balance of 15 days..."
RAG in action — visual flow
RAG vs fine-tuning — what is the difference?
This is one of the most common interview questions in Indian IT AI roles. Both approaches customise an LLM for specific use, but they work very differently.
Dimension
RAG
Fine-tuning
How it works
Retrieves relevant documents at query time and adds them to the context
Retrains the model weights on your specific data — changes the model permanently
When data updates
Update the document store — no model retraining needed. Changes take effect immediately
Must retrain the model — expensive, slow, and requires ML expertise
Best for
Factual Q&A over documents, internal knowledge bases, current information
Teaching a specific style, format, or task pattern the model must perform consistently
Cost
Lower — vector database + API calls
Higher — GPU compute for training, ongoing maintenance
Hallucination risk
Lower for factual queries — answer grounded in retrieved documents
Similar to base model — model may still confabulate if trained data was incomplete
Indian IT context
Used by Infosys Topaz, TCS AI platforms, internal knowledge tools at most major firms
Used for specialised models: code completion tuned to company style guides, domain-specific classifiers
The one-sentence answer for interviews: RAG gives the model access to external documents at query time — no retraining needed, always up to date. Fine-tuning changes the model's weights to teach it a new behaviour permanently — powerful but expensive. For most enterprise document Q&A use cases, RAG is the right choice.
Where RAG is being used in Indian IT right now
🏢
Internal knowledge bases
Employees ask questions about HR policies, project documentation, SOPs, and compliance rules in natural language. The RAG system retrieves the relevant document and answers accurately.
TCS, Infosys, Wipro — internal HR and policy chatbots. Infosys uses this within its Topaz platform across 200+ client deployments.
🏛
Customer support and banking
Indian banks and fintechs use RAG to power customer service chatbots that answer questions about products, interest rates, and eligibility using the bank's actual product documentation — not generic LLM knowledge.
HDFC, ICICI, SBI digital assistants. The system retrieves from the latest product circular, not training data from months ago.
💼
Legal and compliance
Legal teams query contract repositories, regulatory documents, and compliance frameworks in natural language. RAG retrieves the specific clause or regulation and generates a grounded summary.
Used at Indian law firms, SEBI-regulated entities, and IT companies managing client contract libraries. 92% accuracy in legal document analysis reported using Knowledge Graph RAG.
📈
Code and technical documentation
Developers query internal codebases, API documentation, and architecture wikis. The RAG system retrieves relevant code files or documentation pages before the LLM generates an explanation or recommendation.
GitHub Copilot Enterprise, Cursor IDE, and internal developer tools at Indian IT companies use RAG over private code repositories.
🎓
Education and training
University systems and corporate LMS platforms use RAG to answer student and employee questions from course material, policy documents, and knowledge bases — updated in real time as new material is added.
Indian ed-tech companies (Byju's, Unacademy), corporate training platforms at TCS iON. Course content is added to the document store — no model retraining needed.
⚒
Manufacturing and operations
Maintenance engineers query equipment manuals, failure history, and SOP libraries. The RAG system retrieves the relevant section of a 500-page technical manual and answers a specific question about a machine fault.
Infosys Engineering Services clients in auto and aerospace manufacturing. Replaces manual search through technical documentation that used to take hours.
What RAG cannot do — knowing the limits
RAG is excellent for
Answering factual questions from documents
Keeping answers up to date without retraining
Citing sources so users can verify answers
Internal knowledge Q&A at any scale
Reducing hallucination on domain-specific queries
RAG cannot
Answer questions not covered in the document store
Reason across documents if the connections are implicit
Replace fine-tuning when behavioural change is needed
Work well with very poorly structured or noisy documents
The most important insight from enterprise RAG deployments in 2025 and 2026 is straightforward: RAG is only as good as the quality and reliability of the underlying knowledge source. Garbage documents in, garbage answers out — regardless of how sophisticated the retrieval system is.
RAG in 2026 — what has evolved
By 2026, Agentic RAG is the baseline for any serious AI application — trading a small amount of latency and token cost for a massive increase in reliability. Three developments matter for your career awareness:
Agentic RAG — the system does not just retrieve once; an AI agent decides what to search for, evaluates what it finds, and may search again with a refined query. This produces better answers on complex questions but takes longer.
GraphRAG — instead of retrieving text chunks, the system retrieves a knowledge graph — entities, relationships, and connections. Better for questions that require reasoning across multiple related pieces of information. Microsoft has open-sourced a GraphRAG framework actively used by Indian IT companies.
Hybrid RAG — combines keyword search (traditional) with semantic search (vector). More robust than either alone for enterprise datasets where exact terms (product codes, regulation numbers) matter alongside meaning.
For a fresher or junior professional, you do not need to build any of these. You need to recognise the terms, understand what problem each solves, and be able to discuss them intelligently when they come up in a project context or interview.
The three things to remember
KEY TAKEAWAYS — MODULE 4.1
RAG = LLM + your documents, at query time — the model retrieves relevant content from your knowledge base before answering. This gives it access to current, domain-specific information without retraining. It is why enterprise AI chatbots can answer accurately about internal policies, product details, and recent documents.
RAG vs fine-tuning: retrieval vs retraining — RAG is the right choice when you need accurate, updatable factual Q&A over documents. Fine-tuning is the right choice when you need the model to consistently behave in a specific way. For most Indian IT enterprise use cases in 2026, RAG is deployed far more often than fine-tuning.
Answer quality depends on document quality — a RAG system is only as reliable as the knowledge base it retrieves from. Poorly organised, outdated, or inaccurate source documents produce poor answers even with the best retrieval system. In any RAG project, data quality is the most important non-technical factor.
✎
Assignment 4.1 — Identify a RAG Use Case in Your Domain
This assignment develops your ability to apply RAG thinking to real situations — the skill interviewers test when asking about AI architecture.
Identify one RAG opportunity in a domain you know well — your college, your target industry, or a company you have researched. Ask yourself: "What questions do people repeatedly ask that could be answered from a set of documents?" Good candidates: student handbook Q&A, technical documentation search, policy lookup, product catalogue questions.
Describe the system in five parts:
The document store: What documents would you put in it? (Be specific — not "company documents" but "the 2026 HR policy manual, leave circulars, and the FAQ document")
The user: Who asks questions, and what kind of questions do they ask?
A sample query: Write one realistic question a user would ask
Expected retrieval: Which specific document or section should the system retrieve?
The benefit: What does this replace or improve compared to the current situation?
Add to your AI Learning Journal under "Module 4.1 — RAG Use Case." This becomes part of your Company AI Research document from Assignment 1.3 — concrete evidence that you can think in AI architecture terms, not just tool terms.
The ability to identify a RAG opportunity in a domain, describe the components clearly, and explain the benefit is exactly what TCS, Infosys, and Wipro look for when hiring for AI delivery roles. This assignment is interview preparation, not just coursework.