GENERATIVE AI: ZERO TO JOB-READY › UNIT 1 › MODULE 4 OF 4
Risks, Hallucinations, and Responsible Use of GenAI
⏱ 13 min read📝 Assignment insideUnit 1 Finale
This is the final module of Unit 1 — and arguably the most important one for your professional reputation.
Every module so far has focused on what GenAI can do. This one focuses on what it gets wrong, and why that matters more than people realise. The engineers and analysts who get ahead in Indian IT are not the ones who use AI most — they are the ones who use it most responsibly. Knowing the failure modes makes you the person who catches the mistakes before they reach a client.
How big is the problem?
51%
of organisations using AI have experienced at least one negative consequence (McKinsey 2025)
1,450+
legal cases worldwide involving AI hallucinations identified in court records as of 2026
0.7%–25%
hallucination rate range across current models — best is Gemini Flash at 0.7%, worst exceed 25%
These are not edge cases. They are the normal operational reality of AI tools in 2026. Knowing this is not a reason to avoid AI — it is a reason to use it with a clear understanding of where it fails.
Risk 1: Hallucination
Hallucination is the most misunderstood AI failure mode. It is not a bug in the traditional sense — it is a structural consequence of how LLMs work.
Recall from Module 1.1: LLMs predict the next word based on patterns in training data. When asked about something outside their training data, or something they were not trained on reliably, they do not say “I don’t know.” They predict the most statistically plausible next words — which can result in a fluent, confident, completely false response.
Critical to understand: An AI hallucination does not look like a mistake. It looks like a correct answer. The model uses perfect grammar, cites plausible-sounding sources, and states false information with full confidence. This is what makes it dangerous.
Real-world hallucination cases
CASE 1 — The Lawyer Who Cited Fake Cases (USA, 2023–ongoing)
A New York attorney used ChatGPT to research legal precedents for a court filing. The AI generated citations to six legal cases — complete with realistic case names, court names, dates, and legal reasoning. None of the cases existed. When the opposing side challenged the citations, the lawyer claimed he did not know ChatGPT was a generative tool rather than a legal database.
A federal judge issued a standing order requiring all attorneys appearing before the court to certify that AI was not used to draft filings, or to flag any AI-drafted language for accuracy review. As of 2026, over 1,450 similar cases have been identified in court records worldwide.
Lesson for you: Never cite a source you found via AI without independently verifying that source exists and says what the AI claims it says.
CASE 2 — The Fake Reading List (Chicago Sun-Times, 2025)
A publisher used AI to generate a "Summer Reading List for 2025" that was syndicated to the Chicago Sun-Times. Readers discovered that 10 out of 15 recommended books were entirely fabricated — non-existent titles attributed to real, well-known authors. The titles sounded plausible, the author attributions were correct, but the books did not exist.
The newspaper removed the list from its online edition, but readers who had already received the printed version had paid for content that was largely fiction presented as fact.
Lesson for you: When AI recommends specific books, papers, tools, or resources by name — verify they exist before sharing or citing them.
CASE 3 — Stanford Legal AI Study (2025)
Stanford researchers asked multiple LLMs about legal precedents in a controlled study. The models collectively invented over 120 non-existent court cases — complete with convincingly realistic names, dates, courts, and detailed (but entirely fabricated) legal reasoning. Notably, even purpose-built legal AI tools designed specifically for legal research still hallucinated on 17% to 34% of challenging queries.
Lesson for you: Domain-specific AI tools are better but not reliable enough to trust without verification. This applies equally to code, medical information, financial data, and technical specifications.
Why hallucination rates vary
Even the best models hallucinate. Gemini Flash’s 0.7% rate means roughly 1 in 143 responses contains a fabricated element. In a 50-response work session, you can statistically expect at least one hallucination — and you will not know which one unless you verify.
Risk 2: AI Bias
AI bias occurs when a model produces outputs that systematically favour or disadvantage certain groups — based on imbalances, stereotypes, or gaps in its training data.
What causes bias in AI?
Training data reflects the real world — including its inequalities. If training data contains more examples of men as engineers or certain groups in negative contexts, the model learns those patterns. It is not intentional — it is statistical inheritance from historical data.
Where does it show up in Indian IT work?
CV screening tools that favour certain name patterns or universities. Code generation that defaults to Western libraries or approaches. Customer service AI that performs differently across languages. Resume feedback that differs based on implicit demographic signals in writing style.
Why it matters in India specifically
India's AI Governance Guidelines 2025 explicitly address algorithmic bias. The Constitution's equality provisions apply — biased AI outputs affecting Indian citizens can trigger regulatory scrutiny under the Consumer Protection Act, 2019 and the DPDP Act, 2023.
What you should do
Do not rely on AI outputs in hiring, performance review, or access decisions without human review. If you are building AI tools as a developer, test outputs across diverse inputs. Flag unexpected patterns to your team lead — this is increasingly a compliance issue, not just an ethics one.
Risk 3: Data Privacy
This is the risk most Indian IT professionals underestimate — and it has direct legal consequences under India’s new data protection law.
Never paste client data, personal information, or confidential company information into a public AI tool. When you type into ChatGPT, Claude, or Gemini on their consumer tiers, your input may be used to improve the model. A message containing a client's Aadhar number, medical record, salary information, or trade secret could become part of training data accessible to others.
INDIA'S AI GOVERNANCE — WHAT YOU NEED TO KNOW AS AN IT PROFESSIONAL
DPDP 2023Digital Personal Data Protection Act, 2023 — India's primary data protection law. Requires consent for processing personal data. Using someone's personal data as AI input without their consent may violate this Act. Your employer's data handling policies are built around this law.
AI GOV 2025India's AI Governance Guidelines 2025 — Require AI systems used in India to avoid bias, label AI-generated content, and maintain accountability for AI outputs. Platform obligations are now defined — companies deploying AI tools are responsible for their outputs.
COPYRIGHTCopyright Act + AI — DPIIT set up an expert committee in April 2025 to examine whether training AI on copyrighted works requires licensing. Until India's AI Bill is passed, the legal status of AI-generated content and training data remains evolving. Do not assume AI-generated code or content is automatically copyright-free for commercial use.
ERASURERight to Erasure challenge — Under the DPDP Act's right to erasure, if a user revokes data consent, you may be legally required to remove their data from an AI model — a technically complex "machine unlearning" problem that is already creating compliance challenges for Indian fintech and SaaS companies.
Risk 4: Copyright and Intellectual Property
The key question: When an AI generates code, an image, or a document — who owns it? In India in 2026, the honest answer is: nobody is completely sure yet.
Here is what is known and what remains unsettled:
AI-generated code — if you use GitHub Copilot or Claude to generate code at work, your employer likely owns that code under your employment agreement. However, if the generated code reproduces a segment from an open-source library with an incompatible licence (which Copilot has been found to do), your company could face a licence violation.
AI-generated written content — India’s Copyright Act 1957 protects works created by human authors. Content generated entirely by AI currently has uncertain copyright status in India. DPIIT’s April 2025 expert committee concluded that no existing international approach perfectly balances creators’ rights with AI innovation.
The practical rule for freshers — when using AI to generate work product for a client or employer, always disclose that AI was used, always review the output for licence-incompatible code, and always follow your company’s AI usage policy. Most major Indian IT companies have published internal AI usage guidelines in 2025–2026.
Responsible use — the practical checklist
BEFORE SUBMITTING ANY AI-ASSISTED WORK
1
Verify every specific fact, number, name, date, or citation independently. Do not assume AI is correct because it sounds confident. Look it up.
2
Never paste confidential client data, personal information, or trade secrets into a consumer AI tool. Use your company's approved enterprise AI tool if one is available.
3
Disclose AI use when submitting work to clients, managers, or academic institutions. Many now have explicit policies — follow them.
4
Review AI-generated code for licence compatibility before committing to a client repository. A hallucinated open-source library or an incompatible licence can create legal exposure.
5
Check outputs for bias when AI is used in any decision-affecting context — screening, summarising, or categorising information about people.
6
Know your company's AI policy — TCS, Infosys, Wipro, and HCL all have internal guidelines on approved tools, data handling, and disclosure requirements. Read yours on Day 1.
Dos and don’ts for Indian IT freshers
✓ DO THESE
Use AI to speed up first drafts — then edit carefully
Verify every fact, citation, and statistic independently
Disclose AI use when your employer or institution requires it
Use Gemini with search grounding when you need current facts
Read your company's AI usage policy on Day 1
Use enterprise-approved AI tools for client work
Flag AI-generated content that seems biased or inconsistent
Keep learning — AI tools change fast and so do the rules
✗ NEVER DO THESE
Paste client data or personal information into consumer AI tools
Submit AI output without reviewing it for factual accuracy
Cite AI-generated sources without verifying they exist
Assume AI-generated code is licence-compatible without checking
Use AI output in hiring or access decisions without human review
Claim AI-generated work as entirely your own when disclosure is required
Rely on AI for medical, legal, or financial advice without expert verification
Assume the same rules apply globally — India's laws are evolving fast
Why responsible use is a career advantage, not a burden
Here is the counterintuitive truth: being the person on your team who understands AI risks makes you more valuable — not less.
Indian IT companies deploying GenAI at scale are desperately looking for people who can serve as “responsible AI practitioners” — developers and analysts who understand both the capability and the failure modes. TCS’s agentic AI solutions and Wipro’s AI-powered operating model both require humans in the loop to review, validate, and govern AI outputs. That human-in-the-loop role is a well-paid, high-trust position.
The career opportunity: Being the fresher who asks "did we verify this AI output?" and "does this comply with our data policy?" is not being cautious — it is demonstrating exactly the professional judgement that companies are struggling to find. Responsible AI use is not the opposite of being AI-savvy. It is the next level of it.
The three things to remember
KEY TAKEAWAYS — MODULE 1.4
Hallucination is structural, not a bug — all LLMs fabricate facts at some rate (0.7% to 25%+). Confident delivery does not mean correct content. Always verify specific facts, citations, names, and data independently before using AI output in professional work.
Data privacy has legal teeth in India — the DPDP Act 2023 and India's AI Governance Guidelines 2025 create real compliance obligations. Never paste confidential or personal data into consumer AI tools. Know your company's AI policy before using any AI tool on client work.
Responsible use is a career advantage — the most valued AI practitioners are not the ones who use AI most, but the ones who combine AI capability with the judgement to catch its failures. That human-in-the-loop role is what Indian IT companies are actively hiring for right now.
Unit 1 — What you have learned
MODULE 1.1
What is Generative AI
LLMs predict the next word at massive scale. They can hallucinate. Context is everything. Understanding the mechanism is the foundation for everything that follows.
MODULE 1.2
ChatGPT vs Claude vs Gemini
ChatGPT for everyday tasks and UPI-friendly pricing. Claude for code and long documents. Gemini for research and Google Workspace. Start with zero-shot on all three — free.
MODULE 1.3
How Indian Companies Use GenAI
TCS, Infosys, Wipro, and HCL are deploying GenAI in production at scale. AI skills are a baseline expectation. Frame your knowledge in terms of business outcomes, not tool names.
MODULE 1.4
Risks and Responsible Use
Hallucination is structural. Data privacy has legal teeth under the DPDP Act. Responsible use is a career advantage — the human-in-the-loop role is what Indian IT companies are actively hiring for.
Unit 2 builds directly on this foundation — you will start writing prompts that actually work, and the hallucination rate you experience in Assignment 1.4 will drop noticeably as your prompts improve.
✎
Assignment 1.4 — Catch an AI Hallucination
The best way to understand hallucination is to experience it firsthand. This assignment takes 15–20 minutes.
Open any AI tool — ChatGPT, Claude, or Gemini (free tier is fine).
Ask it a specific factual question in your area of study or interest. Examples:
"What are the top 5 research papers on [your specialisation] published in the last 2 years? Give me titles, authors, and journal names."
"What are the exact eligibility criteria for [a specific government scheme or exam you know about]?"
"Who won the [specific recent award or competition in your field] in 2025?"
Verify every specific claim the AI makes — look up each paper title, check each author name, confirm each statistic independently online.
Document what you find — how many of the AI's specific claims were correct? How many were partially correct? How many were completely fabricated?
Write 3 sentences describing what you found and what it tells you about how to use AI responsibly in your work going forward.
This assignment completes Unit 1. You now have a foundational understanding of what GenAI is, which tools to use, how Indian companies are using it, and how to use it responsibly. Unit 2 builds on all of this — you will start writing prompts that actually work.