Skip to content

All lessons

Area 06 // AI SecurityLesson 106 / 14015 min read

Responsible AI basics

Responsible AI is the practice of building AI systems that are fair, private, transparent, safe and legally compliant, which in practice means checking for bias, protecting personal data, respecting copyright and licences, telling people when they're dealing with AI, keeping humans accountable, and following regulations such as the EU AI Act.

Think of it like this

Analogy

A pharmacy can sell powerful medicine because of rules around it: tested before sale, labelled with who it's for and the side effects, sold by trained staff who can say no, records kept, and regulators who check. Nobody thinks those rules are anti-medicine; they're what lets people trust it. Responsible AI is the same set of habits for AI systems.

The problem it solves

AI systems can treat groups of people unfairly, expose personal data, reproduce copyrighted work, mislead people who don't realise they're talking to a machine, and make high-stakes decisions nobody can explain. These harms damage people, brands and businesses, and regulators now set legal requirements. Engineers don't need to be lawyers, but they need to know the basics and build the checks in.

How it works, step by step

  1. Start with the use case and its stakes: who is affected, what decisions the system influences, what happens when it's wrong. A recipe bot and a loan-screening tool need very different care.
  2. Fairness and bias:
    • Models learn patterns from data, including historical unfairness.
    • Measure: outcome rates and error rates per group on an eval set (see the code); test with paired prompts that differ only in a name or gender.
    • Mitigate: better data, removing proxy features, adjusted thresholds, human review; re-measure (error analysis).
  3. Privacy: minimise personal data, have a lawful basis to process it (GDPR and similar laws), honour access and deletion requests, check provider retention and training settings (PII and data leakage).
  4. Copyright and licences: check licences for datasets, models and code (open-weight models have varied licences: open vs closed models); respect content rights in RAG sources; be careful with generated content that imitates specific works, people or brands; follow your provider's terms.
  5. Transparency: tell people when they're interacting with AI; label AI-generated media where required; document the system's purpose, data, limits and evaluation (model or system cards); give users a way to reach a human.
  6. Human oversight and accountability: a named owner for each system; human review for high-stakes decisions; ability to explain and contest outcomes (human in the loop).
  7. Safety and reliability: evals, guardrails, monitoring and incident response (guardrails, red teaming).
  8. Regulation (as of November 2026; always check the current legal text):
    • EU AI Act sorts uses by risk: some practices are prohibited (since February 2025); general-purpose AI model obligations apply since August 2025; transparency duties apply to chatbots and synthetic content; high-risk uses (hiring, credit, education, essential services and more) face strict requirements, now scheduled for December 2027 (and August 2028 for AI built into regulated products) after the 2026 "Digital Omnibus" changes.
    • Many other places have privacy laws, sector rules and AI guidance (for example the NIST AI Risk Management Framework in the US, and ISO/IEC 42001 for AI management systems).

See it in code

Needs
Nothing: plain Python 3. The numbers are a made-up evaluation set.

A first fairness check: approval rates per group and the ratio to the best-treated group.

fairness_check.py
# A basic fairness check: does an AI screening tool approve groups at very different rates?
decisions = {                                   # (approved, total) per group, from an evaluation set
    "group A": (180, 300),
    "group B": (95, 250),
    "group C": (140, 240),
}
rates = {g: a / t for g, (a, t) in decisions.items()}
best = max(rates.values())
for g, r in rates.items():
    ratio = r / best
    flag = "  <- below 0.8 of the best rate: investigate" if ratio < 0.8 else ""
    print(f"{g}: approval rate {r:.0%}, ratio to best {ratio:.2f}{flag}")
# The "four-fifths rule" is a screening heuristic, not proof of bias either way:
# look at the data, the features used and the error rates per group before deciding.
Output
group A: approval rate 60%, ratio to best 1.00
group B: approval rate 38%, ratio to best 0.63  <- below 0.8 of the best rate: investigate
group C: approval rate 58%, ratio to best 0.97

A gap isn't automatically unfair (groups may differ in relevant ways), and equal rates don't prove fairness (error rates may still differ). The point is to measure, then investigate with people who understand the domain.

A real example

Example

A hiring assistant, done carefully

An HR team wanted an AI assistant to summarise CVs against job requirements. Their responsible-AI review classed it as high-risk (hiring), so: the assistant summarised and highlighted evidence but never ranked or rejected candidates; names, photos, ages and addresses were removed before the model saw CVs; paired tests swapping names and genders were added to the eval suite, which revealed and led to fixing a tendency to describe identical experience differently; candidates were told AI assisted the review and could ask for a fully human one; and every decision was made and logged by a recruiter. A system card documented all of this for legal and works-council review.

The easy way

Start here

For any AI feature, answer five questions in a short document: who could be harmed and how? What personal data do we use and why? Do we have the rights to our data, model and content? Do users know it's AI? Who is accountable and how can people get a human? Revisit it at every major change.

When you grow: the production setup

Organisations run AI governance: an inventory of AI systems with risk classification, impact assessments for high-risk uses, fairness and safety evals in CI, data protection reviews, licence tracking, system cards, human-oversight procedures, incident reporting, and mapping to regulations and standards (EU AI Act, NIST AI RMF, ISO/IEC 42001), with legal and domain experts involved.

Trade-offs

What you gain

  • Fewer harms to people and fewer legal and reputational risks.
  • Builds user trust and makes adoption easier.
  • Forces clarity about purpose, limits and ownership.

What it costs you

  • Takes time and cross-team effort.
  • Fairness definitions can conflict; trade-offs need judgement.
  • Regulations change and differ by country.

When not to use it

  • There's no situation where it doesn't apply; scale the effort to the stakes of the use case.

Common mistakes

  • Treating responsible AI as a final checkbox instead of part of design.
  • Measuring overall accuracy but never per group.
  • Fully automating high-stakes decisions with no human review or appeal.
  • Ignoring licences of open-weight models and datasets.
  • Not telling users they're talking to AI.

Interview questions

Try answering out loud first, then open the model answer.

How would you check an AI system for bias?

Model answer

Define the groups and harms relevant to the use case; build an eval set with group information (or proxies, carefully); compare outcome rates, error rates (false positives and negatives) and quality per group; run paired tests that change only protected attributes; investigate gaps with domain experts; mitigate through data, features, thresholds or human review; and monitor in production.

What does the EU AI Act mean for an engineer?

Model answer

It regulates by risk: some uses are banned; general-purpose model providers have documentation and transparency duties; chatbots and generated content need transparency (users should know it's AI, synthetic media labelled); and high-risk uses (hiring, credit, education and similar) require risk management, data governance, logging, human oversight, accuracy and documentation, with deadlines moved to late 2027 and 2028 as of November 2026. Engineers implement logging, evals, oversight and documentation, with legal guidance on classification.

What goes into a model or system card?

Model answer

Intended use and out-of-scope uses, model and data sources, evaluation results (overall and per group), known limitations and risks, safety measures, human-oversight arrangements, versioning, contacts and how to report problems.

Quiz

Pick an answer to see whether it’s right and why.

  1. 01What's a simple first check for bias in an AI screening tool?
  2. 02What does the EU AI Act base its rules on?
  3. 03What is a model card?
  4. 04A user doesn't know they're chatting with a bot. Which principle is at stake?
  5. 05Why keep a human accountable for high-stakes decisions (loans, hiring, medical)?

0 of 5 answered

Key terms

Bias (in a neuron)
A learned number added to a neuron's weighted sum, shifting how easily it activates. Not the same as unfair bias in data.
Fairness (AI)
Whether an AI system treats people and groups equitably, measured with outcome and error rates per group and judged in the context of the use case.
Transparency
Being open about AI use, such as telling people when they deal with AI, labelling generated content, and documenting how a system works and its limits.
EU AI Act
The European Union's AI law, which sets obligations by risk level, from banned practices to strict rules for high-risk uses like hiring and credit, plus duties for general-purpose models.
Model card
A short document describing a model's intended use, training data, evaluation results, limitations and risks. A system card does the same for a whole AI product.
Human oversight
Keeping a responsible person able to monitor, review, override and stop an AI system, especially for high-stakes decisions.

Before and after

Read first

Read next

Progress is saved in this browser only.