Lesson 22 of 22Advanced16 min

Using AI Tools as a Developer

By the end of this lesson

Use AI assistants to learn faster while verifying everything you accept.

This lesson is about the other side of the course: not the feature you are building with a model, but the model helping you build it. The same properties apply, because it is the same technique. Fluent output, no reliable sense of its own uncertainty, and a confident answer whether or not the answer is right.

That leads to one rule that organises everything else. An assistant is at its strongest where you can verify the result in seconds, and at its weakest where verification is slow or you would not know how. Generating a test you then run is the strong case: the test passes or it does not, and you find out immediately. A claim about how a library behaves under load is the weak case, because nothing tells you it is wrong until production does.

None of that is an argument for avoiding these tools. Used with a verification habit they will teach you a codebase faster than reading it alone. Used as an oracle they will hand you plausible code you cannot explain, which is the worst artefact you can commit.

Where an assistant reliably earns its place, and what to ask for in each case:

Explaining unfamiliar code: The strongest everyday use. Paste a function you did not write and ask what it does, what it assumes, and what would break it. The value is that it gives you vocabulary and a starting model of the code, which makes reading the rest much faster. Verify by running the code, or by finding the behaviour it describes in a test.
Generating test cases: Ask for the edge cases you have not thought of — empty list, single element, duplicate ids, a date crossing a month boundary, a claim with no receipt. It is genuinely good at enumerating cases and only adequate at writing the assertions, so read each test and check it fails when you break the code on purpose.
Drafting boilerplate: Configuration, a pydantic model matching a JSON payload, the shape of a client with retries, a migration. Structural work you can check by eye against a schema or a reference. The saving is real and the risk is low, provided you read what you accepted.
Reviewing a diff: Ask what a change misses rather than whether it is good. Unhandled failure paths, a missing index, an off-by-one in a slice, a case the tests do not reach. Treat every finding as a hypothesis to check, because it will raise things that are already handled elsewhere in the file.
Naming and finding the standard approach: "What is the usual name for this pattern" and "does the standard library already do this" are small questions with disproportionate value, because they turn an unknown unknown into a term you can search properly.

Give it the right scope, then verify with the commands you already had

Shell

# Review what you are about to commit, not the whole repository
git diff --staged > /tmp/review.diff
wc -l /tmp/review.diff            # 2,000 lines is not a review, it is a wish

# Narrow further when the change is large: one file, with context
git diff --staged -- src/assistant/retrieval.py > /tmp/review.diff

# After accepting any suggestion, the same commands as always. Nothing here is new.
ruff check . && mypy src && pytest -q

# The question an assistant cannot answer for you: does anything test this line?
pytest --cov=src --cov-report=term-missing -q

# And the one that catches a confidently invented dependency
pip install --dry-run -r requirements.txt

Scope is the setting that most affects whether a review is useful. A diff of a few hundred lines produces specific findings; a whole repository produces generalities about naming and structure that apply to any codebase.
Reviewing the staged diff rather than the working tree means you are reviewing the change you are actually about to commit, which is a habit worth having independently of any assistant.
The lint, type-check and test commands are the verification loop. An assistant does not replace any of them, and the reason to run them immediately after accepting a suggestion is that you want the failure attached to the change that caused it rather than to a batch of five.
The coverage report answers the question people most often assume the assistant answered. Generated tests can look thorough and miss the branch you changed, and term-missing tells you which lines nothing touches.
The dry-run install is a small, specific defence against a plausible-looking package name that does not exist, or exists and is not the library you were told about. Check a new dependency's real name, its repository and its maintenance before it enters the file.

A prompt shaped to produce something checkable

Text

Explain what this function does, line by line, for someone who has not seen
this codebase. Then answer these three separately:

1. Which behaviour here depends on a library default rather than on this code?
2. Which lines change behaviour if the passages list is empty?
3. Exactly which part of which library's documentation should I read to confirm
   your answer to question 1?

Do not suggest improvements yet. If you are not sure how a library behaves,
say you are not sure rather than describing what it probably does.

<the function>

The line-by-line explanation is the part you wanted. Everything after it exists to convert a fluent description into something you can check against a source that is not the model.
Question 1 separates the code in front of you from assumptions about libraries. That boundary is where the confident wrong answers concentrate, because a library default is exactly the kind of detail that sounds authoritative and varies by version.
Question 2 is the cheap correctness probe. You know what the empty case should do; if the answer gets it wrong, you have learned something about how much of the rest to trust before you have spent any time on it.
Question 3 is the most useful line in the prompt. Asking where to confirm an answer gives you a reference to open, and it reframes the exchange from an answer you accept to a pointer you follow.
Holding back suggestions matters more than it sounds. Ask for an explanation and improvements together and you get a rewrite, which means you are now reading unfamiliar code and unfamiliar changes at once, with no way to tell which part you failed to understand.
The instruction to admit uncertainty helps somewhat and does not solve the problem. A model has no dependable signal about which of its statements are shaky, so treat an unhedged claim about library behaviour as unverified rather than as confident.

A plausible suggestion, and what verifying it actually changed

Python

# Drafted in one keystroke, in response to "total this employee's claims for a month".
def month_total(employee_id: str, month: str) -> int:
    rows = db.fetch(CLAIM_SQL, employee_id, month)
    return sum(row["amount_pence"] for row in rows)


# It reads correctly and it is wrong in two ways, neither of them visible here.
#
# 1. The explanation offered alongside it: "db.fetch returns an empty list when
#    nothing matches." Confidently stated, and not true of every driver — some
#    return None, and sum() over None raises TypeError. Checked against the
#    driver's own documentation, not against the explanation.
#
# 2. Rejected and withdrawn claims are rows too. The assistant could not know
#    that, because the rule lives in the finance policy and in a status column
#    it was never shown. This is the class of error no amount of reading the
#    code would surface.

REIMBURSABLE = {"approved", "paid"}

def month_total(employee_id: str, month: str) -> int:
    rows = db.fetch(CLAIM_SQL, employee_id, month) or []
    return sum(row["amount_pence"] for row in rows if row["status"] in REIMBURSABLE)


def test_month_total_ignores_rejected_claims():
    seed_claims(
        [
            {"amount_pence": 5_000, "status": "approved"},
            {"amount_pence": 9_900, "status": "rejected"},
            {"amount_pence": 1_250, "status": "paid"},
        ]
    )
    assert month_total("EMP04417", "2025-02") == 6_250


def test_month_total_handles_no_claims():
    assert month_total("EMP04417", "1999-01") == 0

The first version is the normal output of an assistant given a reasonable request: idiomatic, readable, and short enough that it invites acceptance without much thought. Nothing about its appearance signals a problem.
The claim about the driver returning an empty list is the failure mode this lesson exists for. A confident explanation of a library's behaviour may be wrong, and it is wrong in a way that no reading of the snippet reveals. The fix is not scepticism in general, it is opening the driver's documentation for that one sentence.
The rejected-claims bug is a different kind, and the more expensive one. The rule is not in the code, so an assistant working from the code cannot infer it. Only someone who knows the domain catches this, which is the clearest statement of what you have to supply.
The two tests are the part to write yourself, and they are what makes the whole exchange safe. Verification you can run in a second is exactly where these tools are strongest — and note that the assistant would very likely have written these tests correctly if asked for them after you had established the rule.
The standard to apply before committing any of this: could you explain each line to a colleague who asked why? For the second version you can, including why the status filter is there. For the first you could only repeat what you were told.

The same tool, two situations. The difference is how quickly you can find out whether it was right:

	You can verify in seconds	You cannot verify quickly
Typical request	Draft this test, convert this payload to a model, explain this function	How does this library behave under concurrency, is this query fast enough, is this design sound
How you check	Run it, type-check it, read it against a schema	Documentation, a benchmark, a load test, a colleague who knows
Cost of a wrong answer	Seconds, and a red test	Discovered in production, in a change nobody suspects
Why it is hard to spot	It is not. The failure is immediate	The answer is fluent, specific, and indistinguishable from a correct one
Sensible use	Freely, at speed, as a draft you read	As a source of candidate answers and search terms, then verify elsewhere
In this course's project	The pydantic models, the test seeds, the retry decorator	Chunk size for your documents, whether a smaller model suffices, provider rate limit behaviour

Watch out

Licence and confidentiality: check before you paste, not after

What you paste into a third-party tool leaves your organisation. Proprietary source, a customer's data, an internal document, a log containing personal data, a credential in a configuration file — all of it is a transfer to a supplier, and pasting it may breach an agreement your employer has signed. Many companies have a policy on exactly this. Find out what yours says before you need it, and if there is no policy, ask rather than assuming permission.

The provider's terms matter for the same reasons as in the security lesson: retention, whether input is used to improve models, which region processes it, and whether the paid tier you are on differs from the free one. Business and enterprise agreements often answer these differently, and the difference is contractual.

Generated code can also carry licence implications. These models are trained on large quantities of public code under a range of licences, and there is no reliable way to tell from a suggestion whether it closely reproduces something licensed in a way your project cannot accept. The risk is highest for a distinctive block — a recognisable algorithm, a long fragment, something you could search for verbatim. Where a suggestion looks like it came from somewhere specific, check, and prefer writing it yourself.

Practical habits that cost almost nothing: redact before pasting, use placeholder names and figures, never paste a key or a token, work from a reduced example rather than the real file when the real file contains anything sensitive, and check whether your organisation provides an approved tool with terms already reviewed.

Say plainly what this warning is not. It is not a reason to avoid these tools. It is the small amount of care that keeps a genuinely useful habit from becoming a problem for somebody else in your organisation.

Common mistake

The habits that turn a useful tool into a liability

Accepting code you could not explain. This is the one that matters. If you cannot say why each line is there, you cannot review it, debug it at 2am, or defend it when someone asks. The rule that holds up: never commit a change you could not explain to a colleague who asked why.

Trusting a confident statement about a library. Version-specific behaviour, default arguments, return types on an empty result and error classes are the exact details these tools get wrong while sounding certain. One minute in the real documentation settles it, and the habit of checking that one sentence is most of the discipline.

Asking for too much at once. "Add caching to the retrieval layer" returns a large diff touching several files, and reviewing it properly takes longer than writing it. Ask for one function, read it, run it, then ask for the next.

Letting it write the tests for code it just wrote. The tests will encode the same misunderstanding and pass convincingly. Establish the expected behaviour yourself, from the requirement, then let it help with the mechanics.

Using it to skip the fundamentals. An assistant makes it possible to produce working code in a language or framework you do not understand, and that is a trap rather than a shortcut — you will be unable to tell a good suggestion from a bad one, which is precisely the judgement the tool cannot supply. Use it to learn faster, by asking why and following the reference, not to avoid learning.

Pasting whatever is on the clipboard. Logs contain personal data, configuration contains secrets, and a stack trace can contain both. Reduce the example first.

Treating its review as sign-off. It raises hypotheses. Some are already handled two lines above, some are wrong, and the ones that are right still need a human to decide whether they matter here.

Summary

An assistant is strongest where you can verify the result in seconds and weakest where verification is slow or you would not know how
Use it to explain unfamiliar code, enumerate test cases, draft boilerplate and review a scoped diff — then run your usual lint, type and test commands
A confident explanation of a library's behaviour may be wrong; check the one sentence your code depends on against the real documentation
Never accept a change you could not explain to a colleague, and never let it write the tests for code it just wrote
Check your organisation's policy and the provider's terms before pasting proprietary code or data, and treat a distinctive generated block as carrying possible licence implications

Practice

Attempt each one before opening the solution. Getting it wrong first is how the idea sticks.

Try it yourself

Verify one claim, properly

Take a function from a library you use but have not read — a database driver, an HTTP client, a validation library. Ask an assistant to explain its behaviour on the empty or missing case, and to name where in the documentation that is stated.

Then open that documentation and check. Note whether the answer was right, whether the reference existed, and how long the check took.

Show solution

The common outcomes are worth predicting before you start: the answer is right and the reference is roughly right; the answer is right and the reference points at a page that does not say it; or the answer is subtly version-specific and the documentation says something different for the version you have installed. All three are useful, and the third is the one that would have reached production.

Time the check. It is usually under two minutes, which is the number that matters — the objection to verifying is always that it is slow, and measuring it once removes the objection.

A reference that does not exist is the most informative result you can get, because it demonstrates the mechanism directly. A plausible documentation path is the same kind of output as a plausible function name: likely-looking text, produced by the process described in the first module of this course.

Why this exercise rather than a coding one: the failure this lesson is guarding against is not bad code, which you would notice. It is a correct-looking claim about something you did not check, and the habit that prevents it is small and specific — verify the one sentence the code depends on.

Worth doing on a library you think you know. That is where an unverified assumption has been sitting longest.

Think about it

Which of these four would you delegate?

You have four tasks on the knowledge assistant. Write the pydantic models for a provider's tool-call payload. Decide the chunk size for your own handbooks. Add retries and a timeout to the embedding client. Work out why answers about one policy document are consistently ungrounded.

Decide what you would ask an assistant for in each case, and what you would keep for yourself.

Show solution

The pydantic models are the clearest delegation on the list. You have the payload in front of you, the output is checkable by eye and by running it, and a mistake surfaces as a validation error immediately. Paste a redacted sample response and read what comes back.

Chunk size is not delegable, and the reason is worth stating: it depends on your documents, your embedding model and your retrieval results, none of which the assistant can see. What you can usefully ask for is the list of things to measure and a script that reports retrieval quality at three settings — then you run it on your own content and decide.

Retries and a timeout sit in between. The shape of the code is standard and a good candidate for a draft, and the parameters — which exceptions are transient, how many attempts, how long a person will wait — are decisions about your system. Take the structure, set the numbers yourself, and check the exception classes against the current SDK rather than trusting the ones in the suggestion.

The ungrounded-answers investigation is yours, and it is the most valuable use of your time on the list. It needs the retrieved chunks, the assembled prompt and the source document side by side. An assistant is genuinely useful inside it — reading a chunk and telling you what it appears to be about is a fast way to spot a split that went wrong — but the diagnosis depends on data it never sees.

The pattern across the four is the one from the comparison table. Delegate where you can check the result in seconds, keep the decisions that depend on your own data and your own users, and remember that the tasks you cannot delegate are the ones worth developing judgement in.

Saved in this browser only.

End of the published lessons

That is everything written so far in AI & Generative AI

More lessons in this course are on the way. In the meantime, the course page shows the full roadmap, and the projects are the best way to consolidate what you have covered.