Deploying & Evaluating AI Apps (LLMOps)

You built an AI feature. Now keep it trustworthy in production. This is LLMOps: the discipline of measuring, guarding, observing, and operating an AI app once real users depend on it. Build an eval set and run offline and online tests, use LLM-as-judge (and know when not to), add guardrails for injection, PII, and moderation, instrument tracing and cost/latency/token dashboards, make calls reliable with timeouts, retries, fallbacks, and caching, cut cost and latency, and ship responsibly with staged rollout, a kill switch, rollback, red-teaming, and drift monitoring. Throughout we carry the Learn.WitUS trust DNA: you can't improve what you don't measure, and you treat every model output as untrusted. F2 (Building with AI) is the recommended prerequisite.

Coming soon

This course is being reviewed

Every course on Learn.WitUS is checked against its sources before it opens, so the lessons are not available yet. What the course covers is above, and any academic standards it meets are listed with it. Enrollment opens when the review is finished.

Want to help this course open? Test it, or vet it →

Put your name in

A course is checked against its sources before it opens. If you want to read it early and send notes, or you know this subject well enough to check the claims, say so here. The instructor decides who reviews their own course, so this is a request, not a login.

How would you like to be involved?

I want to take this courseTell me when it opens. Nothing else is expected of you.I will test this course before it opensRead it early and send notes on what is confusing, broken, or wrong. Nothing you do is recorded, so no progress, no scores, no certificate.I can vet this course as a subject matter expertCheck the claims and the sources against what you know. Tell us your background below so the instructor can weigh it.

Your name

Email address

How we reply, and how a review invitation would reach you.

Phone number (optional)

Include your country code, for example +44 20 7123 4567 or +1 555 0100.

Anything else (optional)

Anything you want the instructor to know.

Your phone number and your background are shown only to this course's instructor and this school's admins. They are never published, never sold, never added to a mailing list, and never sent to any other service. Both are optional: leave them blank and email is enough.

Your name, your email address, and which of the three you picked go to the WitUS team's inbox so a person actually reads this and replies.

Already studying this course? Sign in and it will open as usual.