Sample | AI governance record
Sample: the governance record of a live AI assistant
I wrote this review method for other people's AI systems, so my own system went through it first. It has not passed yet.
The system is the AI answer box on jasminahomes.ca/search-ai, a real-estate website I built and run. Nothing here is invented: the proposed verdict is a conditional yes, with three conditions still open.
No regulator asked for this review. OSFI's Guideline E-23 does not apply to a real-estate website. I ran the 26-question intake from the bank example anyway, because a method you would not run on your own system is not a method.
Read the reviewer pack (PDF, 8 pages)
No regulator behind your AI either? The three open conditions below are questions for any assistant in front of customers: where what people type goes, what the vendor's terms allow, and whether you can switch it off.
The bottom line
Proposed verdict: CAVEAT, a conditional yes. This is my assessment for review, and nobody has approved it. The assistant only reads public pages and takes no actions, but three conditions on privacy, vendor terms and shutdown must be met first, and two of them could still change the verdict.
Two rounds, each dated. October 2026 (runs on the 15th and 16th, same result): six open conditions, score 50. 9 November 2026: three open conditions, score 60. The three that closed did so on my written confirmation as the owner, and this page says so wherever that is the only evidence.
What the score counts. The share of recommended safeguards that are in place: 6 of 10 as of 9 November 2026, where 100 would mean all of them. It counts what is present, so it says nothing about how good the answers are. On that, the record holds a 16-case test harness and a question set I rate by hand, and it names the gap: nothing grades the answers automatically between my reviews.
What has not happened yet. The pack was prepared before the security, architecture and privacy reviews it asks for. The record lists no independent security testing and no formal privacy assessment so far, and it has no approver and no next review date set.
Three open conditions, each with an owner
Approval is proposed on these three. The record names the function that closes each one and the function that signs off, the way an institution's record would. A smaller company may have one person in several of those seats, and the record still names each seat. So far it names one person: me, as business owner and risk owner.
1. A privacy ruling on what visitors type
- •Open: a formal privacy assessment rules on personal information that visitors volunteer, and the data-flow contract is signed off or a filter is added in front of the model.
- •Why it matters: the assistant never asks for personal details, but a visitor can type them. They reach OpenAI in the United States, Algolia's 30-day conversation log and my analytics copy. The privacy notice has disclosed this since 15 October 2026; the data-flow contract is still a draft.
- •Who acts: privacy closes it. I sign off that it is closed, as business owner and risk owner.
- •Could it change the verdict? Yes. The intake method's example rules do not permit personal information to go out to a model in a vendor's tenant. If privacy rules that what visitors type counts, the verdict changes.
2. Vendor terms on file
- •Open: Algolia's data region, OpenAI's retention and no-training terms for this route, and any data processing agreement, on file.
- •Why it matters: if the vendor terms allow training on visitor text or long retention, the vendor-embedded route fails.
- •Who acts: third-party risk closes it. Privacy and security sign off that it is closed.
- •Could it change the verdict? Yes.
3. The kill switch, exercised once
- •Open: the written kill-switch runbook exercised once, with the date recorded.
- •Why it matters: reviewers need proof that the assistant can be switched off, and the vendor left, quickly during an incident. I wrote the runbook on 9 November 2026 (stop the agent, remove the Ask tab, roll back the prompt, leave Algolia). Nobody has run it yet.
- •Who acts: the platform owner closes it. Security signs off that it is closed.
- •Could it change the verdict? No. It is a condition of the conditional yes.
The timeline: October, November, and what changed between them
Each run asks the same 26 questions and answers them from the project's own documents. The questions cover who owns the system, what it does, the model and its provider, the data and where it goes, what the system can reach, how it is limited, logged, tested and switched off, and who decides. A condition closes only on evidence, and the evidence is named beside it.
October 2026: the first round
- •Result: CAVEAT, six open conditions, score 50 (5 of 10 recommended safeguards in place). The run of 15 October 2026 and the rerun of 16 October 2026, after the two changes below, gave the same result.
- •The six conditions: a privacy ruling on personal information that visitors volunteer; vendor terms on file; a named risk owner and a written statement of who the data controller is; confirmation that the AI credit is a hard stop, plus a spend alert to a named person; a dated decision on Algolia's Model Context Protocol (MCP) setting, which was on with no recorded decision; a written kill-switch runbook, exercised once.
- •Changed that week, the notice: since 15 October 2026 the site's privacy notice names Algolia, OpenAI and Google Cloud as the chat providers, says the processing is in the United States, and discloses a copy of each conversation kept in my own Google Cloud project to improve the assistant and the guides, deleted on request. Evidence: the published notice.
- •Changed that week, an accepted risk: on 16 October 2026 I recorded that analytics copy as an accepted risk, for analytics use only, with a review date of 16 October 2027. Evidence: the dated entry in the record's decision log.
- •What that did not close: the notice was one part of the privacy condition. The formal assessment was still missing, so the count stayed at six.
9 November 2026: the second round
- •Result: CAVEAT, three open conditions, score 60 (6 of 10).
- •Closed, the roles: I named myself the risk owner and stated that Jasmina Homes is the data controller. Evidence: my written confirmation as the owner, dated 9 November 2026. That is the only evidence, and the record says so.
- •Closed, the MCP setting: a dated decision to leave the setting on, because it only changes the shape of the agent's internal search tool. Evidence: my decision of 9 November 2026 in the decision log, and a configuration snapshot taken that evening that matches it.
- •Closed, the spend: I confirmed that the AI credit is a hard stop, and that an automated check puts a test question to the live agent three times a day and emails me when it fails. Evidence: my statement as the owner for the hard stop, to be checked on Algolia's plan page at the next review, and the check's schedule in the project's files. The alert comes after the fact, and there is still no spend threshold.
- •What moved the score: I confirmed that the agent runs on one named model snapshot, gpt-5.6-luna, and no moving alias. In October the record could not say whether the version was pinned. Version pinning now counts as in place, so 5 of 10 became 6. Evidence: my confirmation, the same configuration snapshot, and OpenAI's model page, which lists that name as a snapshot. OpenAI does not state whether the model behind the name can still change, and the record keeps that as a gap.
- •Written, not yet closed: the kill-switch runbook. Its condition closes once I have run it and recorded the date.
What comes next
- •First, the drill: run the kill-switch runbook once and record the date.
- •Then the vendor terms and the privacy assessment. Those two can still change the verdict, so the yes stays conditional until they close.
- •At the next review: check the hard stop on Algolia's plan page and record the date.
- •Not set yet: an approver for this record, and a next review date.
This page describes the record as of its run of 9 November 2026.
One method, two settings
The same 26-question intake, run on an invented bank assistant and on my own live one. Both records use the same three verdict words (PROCEED, CAVEAT, REFUSE), name an owner for every open item, and tag their risks on the same open FINOS catalogue.
The regulated setting is on the E-23 page: UC-0417, a campaign-brief assistant inside a financial institution. It is invented and labelled as invented, with no institution behind it. It shows what the record looks like inside a model-risk review.
The live setting is this page: an AI answer box on a public real-estate website, run by the person who wrote the method. Everything in it is real, the gaps included. It shows what happens when the method meets a real system.
No regulator required the second one. E-23 is OSFI's model-risk guideline for federally regulated financial institutions, so this record shows the intake method on a live system and is not evidence of any institution's E-23 readiness. What applies to credit unions and to institutions the AMF supervises is set out on the E-23 page, and a company with an AI feature in front of customers can use the same intake where no regulator requires a review.
In both settings the record prepares a decision, and approval stays with the organization's own functions.
The facts the verdict rests on
This table scrolls sideways.
| Fact | What the record says | Status |
|---|---|---|
| Data sensitivity | Public. Only pages already published on the site reach the search index. | Proposed (data steward) |
| Personal data | None indexed or asked for, but a visitor can volunteer personal information and it is kept with the conversation record. The contact form never reaches Algolia or OpenAI. User memory is disabled and must stay disabled. Payment card information: no. Personal health information: no (proposed). | Proposed (privacy) |
| Direction of data | Data goes out to the model: visitor text plus retrieved public passages go to OpenAI through Algolia. The output is display text with links. Nothing is written into a system of record. | Proposed (third-party risk) |
| Hosting and retention: Algolia, the agent host | Software-as-a-service tenant. Conversations (metadata, messages and tool calls) kept 30 days; answer cache lifetime undocumented; data region unknown. | Missing (third-party risk) |
| Hosting and retention: OpenAI, the model provider | Processing in the United States. Retention and no-training terms for this route unknown. | Missing (third-party risk) |
| Hosting and retention: my analytics copy | A copy of each conversation in my own Google Cloud project (US multi-region), to improve the assistant and the guides; deleted on request; disclosed in the privacy notice (15 October 2026); risk accepted (16 October 2026). | Stated |
| Autonomy | Read-only: one answer per visitor message, up to 10 steps, no actions. It runs only when a visitor clicks the Ask tab. | Stated |
Status words. Stated: the answer is in one of the project's own documents. Proposed: inferred, and the function in brackets must confirm it. Missing: the function in brackets owns supplying it.
Who would review it, and how deeply
This table scrolls sideways.
| Function | Why, in this record | Proposed depth |
|---|---|---|
| Privacy | Personal information that visitors volunteer goes to OpenAI in the United States, stays 30 days in Algolia's conversation log and is copied to my analytics storage. The privacy notice discloses this; the data-flow contract is a draft; there is no formal privacy impact assessment. | Full review |
| Security | A public search key behind a domain allow-list that a forged Origin header gets past; the vendor's content-safety filter is off; the MCP setting is on by my decision; no independent security testing. | Full review |
| Platform or cloud owner | Platform owner: the AI credit is a hard stop by my statement, to be verified on the plan page at the next review; the alert is a failure email after the fact, with no spend threshold; the kill-switch runbook is written but not exercised. Cloud owner (light touch): my own project holds the analytics copy. | Full review |
| Third-party risk | An Algolia to OpenAI chain inside a vendor product; Algolia's data region and OpenAI's terms are unknown. | Full review |
| Model risk | A large language model agent pinned to one named snapshot. It makes no decision about a person. | Light touch |
| Data steward | Confirm that only published public content enters the search index. | Light touch |
| Architecture board | Check whether the rule that the paid AI runs only when a visitor clicks Ask covers every application that calls the agent. | Light touch |
Proposed by the record, for the organization's own functions to confirm or change. Model risk is proposed as a light touch because the assistant makes no decision about a person; an institution's own tiering could say otherwise. The 8-page pack lists these functions beside the 14 catalogue risks that apply or are likely, each with its main gap.
Appendix: what the score counts
This table scrolls sideways.
| Recommended safeguard | Part of the design | Counted as in place? |
|---|---|---|
| Model version pinning (mi-10) | The model | Yes, since 9 November 2026: one named snapshot |
| Filtering between users, the application and the model (mi-3) | The model | Yes: a domain allow-list and a bot check, both of which can be bypassed |
| Data leakage prevention and detection (mi-1) | The model | No: there is no filter on chat text |
| Automated answer grading by a second model (mi-15) | The model | No: none is described |
| Least privilege for the agent (mi-18) | The agent | Yes: one read-only search tool and a search-only key |
| Audit trail of the agent's decisions (mi-21) | The agent | Yes: messages and tool calls are logged for 30 days |
| Isolation between agents (mi-22) | The agent | No: not present. The design has a single agent, and the count does not adjust for that |
| Data quality and classification (mi-6) | The search index | Yes: only published pages reach the index |
| Source access controls preserved (mi-16) | The search index | Yes: every page in the index is public, so there are no per-user permissions to preserve, and it still counts |
| Validation of tool inputs (mi-19) | The search tool | No: input checks are not described |
Six yes out of ten: score 60. In October it was five. For each AI component in a design, CALM Studio, a tool in FINOS's open architecture-as-code project, recommends a set of mitigations from the FINOS AI Governance Framework, and this design has ten recommended in all. A safeguard counts when it is present, including a weak one, so the score is a count and never a grade of the system or of its answers. Safeguard names are plain-language versions of the framework's mitigation titles; the IDs in brackets are the framework's (CC BY 4.0).
Appendix: the 26 questions
Every record answers the same 26 questions, numbered 0 to 25, in this order. Each answer is marked stated, proposed or missing, with its source.
Who and what (0 to 4)
Use case ID; name and one-line purpose; business, technical and risk owner; stage; the job it does.
The model (5 and 6)
Kind of intelligence; the models and providers under it.
The data and the people (7 to 12)
Highest data classification; personal, payment and health data; who is affected and what is decided about them; direction of data; where it runs, in which country and for how long; bought inside a vendor product or brought in.
What it can do (13 to 16)
Identity and access; what it can reach and change; how much it may do without a person; fixed checks and spending limits.
How you know, and how you stop it (17 to 21)
Logging and audit trail; evaluation and monitoring; threat model and testing; privacy impact assessment; kill switch, rollback and exit.
The decision (22 to 25)
Risk rating, route and verdict; reviewer, approver and dates; what others can reuse; the decision log.
The same review, on your system
Want this for the AI systems you answer for?
For a bank or an insurer under OSFI, the E-23 Readiness Review gives up to a dozen use cases one record each in three weeks, inside the model-risk review you already run. The same page says how it maps for credit unions and for institutions the AMF supervises.
See the E-23 Readiness ReviewNo regulator asking? Start with the AI Use Case Triage: one 90-minute session on up to eight AI uses, live or planned, and a two-page decision brief within 48 hours, for USD 1,500. If you want the same review as this page for one system, write to me at [email protected].