Last Thursday, in an agency I will not name, a principal walked past one of his best account managers and saw the screen. Open in a browser tab, in a personal ChatGPT account, was a Medicare client's date of birth, plan ID, and the first half of a dec page. The account manager had pasted them in twenty minutes earlier to get a faster plain-English summary before an outbound call. She did not think she had done anything wrong. By every measure that mattered to her, she was right to want the time back. By every measure that matters to the agency's E&O carrier, she had just created a problem.
That moment is happening in your agency too. Probably this week. Probably more than once.
This is Shadow AI, and it is not a productivity question or a technology question. It is a liability question, and it is the one your principal-level peers are quietly losing sleep over for very good reasons. This guide lays out where the exposure actually lives, why the regulatory direction is squeezing it from above, what an AI policy actually means for an agency your size (hint: it is not a forty-page document), and the order of operations that lets you say yes to AI safely instead of letting your team go around you to do it anyway.
The state of agency AI use right now
Two things are true at the same time, and the gap between them is your entire problem.
The first is that your team is already using AI. MIT's research found that workers at more than 90% of companies already use personal AI accounts for daily work, while only about 40% of companies have actually bought sanctioned tools. An agency is not exempt from that. It is, if anything, more exposed than the average company, because the work your team is doing with those tools is heavy with client information.
The second is that nobody has written anything down. IBM's 2025 Cost of a Data Breach report found that 63% of organizations still have no AI governance policy, and that breaches involving heavy shadow AI ran about $670,000 higher on average and took 247 days to detect. Two-thirds of a year to find out, and a six-figure premium on the cost when you do.
So the picture for the average agency right now is high adoption, no policy, and a long detection lag. That is the perfect environment for an E&O surprise, which is the kind of surprise you do not want.
What Shadow AI actually looks like inside an agency
Let me describe this concretely, because "shadow AI" is one of those terms that sounds dramatic and ends up meaning nothing.
Shadow AI is any AI tool your team is using on agency work that the agency itself has not approved, configured, or supervised. The defining feature is not the tool. It is the absence of oversight.
You will find it in five places in almost any agency. The account manager who pastes a client's dec page into a personal ChatGPT to get a faster plain-language summary for the phone call. The producer who drafts a client email in a free chatbot, complete with the client's name, conditions, and policy details. The service rep who runs an AI meeting recorder on Zoom without telling the client it is on, and certainly without checking whether that violates anyone's recording-consent rules. The marketing assistant who runs a list of senior leads through an AI tool to "improve" the messaging before a campaign goes out. The new hire who built a personal automation that watches her inbox and drafts replies in her name to anything that looks like a renewal question.
None of those people are bad employees. Most of them are exactly the kind of resourceful person you want on the bench. The problem is not their character. The problem is that none of those moments was visible to you, and every one of them just put a piece of regulated client information somewhere your E&O carrier has never heard of.
Where the E&O exposure actually lives
Here is the part most agency leaders have not connected yet.
Your existing E&O coverage was written before any of this. It assumes a world where your team writes proposals in Word and sends emails from Outlook and stores client records in your AMS. It does not assume that a dec page has been pasted, by 9:14 on a Tuesday morning, into a service that retains the input and may use it to train a model.
There are four places the exposure actually lives.
First, data confidentiality. The moment client information enters a consumer AI tool, it has left your control. You no longer know where it is stored, how long it is retained, who has access to it, or whether it ends up reflected in someone else's output. If that client later alleges a breach of confidentiality, your defense depends on a chain of custody you can no longer prove.
Second, the output going to the client. When AI drafts a renewal email, a coverage summary, or a recommendation, and that draft goes out with your producer's name on it, your agency is on the hook for what it says. AI is fluent and confident, and it is sometimes confidently wrong. Hallucinated benefits, mischaracterized exclusions, misstated effective dates: these are the textbook ingredients of a professional negligence claim.
Third, your carrier appointment agreements. Most appointment papers include data-handling language and require that you protect non-public personal information consistent with applicable law. AI use that puts NPI into an unvetted tool is, at minimum, a question your carriers are now within their rights to ask at audit. Many carriers are already starting to ask.
Fourth, your E&O carrier itself. AI is now on the renewal questionnaire for a growing list of E&O markets, the same way cyber controls were five years ago. "Do you have a written AI use policy?" is the new "do you have multi-factor authentication on email?" If the answer is no, you have not been declined. You are just being priced as a higher risk.
The 247-day detection lag matters here. By the time you discover any of the four, it is rarely a question you can quietly fix. It is a question you are answering inside an actual claim.
The senior-market layer: CMS rules and Medicare client information
If you sit anywhere in the senior market, the exposure is sharper, because the rules are stricter and the eyes are sharper.
There is no AI-specific rule from CMS yet for agents and TPMOs. There does not need to be. The rules CMS already enforces apply to anything AI touches in your shop.
The Scope of Appointment rules apply whether the appointment was set by your producer or by an AI scheduling assistant working her inbox. The Third-Party Marketing Organization rules, including the call recording and multi-year retention requirements, apply whether the call was joined by a human note-taker or an AI transcription bot. The marketing material rules apply whether the email was hand-written or AI-drafted, and CMS does not care that an algorithm chose the wording. If the wording misrepresents a benefit or steers a beneficiary inappropriately, you carry that. The carrier's compliance team, when it lands on audit, is not going to soften the finding because the language came out of a free chatbot.
And the underlying data is regulated. Medicare Beneficiary Identifiers, Health Insurance Claim Numbers, dates of birth tied to a Medicare ID, plan and provider details. None of that belongs in a consumer AI tool, and the moment it goes there, you have created a record problem you cannot un-create.
You can see why the senior market is the room where this conversation gets serious first. The penalties have teeth, the audits are real, and the AI adoption is already happening upstream. A 2025 NAIC survey reported by Cherry Bekaert found 84% of health insurers already using AI in some capacity. Those are your carriers. They are watching how their distribution behaves with it.
The regulatory direction of travel
Pull back for a second, because the air pressure in the room is changing.
The National Association of Insurance Commissioners issued a Model Bulletin on insurer use of AI in late 2023, and as of October 2025 it had been adopted in 24 jurisdictions. NAIC then launched a multi-state pilot of its AI System Evaluation Tool through 2026 to test how insurers actually govern AI in practice. A handful of states have gone further with their own statutes. Colorado's SB 21-169 imposes algorithm bias-testing and risk-management requirements on life insurers using AI, Texas has the Texas Responsible AI Governance Act (TRAIGA) effective in 2026, and the New York Department of Financial Services has issued binding guidance through an Insurance Circular Letter on the use of AI systems.
Here is the careful, accurate way to read all of that for your agency.
The Model Bulletin applies to insurers, not to producer entities. Your agency is not the directly regulated party. But your carriers are. And carriers do not absorb regulatory pressure quietly. They push it down their distribution channels through appointment-agreement updates, through new questions in carrier audits, through compliance attestations, and eventually through declined renewals for partners who cannot answer cleanly.
The direction of travel is unambiguous. You can be in front of it by quietly having a written policy when the audit question arrives, or you can be behind it by scrambling when it does. Those are the two doors.
What an AI policy actually means for a 12-person agency
Now the contrarian reframe, because when you hear the words "AI governance policy" your stomach drops, and that reaction is the whole reason most agencies still have nothing in place.
You are not building what the carriers are building. The carriers are operationalizing a forty-page AI Systems Program with cross-functional governance committees, model documentation, bias-testing protocols, and board oversight. That is appropriate for an enterprise with thousands of employees and AI making coverage decisions. It is not appropriate, and it is not necessary, for an agency with twelve people whose AI is mostly summarizing emails and drafting first-pass proposals.
What you need is much smaller and much more useful. A short, written, plain-English Acceptable Use Policy that does five things, all of which fit on a single page.
It names which tools your team is allowed to use, by name, and which they are not. It names which categories of client information are absolutely off-limits to consumer AI accounts, in language your account managers actually use. It sets a simple disclosure rule for client-facing AI work. It names the person on your team who owns the policy and approves new tools. And it carries one sentence at the bottom that says how often the policy is reviewed.
That is the entire document. It is what your E&O carrier wants to see. It is what your appointment carriers will start asking for. And it is what turns Shadow AI into Citizen AI, the same activity moved into the light and made faster, not slower.
The full template, with the language you can paste into your own version, sits inside the Shadow AI Governance Launchpad. I will point you to it at the end.
How to actually start
There is an order of operations here, and almost everyone runs it backwards, the same way they did with the tools.
Start with amnesty. Tell your team, on the record and in the same conversation, that you are not in trouble-finding mode and that you genuinely need to know what they are actually using. The list will be longer than you expect. That is fine. You cannot govern what you cannot see, and you will not see it if your team has a reason to hide it.
Then draw the lines. Decide which tools are in, which are out, and which kinds of information do not go into either category under any circumstances. Be specific, not abstract. Your team can act on "no document containing an MBI, an HICN, or a date of birth tied to a client name goes into a personal AI account" in a way they cannot act on "protect non-public personal information."
Then write the policy. One page. Plain language. The Launchpad's template is built precisely for an agency this size, but if you would rather draft your own, the components are the five from the section above.
Then train once, and govern always. A single conversation with your team to walk through the policy. Then a lightweight rhythm to keep it alive: a named owner, a short review every quarter, and a five-minute new-hire briefing so this never has to be a "we need to talk about AI again" meeting.
Then document for renewal. When your E&O carrier asks the AI question on the next renewal questionnaire, your answer is yes, here is the policy, here is the date it was last reviewed, here is the person who owns it. That conversation lasts thirty seconds and ends well.
That is the whole method. Amnesty, lines, policy, training, documentation. Unglamorous, sufficient, and a long way ahead of where most of your competitors will be when their carriers start asking next year.
Frequently Asked questions
Yes, and the reason is less about regulation than about exposure. The NAIC Model Bulletin and most state AI laws apply to insurers directly, not to producer entities. But your appointment carriers will start requiring an answer, your E&O carrier will start asking on renewal, and the underlying laws around unfair trade practices and non-public personal information already apply to whatever your team does with AI. A short written policy gets you ahead of all three conversations.
That is an open question, and the honest answer is that it depends on your policy language, your carrier, and the facts of the claim. Newer policies are beginning to address AI explicitly. Older policies were not written with AI in mind, which means coverage is at minimum debatable. The single thing that helps in either case is having a written AI use policy on file, because it shows that the agency took reasonable steps to manage the risk.
They can, but not the consumer version with client information, and not without a policy that names exactly what is and is not allowed. The free, personal version of any consumer AI tool retains your inputs and is not the right place for non-public personal information. A team account on a business tier, used inside a written policy with clear rules about what does not go in, is a different conversation.
CMS has not issued AI-specific rules for agents and TPMOs. What it does enforce is the existing framework around marketing material, the Third-Party Marketing Organization rules, call recording and retention, and Scope of Appointment, regardless of whether the work was done by a human or by AI. If AI drafts a marketing email that misrepresents a benefit, that is still a marketing violation. If AI joins a sales call, the call still has to be recorded and retained. Existing CMS rules apply to AI-touched work without modification.
At minimum, five things: an approved tools list, a list of data types that never go into consumer AI accounts, a disclosure rule for client-facing AI use, a named owner, and a review cadence. The Shadow AI Governance Launchpad walks through each component with language an agency can adapt directly.
Where to go from here
Two next steps, depending on which problem feels most pressing right now.
If you can already feel where the exposure lives in your agency and you just want the policy in your hands, the Shadow AI Governance Launchpad is built exactly for this. It walks you through the components, gives you adaptable language, and is sized for an agency, not for an insurer.
If you'd like to see where your team's hours are actually going before you change anything, take the AI Opportunity Score. It takes about three minutes, shows you how many hours a week your team could get back, and gets you a starter crew member built for the biggest one.
And if you would rather do this with help, in your agency, with your team in the room, that is the work I do at agency workshops and at events like the WAP Agency Leadership Summit and the Kellogg Business Development Summit this spring.
