Ask a small CPA firm owner whether their team uses AI tools, and most will say "a little, informally." Ask whether they have a written AI policy, and the room usually goes quiet. That gap is bigger than it sounds. Recent industry data puts AI adoption in accounting and CPA firms at 73%, a 340% jump from 2022 levels — yet according to Karbon's State of AI in Accounting Report, only 37% of firms have invested in any formal AI training at all. Most of the growth happened without any governance layer underneath it.
That mismatch is exactly what the Journal of Accountancy's July 2026 piece on drafting an AI policy is trying to close. It's aimed broadly at CPA firms of every size, but the advice lands hardest on small and mid-size practices — the ones least likely to have a compliance department, and most likely to have a solo practitioner or two-person team quietly experimenting with ChatGPT between client calls.
The Real Risk Isn't the AI — It's the Silence
Here's the uncomfortable part: your firm doesn't need to have "adopted AI" to have an AI risk problem. If even one staff member has ever pasted a client's trial balance into a chatbot to get a second opinion on a journal entry, you already have exposure — whether or not there's a policy on file.
This is often called "shadow AI": tool use that happens outside any sanctioned list, outside any contract, and outside anyone's awareness. It's not a hypothetical. Recent security research found that roughly 34.8% of employee ChatGPT prompts now contain sensitive company data, up from just 11% in 2023. In accounting specifically, that means client revenue figures, cash flow statements, and tax documents ending up in a third-party system the firm never vetted, never contracted with, and can't audit.
The legal exposure isn't abstract either. When an employee inputs client financial data into a personal AI account, that data is transmitted to and stored by a third party outside any existing client agreement — which can meet the legal definition of a data breach and trigger notification obligations under state law. For a firm bound by the AICPA's Confidential Client Information Rule (Section 1.700.001 of the Code of Professional Conduct), that's not a minor technicality. It's the same rule that governs whether you can discuss a client's finances with anyone outside the engagement — and it applies to AI tools exactly the same way it applies to a chatty conversation at a networking event.
What a Real AI Policy Actually Covers
The Journal of Accountancy guidance breaks a usable policy into eight building blocks: scope and purpose, governance structure, data privacy, quality and accuracy controls, documentation protocols, bias assessment, client transparency, and acceptable-use rules. That's a lot for a five-person firm to tackle at once, so it's worth focusing on the three that do the most protective work.
1. Document the prompt, the output, and who checked it
The guidance is blunt about this: "If it isn't documented, it didn't happen." That's not a new standard for CPAs — it's the same logic that governs workpapers and engagement files — but it now needs to extend to AI. When a staff member uses an AI tool to draft a client memo, research a technical question, or summarize a document, the firm should be able to show:
- What prompt was used
- What the tool returned
- Who reviewed the output, and how
That record belongs in the client file alongside any other third-party tool documentation — not in a Slack thread that disappears after 90 days.
2. Require human review before anything reaches a client
This is the line that should be posted above every workstation: AI should not be used to make, finalize, or support decisions related to client services without thorough human review, consistent with the firm's existing supervision policies. In practice, that means AI-generated research, technical explanations, or citations need to be verified directly against primary sources — the Internal Revenue Code, Treasury regulations, FASB standards — not accepted because the answer sounded confident. Generative AI tools are well known for producing plausible-sounding citations that don't actually exist; a staff member who doesn't independently confirm a cited code section before it goes in a client letter has skipped the entire point of professional judgment.
3. Know exactly what happens to the data you type in
Before any tool touches client information, someone at the firm needs to answer four questions about it: Where is the data stored? Who can access it? Is it de-identified? Does it get used to train the model? If the answer to any of those is unclear — and for most consumer-grade AI tools, it will be — the policy should simply prohibit entering personal, confidential, or proprietary information into that tool. Firms that manage this well don't try to police every individual interaction; they maintain an approved-tool list and a lightweight process for staff to request additions, treating AI governance as a firm-wide decision rather than an individual one.
The Mistakes Firms Keep Making
A few patterns show up again and again in how small firms stumble into AI risk, and none of them require malicious intent — just the absence of a policy.
Treating "we haven't rolled out AI" as protection. A firm can have zero AI tools in its official tech stack and still have every staff member using a personal ChatGPT account on their phone during work hours. Policy gaps don't wait for official adoption; they exist the moment a staff member has internet access and a deadline.
Assuming free and paid tiers behave the same way. Many consumer AI tools use free-tier conversations to train their underlying models by default, while paid or enterprise tiers often include contractual data protections. A staff member who signs up for a free account to save time is frequently opting the firm's client data into a training pipeline without realizing it. This is exactly why the four data-handling questions above — storage, access, de-identification, training use — need answering per tool, not assumed once and forgotten.
Letting "it sounded right" substitute for verification. AI-generated tax citations, code section references, and technical summaries can be fluent and entirely wrong. A staff member under deadline pressure is the person most likely to skip the verification step — which is precisely why the policy needs to state the human-review requirement as a rule, not a suggestion left to individual judgment.
No record of what was asked or reviewed. Even firms that get the substance right often skip the documentation. If a client or regulator later asks how a particular figure or recommendation was produced, "we used AI to help" isn't an answer without a record of the prompt, the output, and who signed off.
A Starting Checklist for a Two-Page Policy
You don't need outside counsel to draft a first version. A workable policy for a small firm can fit on one or two pages and cover:
- An approved-tool list — name the specific AI products staff may use for client work, and a simple process (even just "ask the partner") for requesting new ones.
- A data-handling rule — no personal, confidential, or proprietary client information goes into any tool that isn't on the approved list.
- A human-review requirement — no AI output is used to finalize client work, deliverables, or advice without review by a qualified staff member, logged in the file.
- A citation-verification rule — any AI-generated reference to a code section, regulation, or standard must be checked against the primary source before it's cited to a client or used in a filing.
- A documentation habit — where staff log AI-assisted work (a field in the practice management system, a note in the engagement file — anything durable and searchable).
- A client-transparency note — a plain-language line in engagement letters describing that the firm may use AI tools as part of service delivery, subject to the safeguards above.
None of this requires a technology overhaul. It requires deciding, in writing, what the firm already assumes informally — and making sure every staff member is working from the same assumptions.
Small Firms Carry More of This Risk, Not Less
It's tempting to assume AI governance is a "big firm problem" — something for national practices with dedicated risk officers. The opposite is closer to true. Smaller firms typically lack a compliance function, rely more heavily on general-purpose consumer AI tools rather than vetted enterprise products, and have less redundancy if one staff member's shortcut becomes a liability event. A two-partner firm without a written AI policy is relying entirely on informal habits to protect client confidentiality — and informal habits don't hold up well when a client asks, during an engagement, exactly how their data was handled.
The fix doesn't require a legal team. A one-to-two-page policy that names approved tools, states the human-review requirement in plain language, and tells staff where to log AI-assisted work covers most of the exposure. The building blocks above are a reasonable starting checklist; what matters most is that something exists in writing before the next staff member reaches for a chatbot on a client matter.
Where Financial Record-Keeping Fits Into This
AI governance and financial record-keeping are solving the same underlying problem: making sure there's a clear, reviewable trail behind every number and decision. A firm that can't show how an AI-assisted answer was verified has the same gap as a business that can't show how a transaction was categorized — both come down to whether the trail exists and whether a human actually looked at it.
That's the same principle behind plain-text accounting. Beancount.io keeps your financial records as version-controlled, human-readable text rather than opaque database entries — every change has a visible history, and nothing happens in a black box. For a profession increasingly worried about explainability, that kind of transparency isn't a nice-to-have. Get started for free and see how a fully auditable ledger fits into a firm that takes documentation seriously.