A new dispatcher at a six-truck electrical contractor asks the company's new AI assistant whether a Saturday service call costs extra. The assistant answers quickly, cites its source and is wrong. Its source is the 2023 rate sheet. The owner raised the Saturday rate last spring and announced it in the crew group text, and nobody wrote it anywhere else.
Nothing was wrong with the AI. It did exactly what it was built to do: find the best written answer and quote it. The problem was the written answers. Most assistants that disappoint are built on a knowledge base nobody prepared.
Start with 20 to 40 documents, not the whole drive
The instinct is to connect everything. Resist it. A shared drive holds drafts, duplicates, old price lists and files nobody has opened in years, and an assistant can't tell which ones you still stand behind.
Instead, list the questions people actually ask, then find the document that answers each. Those documents are your starting set. Questions with no written answer go on a separate list; someone has to write those answers before any assistant can give them.
Gathering that list takes about two weeks. Three sources cover most of it:
- Your own inbox and texts. Search for messages that start with 'quick question' or 'where is' and note what was asked.
- A shared log. Ask everyone to jot down any question they ask a manager, or get asked, for ten working days.
- The people who answer most. The office manager, the lead preparer or the kitchen manager can usually name the top fifteen from memory.
| Business | Questions people ask | Documents to start with |
|---|---|---|
| Accounting firm | What's our extension policy? Which clients get the bookkeeping checklist? What do we charge when documents arrive late? | Engagement letter templates, busy-season procedures, fee schedule, document checklists |
| Electrical contractor | What's the Saturday rate? Who handles permits in which town? What should be on every truck? | Current rate sheet, permit contacts by jurisdiction, truck stock list, job close-out checklist |
| Restaurant | Does the romesco contain tree nuts? What's the private room minimum? How do we handle a comped meal? | Current recipe and allergen sheets, events packet, comp and void policy, opening and closing checklists |
Allergen questions deserve extra care. Federal law recognized sesame as the ninth major food allergen effective January 1, 2023, a reminder that these lists and your recipes both change. Tie each allergen sheet to the current recipe, and keep the rule that a manager confirms before anyone answers a guest's allergy question.
Just as important is what to leave out at first: drafts, whole email archives, meeting notes, anything holding personal or client data, and any document you wouldn't want quoted to a new hire as the official answer. Each of those can be added later, deliberately, once the core set works.
Where the knowledge base lives matters less than people expect. A dedicated folder in the drive or SharePoint site your staff already use is fine, and so is a simple wiki, as long as it holds only current documents and its sharing is tighter than the rest of the drive. The assistant connects to that one location, not to everything.
One topic, one owner, one review date
Every document in the knowledge base should carry four things at the top. They take a minute to add and prevent hours of wrong answers.
- A plain title that matches how people ask. 'Saturday and after-hours service rates' beats 'Pricing_v4_FINAL.'
- An owner. One named person who decides what it says.
- An effective date and a review date. For example: effective October 1, 2026; review by April 1, 2027.
- A status. Current or archived. Archived documents leave the assistant's search entirely.
Keep each document to one topic. A 40-page operations manual that mixes pricing, safety, vehicles and holidays is hard for search to slice cleanly, and a passage about holiday hours can end up answering a pricing question. Split it into short documents, put the answer in the first few lines and use headings that read like the questions people ask.
A few format habits help retrieval too: real text instead of scanned images, tables saved as tables instead of screenshots, and the full term next to internal shorthand, such as 'ATS (automatic transfer switch).'
Resolve contradictions before launch
Two documents that disagree are the most common cause of wrong answers. The assistant may quote either one, and both look official.
Flowchart
What happens when the assistant finds two answers
- StartWeekly review spots two rate sheets with different Saturday rates
- StepBoth are flagged, and the assistant routes Saturday-rate questions to the office for now
- DecisionIs it clear which sheet is current?NoA personOwner decides the rate and writes it into the rate sheetYes
- StepCurrent sheet gets an effective date; the other is archived
- StepThe original question is asked again to confirm the new answer
- DoneDispatch gets the right rate, quoted from the current sheet
Before launch, run a contradiction sweep: search your starting documents for the topics that change most (prices, deadlines, anything with a date) and make sure each has exactly one current answer. For an accounting firm, that usually means last year's engagement letter template and this year's sitting side by side.
Before and after
A shared drive vs a knowledge base an assistant can trust
The shared drive
- 01Three rate sheets, two of them marked 'final'
- 02Policy changes announced by text and email
- 03Nobody sure who owns the procedures manual
- 04Old 'anyone with the link' sharing still active
- 05Assistant quotes whichever file matches best
ResultConfident answers from the wrong file
The knowledge base
- 01One current rate sheet; older ones archived
- 02Policy changes written into the document that owns them
- 03Every document lists an owner and a review date
- 04Permissions checked before anything is connected
- 05Unanswered questions turned into document updates
ResultOne answer per question, with a source
Audit sharing permissions first
A well-built assistant searches only what the person asking is allowed to see. Microsoft's Copilot documentation, for example, says it only surfaces organizational data that the individual user can at least view. The catch is that most businesses have years of loose sharing, and if the payroll file is open to everyone, a good assistant will find it for everyone.
- 01List the folders you plan to connect.
- 02For each, check who has access, including 'anyone with the link' and outside collaborators.
- 03Move sensitive files (payroll, HR, client tax documents, personal data) into restricted folders, or leave them out.
- 04Decide which groups get the assistant: office staff, field crews, kitchen managers.
- 05Repeat the check whenever a new folder is connected.
At an accounting firm, client files usually stay out of a general staff assistant altogether. Client-specific questions belong in a system tied to each engagement and its access rules, which also helps keep return information within the limits Section 7216 places on tax preparers. That's general background rather than legal advice; confirm the specifics with your own advisers, since state rules can add requirements.
Keep it current without a big project
Knowledge bases decay quietly. The routine that keeps one healthy is small:
- Weekly, about 20 minutes: read the questions the assistant couldn't answer. Each becomes a new document, an edit, or a note that it's a question for a person.
- Monthly: check documents past their review date; owners confirm or update them.
- Whenever something changes: the person who changes a price, policy or procedure updates the document the same day. A group text is not a policy.
- Quarterly: re-run your test questions and repeat the permissions check.
If you're still deciding whether an assistant is worth it, our explainer on how assistants use your documents without being trained on them covers the mechanics. We also build the knowledge base and the assistant together; our chatbots and assistants page explains the process.