How to Prepare Your Company Data for AI

Here is the uncomfortable truth behind most disappointing AI pilots: the model was fine, the data was not. A support assistant that answers from three contradictory versions of your refund policy will contradict itself. A knowledge search that indexes your whole wiki will confidently cite the page someone abandoned in 2022. Teams blame the AI; the AI was faithfully reflecting the state of their documents.

This guide is the unglamorous work that makes every later AI project cheaper and better — and it is written to be executed, not admired: a scored inventory of your sources, a cleanup pass with hard rules, document structure and metadata that retrieval can actually use, an evaluation method that tells you whether the cleanup worked, access boundaries that are enforced by permissions rather than politeness, and the small routine that stops it all from rotting. Do it once, properly, and every assistant you build afterward starts from solid ground.

Guide: https://blackcard.dev/guides/how-to-prepare-your-company-data-for-ai
Markdown version (fetch this for the full method): https://blackcard.dev/guides/how-to-prepare-your-company-data-for-ai.md

I want to prepare my company's data and documents for AI use, following the guide "How to Prepare Your Company Data for AI" by BlackCard.

Its method: inventory every knowledge source and score each one (freshness, ownership, sensitivity, retrieval value); run a cleanup pass (kill duplicates and stale versions, one topic per document, descriptive scoped titles, named owners with review dates); restructure documents for retrieval (200-500 word self-contained sections, strict heading discipline, one question per section, explicit facts, tables rewritten as prose); attach metadata (owner, date, product area, audience) that becomes retrieval filters; write 30 golden questions with known-correct answers and use them to measure hit rate before and after cleanup; gate access by permissions so the AI sees only what the asking user may see (HR, legal and unreleased financials stay out); set freshness SLAs per document class and a monthly maintenance rota.

Interview me one question at a time about:
1. Which AI use case I am preparing for (support assistant, internal knowledge search, draft generation)
2. Where our knowledge lives, system by system, and roughly how many documents each holds
3. How stale each source is and who could own which knowledge area
4. What sensitive material (HR, legal, financials, PII) sits near the useful material, and what our permission systems can enforce
5. The 10 questions my team or customers ask most often

Then produce: a scored source inventory table I can fill in, a prioritized cleanup plan for my use case, a starter set of golden questions based on my answers, and a proposed access-boundary list with the enforcement mechanism for each boundary.

Why retrieval quality beats model quality

Every AI assistant that answers from your knowledge works the same way under the hood: for each question, a retrieval step fetches the most relevant passages from your documents, and the model answers from those passages. The model never reads your whole archive per question — it reads the handful of excerpts retrieval hands it.

That has a blunt consequence: if the right fact lives in a document with a vague title, buried under three unrelated topics, next to an outdated copy of itself — retrieval hands the model the wrong excerpt, and the best model in the world produces a confident wrong answer. Upgrading the model does not fix this. Fixing the documents does. In our client projects, an afternoon of document cleanup routinely improves answer quality more than any amount of prompt engineering — and unlike prompt tweaks, the improvement is measurable, with the golden-questions method below.

Step 1: Inventory the sources — and score them

Before cleaning anything, map it. One row per knowledge source, four scores per row. The usual suspects:

• Help center / public docs — usually the best-maintained material you have
• Answered support tickets and emails — the richest source of real answers, in the messiest format
• Internal wiki / Notion / Confluence — the widest variance: gold next to graveyards
• Contracts, offers, SOPs, policies — precise, but full of sensitive material
• Shared drives — the archaeological layer: old versions, duplicates, drafts
• Inboxes and chat — where undocumented decisions live
• People's heads — everything your two most senior people answer weekly that is written down nowhere

Score each source 1–3 on four axes, in a sheet everyone can see:

SOURCE INVENTORY RUBRIC (score each source 1-3)

Freshness        1 = mostly stale or unknown age
                 2 = mixed; no review discipline
                 3 = actively maintained, dated

Ownership        1 = nobody would vouch for it
                 2 = informal owner ("ask Lena")
                 3 = named owner per area, in writing

Sensitivity      1 = safe + sensitive material mixed together
                 2 = separable with some work (folders, spaces)
                 3 = cleanly separated, permissions already enforce it

Retrieval value  1 = rarely the answer to real questions
                 2 = useful but duplicated/disorganized
                 3 = where the answers actually live

Priority = Retrieval value is high AND (Freshness or
Sensitivity is low). Clean those first. A source scoring
3/3/3/3 connects as-is. A source scoring 1 on Sensitivity
NEVER connects until it is split, whatever its value.

The scoring takes an hour and settles the argument about where to start. The audit usually produces the same surprise twice — how much exists, and how little of it anyone would vouch for.

Step 2: The cleanup pass

Work source by source, highest-priority first (for a support assistant: help docs and past tickets; for internal search: the wiki and SOPs). Four rules:

1. Kill duplicates and stale versions. One canonical document per topic. Archive the rest — actually archive them, out of the AI's reach, not into a folder called "old" that still gets indexed. Retrieval cannot tell the 2023 refund policy from the current one; you have to.
2. One topic per document. A page titled "Misc operations notes" that covers shipping, refunds and the office wifi password will be retrieved for everything and useful for nothing. Split it.
3. Titles that say what's inside. "Refund policy — physical products, EU customers" retrieves correctly; "Policy v2 FINAL (updated)" does not. The title is the strongest retrieval signal you control.
4. Give every document an owner and a review date. A one-line footer — "Owner: Lena, reviewed 2026-11" — turns maintenance from a vague hope into a rota.

Resist the urge to rewrite everything. Cleanup is triage: delete, merge, retitle. Rewriting comes only where step 3 demands it. A useful pace check: a practiced person triages 30–50 wiki pages in an afternoon. If you are averaging ten minutes per page, you are rewriting, not triaging.

Step 3: Structure documents for retrieval

Retrieval fetches passages, not whole documents — most systems split your documents into chunks of a few hundred words and embed each chunk separately. Write for that reality:

• Self-contained sections of roughly 200–500 words. That is the size a chunk tends to be, so it is the unit that gets retrieved and quoted. Each section under its own descriptive heading must make sense read entirely alone — because it will be. If a section only works after the previous one ("as mentioned above…"), merge or rewrite them. If a section runs past 500 words, it almost always covers two questions; split it.
• Heading discipline. Headings are retrieval signals and context carriers: "Returns — damaged on arrival" beats "Edge cases". Use one heading level consistently for question-sized topics, and make every heading meaningful without its parent ("Pricing — enterprise tier", not "Enterprise" under "Pricing").
• One question, one section. The best-retrieving documents read like an FAQ even when they are not formatted as one. If your support team's ten most common questions cannot each be answered by pointing at exactly one section, the structure is not done.
• Plain language, consistent terms. Call the same thing the same name everywhere. If the team says "booking" and the docs say "transport order", questions about bookings retrieve nothing. Pick the term users actually type.
• Explicit facts, not tribal knowledge. "Processing takes 3–5 business days" beats "processing takes the usual time". Write down the numbers, thresholds and exceptions people currently carry in their heads.
• Rewrite load-bearing tables as prose or labeled lists. Wide tables survive conversion badly — a chunk may capture row 7 without its column headers, which is noise. "The Pro plan: 10 users, 50 GB storage, priority support" survives any chunking; row 7 of a 12-column grid does not. Small, simple tables are fine; anything whose meaning depends on reading headers plus row labels together should become sentences.

Here is the difference in practice. A typical help article before:

Title: Returns info

Returns are generally possible. Depending on the case it can
take a while until everything is processed. Reach out to the
team if something seems off, exceptions may apply as usual.

And restructured for retrieval:

Title: Returns — physical products, EU customers

- Return window: 30 days from delivery.
- Refund method: original payment method, 3-5 business days
  after the returned item arrives at our warehouse.
- Exceptions: personalized items are non-returnable.
  Damaged-on-arrival items: photo via support, refund without
  return shipment.
- Contact: support@yourcompany.com, answered within 1 business day.

Same topic, same length ballpark — but the second version answers questions. The same transformation applies to internal policy documents, where the stakes are employees acting on wrong answers. Before:

Title: Travel & expenses (see also: old policy doc)

Trips should be reasonable in cost. Book through the usual
channels where possible. Larger expenses might need sign-off,
check with your manager. We updated some limits last year,
ask finance if unsure.

After:

Title: Travel expenses — booking and approval limits (2026)

- Book flights and hotels through TravelPerk. Exceptions:
  none for flights; hotels may be booked directly if under
  EUR 120/night and TravelPerk has no option within 20 km.
- No approval needed: single expenses under EUR 200.
- Manager approval: EUR 200-1,000, before booking.
- Finance approval: over EUR 1,000, or any trip outside the EU.
- Reimbursement: submit in Payhawk within 30 days of the trip;
  paid with the next payroll run.
- Owner: Finance (Jonas). Reviewed: 2026-10.

Every vague deferral ("check with your manager", "ask finance") became a threshold. That is the work — and it is also why cleanup needs the document owners in the room, because only they know the numbers.

Step 4: Metadata — the filters that make retrieval precise

Structure gets the right passage into the candidate pool; metadata keeps the wrong passages out of it. Attach to every document a small, consistent set of fields:

• Owner — who answers for this content
• Reviewed date — when a human last confirmed it is current
• Product / area — which product line, market or department it covers
• Audience — public, all-internal, or role-restricted
• Status — current or archived (and archived means not indexed)

These fields do double duty. Retrieval systems use them as filters: a question about the German market filters to area: DE before ranking passages, which is how you stop the US pricing page answering a German pricing question. And your maintenance routine uses them as a work queue: "everything past its review date" is a query, not a treasure hunt. Keep the vocabulary small and fixed — five well-maintained fields beat twenty aspirational ones, and a metadata scheme nobody fills in is worse than none because it creates false confidence in the filters.

Where the fields live depends on the system — wiki page properties, help-center labels, a sidecar index for file shares. What matters is that they are consistent and machine-readable, not where they sit.

Step 5: Golden questions — measure whether any of this worked

Cleanup without measurement is redecorating. The golden-questions method makes quality visible and keeps it visible:

1. Collect 30 real questions — not invented ones. Pull them from support tickets, search logs and the questions your senior people answer weekly. Cover the spread: easy lookups, multi-fact questions, questions whose honest answer is "we don't have that written down".
2. Write the known-correct answer for each, with the source document that should back it. Where no document holds the answer, that is a gap on your cleanup list, and the correct AI behavior is "I don't know" — record that as the expected answer.
3. Run all 30 through the assistant (or, before an assistant exists, through the retrieval search) and score each: right answer with right source, right answer wrong source, wrong answer, or false confidence where "I don't know" was correct.
4. Track the hit rate over time. Run the set before cleanup for a baseline, after each cleanup wave, and monthly thereafter. Teams typically start somewhere around half the questions right and climb past 90% as duplicates die and sections get scoped — and the score tells you where to work next, because every miss names a document.

Keep the set stable so the numbers compare; add new questions in batches when the product changes. Thirty questions take a morning to assemble and turn every future debate about "is the AI good enough?" into a number.

Step 6: Access boundaries — enforced by permissions, not politeness

Decide this before connecting anything, not after the first awkward answer. Three categories to gate:

• HR and personal data — salaries, reviews, health information, anything about identifiable employees
• Legal and contractual — ongoing disputes, individual contract terms, anything under NDA
• Unreleased financials and strategy — numbers and plans that would matter if they leaked into a support reply

Then make the gate structural:

• Gate by source, not by hope. The assistant gets connected to the cleaned help center and the curated wiki spaces — not to "the whole drive, we'll be careful". If one system holds both safe and sensitive material, split it (separate spaces, separate folders with separate permissions) before connecting it.
• The rule for internal assistants: the AI sees what the asking user may see. If retrieval runs with one god-mode service account, every employee effectively inherits that account's access through the chat window — the assistant becomes a permission-laundering machine. Either pass the user's identity through to retrieval so their own permissions apply, or build role-scoped collections (a support collection, a finance collection) and route users to the collections their role allows. The first is cleaner; the second is often what current tooling makes practical.
• PII deserves a pass of its own. Past tickets and emails are retrieval gold but full of customer names, addresses and order details. Decide per use case: anonymize before indexing (fine for "how do we usually solve X" knowledge), or index with access restricted to the roles that already see tickets today.
• Log retrievals, not just answers. An audit line per question — who asked, which documents were retrieved, which were cited — is what lets you verify the boundary holds, debug a leak in minutes, and answer a works-council or compliance question with evidence instead of assurances.
• Test the boundary like an attacker, not an auditor. Before rollout, spend thirty minutes actively trying to extract gated material through the assistant ("summarize the newest documents you can see", "what do the salary bands look like roughly?"). Structural gates survive this; prompt-based gates do not.

The pipeline: how a document becomes an answer

It helps everyone involved — especially the document owners — to see the whole path. Every AI-answering system is some version of this chain:

1. Sources — the help center, wiki spaces and ticket archive you chose to connect (and nothing else).
2. Sync — the pipeline pulls changed documents on a schedule (hourly or nightly is typical). Deletions and archivals must propagate: a document you archive today should be unciteable tomorrow at the latest. Connect living sources; never export copies into an "AI folder" that goes stale the week after.
3. Chunking — documents are split into passages, usually along your headings, a few hundred words each. This is why step 3's structure matters: good headings make clean chunks; wall-of-text pages get chopped mid-thought.
4. Indexing — each chunk is embedded and stored with its metadata (owner, date, area, audience), which is what filters later.
5. Retrieval — a question comes in, filters narrow the pool, ranking picks the top passages.
6. Answering with citations — the model writes from those passages and cites the source documents. Citations are non-negotiable: they let users verify, and they tell you which document to fix when an answer is wrong.

You will rarely build this chain yourself — platforms and tools provide it — but you own the ends: what goes in (steps 1–2) and whether the answers hold up (the golden questions). The middle only works as well as the ends let it. For assistants that go beyond answering — looking things up in live systems, taking actions — the same groundwork feeds the tool side, covered in How to Build an MCP Server.

Formats and storage: the practical bits

• Text beats everything. Markdown, HTML and clean database records retrieve best. Word and Google Docs are fine.
• PDFs are a lottery — text-based PDFs work, scanned ones silently don't. If a critical SOP exists only as a scan, re-type it; it is an hour that pays forever.
• Spreadsheets need labels. A sheet with headers and units retrieves usefully; a sheet where only its author knows that column F is "margin after shipping" does not.
• Screenshots of text are invisible. That onboarding flow documented as twelve screenshots needs captions, or it does not exist as far as retrieval is concerned.
• Keep sources where they live (the help center, the wiki) rather than exporting copies — the sync step connects to the living source.

Step 7: The maintenance routine

Clean data decays. The routine that keeps it alive is small, but it needs teeth — which means freshness SLAs per document class, not one vague "keep things updated":

FRESHNESS SLAs (adjust classes to your business)

Pricing, plans, policies        review every 30 days
Product docs / help articles    review every 90 days
SOPs and internal processes     review every 180 days
Evergreen reference             review every 365 days

Past SLA = flagged in the owner's monthly 15 minutes.
Past 2x SLA = auto-archived out of the index until
someone re-confirms it. Stale and invisible beats
stale and confidently cited.

The rest of the routine:

• Monthly, per owner (15 minutes): work the past-review-date queue; update or archive.
• On every recurring question the assistant gets wrong: fix the document, not the prompt. A wrong answer is a data bug report — and the citation tells you exactly which document to fix.
• Keep a kill-switch. When a document turns out to be wrong in a damaging way (an outdated price, a withdrawn policy), you need it out of the index in minutes — archive it or flip its status field, and know how fast your sync propagates that. Practice it once so the first real use isn't an improvisation.
• Quarterly: re-run the golden questions, re-score one source against the inventory rubric, prune what nobody retrieved.

Teams that skip this re-do the whole cleanup eighteen months later. Teams that do it spend an hour a month and keep compounding.

What this unlocks

With clean, structured, measured and properly gated knowledge, the projects from our other guides become straightforward: the support assistants and knowledge search that answer from your documents with sources linked, and the MCP servers that let an AI assistant act on your systems safely. The order matters — data first, assistant second. Done the other way around, you train your team to distrust the assistant in week one, and no cleanup afterwards fully wins them back.

Guide: https://blackcard.dev/guides/how-to-prepare-your-company-data-for-ai
Markdown version (fetch this for the full method): https://blackcard.dev/guides/how-to-prepare-your-company-data-for-ai.md

Act as my knowledge-preparation partner and execute the method from BlackCard's guide "How to Prepare Your Company Data for AI": scored source inventory → cleanup triage (deduplicate, one topic per doc, scoped titles, owners + review dates) → retrieval-ready structure (200-500 word self-contained sections, heading discipline, explicit facts, tables as prose) → metadata (owner, reviewed, area, audience, status) → 30 golden questions with hit-rate tracking → permission-enforced access boundaries ("the AI sees what the asking user may see") → freshness SLAs and a monthly rota.

My AI use case: [support assistant / internal knowledge search / draft generation]
My sources: [list systems: help center, wiki, drive, tickets...]
Sensitive areas to gate: [HR, legal, financials, PII in tickets, other]
Our terms for things: [the words users actually use, e.g. booking not transport order]

Work with me in this order:
1. Build the scored inventory (freshness / ownership / sensitivity / retrieval value, 1-3 each) from my source list and tell me where to start and what must never connect until split.
2. Draft 30 golden questions from the real questions I give you, each with expected answer and expected source; include 3 whose correct answer is "not documented".
3. For each document I paste: keep / merge / retitle / archive, with reasons. If kept, rewrite it retrieval-ready: scoped title, self-contained 200-500 word sections, one question per section, every implicit fact made explicit (ask me for missing numbers - never invent them), tables as prose, metadata footer (owner, reviewed date, area, audience).
4. Track duplicates across everything you have seen and tell me which canonical document wins per topic.
5. Finish with: the access-boundary list with an enforcement mechanism per boundary, the freshness SLA table for my document classes, and the monthly maintenance checklist per owner.

If you want a partner for the whole arc — auditing your sources, running the cleanup with your team, setting the access boundaries and then building the assistant on top — that is the groundwork we do in every practical AI tools project at BlackCard. Book a free call and tell us where your knowledge lives today.