Experiment 1 handoff

For Reid. Drafted by Claude from the Oct 9 and Oct 13 calls. Draft: the items marked Open get settled in tomorrow's meeting, before anything runs.

What we're testing, in one line

Can a scanned document go from the scanner into our records, with every fact traced to its page and the document matched to the right deal, with no person touching it unless it needs a decision?

Ground rules (from the meetings)

• No "today" baseline. This first version's score is the baseline. Later versions are compared to it.
• Everything is a record. Files stay in SharePoint; the database row is the truth.
• Code first, AI only at judgment calls. Every action is a tool, and one tool's decision triggers the next.
• Nothing leaves the building in this experiment. No operator emails, no HubSpot or inventory changes. Results go to a run folder only.
• The transfer plan is a list of problems. Its solutions are suggestions.
• No new software. The scanners' free software plus the existing 707-deal-data flow. No PaperStream Pro.

Scope: which flows

The confirmed first study is 0 Inbox → 1 Records → 4 Find contact → 3 Operator notice → 7 Read replies → 8 Filing, plus 10 Needs-me list. Experiment 1 covers only the first part: 0 Inbox and 1 Records, plus matching each document to its deal. Flows 4, 3, 7 and 8 come in later experiments that build on this one.

What already exists (707-deal-data)

• Reads PDFs, Word and Excel; falls back to OCR (Docling + Tesseract) for scans; masks bank account digits.
• Tags people, tracts, leases, wells and numbers; builds a graph with every mention traced to its file and position.
• Links tracts (county, survey, section, block, abstract), ties wells to leases, and applies the deal rules.
• Grades output against frozen answer keys. On the 45 held-out deals it gets 85.1% of fields right.
• Limit: it runs on a whole deal folder at once, not on single documents arriving one by one.

Steps

1. Pick 10 documents from closed deals that already have answer keys in expected-payloads/. Mix them: some clean, some messy scans, at least two document types, and at least two deals. Open: which 10.
2. Write the answer key for each document (a small table is fine): document type, which deal it belongs to, and the 3–6 facts that matter most, each with its page.
3. Set up the inbox: one SharePoint folder for scans. Set the free scanner software to save one PDF per document there. Check whether it can split a stack using blank pages; if not, scan one document at a time.
4. Scan the 10 printed documents into the inbox. (Scanning real paper tests the scanner and OCR path, not just files already on disk.)
5. For each document: make a record (type, source, received time), run the reader and tagger, and add what it finds to the graph with page and quote. Open: how to run the flow on one document instead of a whole folder.
6. Match each document to its deal using the graph (tracts, leases, names). If it isn't confident, send it to a "needs a person" list instead of guessing.
7. Score it against the answer key (table below). Write a short retro: what was easy, what broke, and what surprised you.

How it's scored

What to bring back

• The scores table for the 10 documents.
• Every miss, with why it happened: OCR, tagging, linking, or matching. That tells us what to fix first.
• The retro.

Open, for tomorrow's meeting

1. How the flow takes one document at a time. Options: wrap each document as a one-file "deal", add a single-document entry point, or match the document to its deal first and re-run that deal.
2. Where records live during the experiment: a new table in the existing Postgres, or files in the run folder.
3. Which 10 documents.
4. Whether entity recognition runs before the document is matched to a deal (Joe's knowledge-graph idea) or after.
5. Whether the free scanner software can split a stack.