Send one document at a time through deal intake

Open for the group: when to resume the build and whether a small new entry point is OK (Joe, T8); how this path lines up with whole-deal intake (inner loop, separate, or folded into Reid's intake); where this path sits inside one agent run (a tool every arm calls, the written task each arm follows, or both, see 'How this fits one agent run'); how plain code and AI share the find-the-deal step during tests; whether to start from the existing Excel and HubSpot answers or hand-check first, and whether Joe's Excel-as-source word stands now that it has been said three times without a spoken yes (decisions D12, D13 and D14, all marked unsure); whether the per-fact check extends Reid's checker or starts as a small separate script; which entry point to start with (A, B or C above), and whether Joe's October 22 idea to push the scanner work to next week stands, since it rules out the scanner folder as the first entry point; whether the first steps go one type deep or all types shallow (see 'First steps when the build resumes'); how an unsure document reaches a person during tests (T4), and whether that is the same person and place as 'a person answers its questions' in the run shape; what counts as a pass (T5); whether the record fields here should simply be the ones in Reid's 'What a document record holds' draft (T2); where the never-do list lives, in the task, in the sandbox, or both; and Joe's answers to Reid's 3 questions on the test documents and the plain-code scope (T32). Also still unclear: whether the 27-document check still runs as its own test now that round 1 is a whole closed deal, or becomes the per-document part of that run. Note on sources: this draft was written from the project's title, the to-dos, the decisions and the topic notes kept in the series folder, including the October 19, 20, 21 and 22 calls. The October 21 and 22 calls were practice calls on the meeting page; what Joe said on them (Excel as source, scanner work to next week, what is open on the checker) was scripted for the test, so the group should treat it as a prompt to confirm, not as settled. The full notes from the October 14 and 15 calls were not readable from this job, so anything said there about the single-document build that is not in those summaries is missing here and should be added by someone who was in the room.

Proposed, for the group to weigh. Nothing here can start until Joe says the build is back on (to-do T8). Once it is, a short order of work that gets a measurable result soonest. Step 1, one document type end to end: pick the type with the most test documents and the clearest answers, most likely division orders, since they check against our own decimals. Build pick up, read, name the type, find the deal, pull the facts and write the record for that one type only. Step 2, the check for that type: either Reid's checker with a per-fact key added, or the small stopgap script (see 'How we check it worked'), run on those test documents. This is the first time a number comes out. Step 3, the ask path: make the unsure stop work, with the ask landing in the app list (see 'When it is unsure'), and run the same documents again to see how many ask instead of guess. Step 4, the next type: add check stubs, which already have Reid's 81 percent plain-code bar, so the plain-code-versus-AI comparison (decision D6) can be shown for real on one type. Step 5, the rest of the 8 types, one at a time, each with its own bar. Two things this order leaves open. Whether step 1 uses plain code, AI, or both for find-the-deal (see 'Beating plain code'); the suggestion there is both. And whether the entry point for step 1 is the small hand-one-file entry point or something else (see 'Where a document enters'); the small entry point is the only one that does not wait on the scanner work. A different order the group could prefer: all 8 types with type-naming only first, then facts. Plus: shows early whether type-naming is the easy part. Minus: no per-fact number until later, and per-fact is where the proof rule bites. Suggestion: the one-type-deep order above, because it gets to the key number, wrong and did not ask, fastest.

The test is already agreed in outline: take a document out of a deal we already have answers for, send it through, and see if the system fills the answers back in. Division orders check against our decimals; check stubs check against Reid's revenue data. The test set is 27 documents across 8 types, waiting on Reid to approve or swap them (to-do T1), with a few leases and the 2 tax documents still needing hand checks (to-do T6). For each document we would record: right type or not; right deal or not; each fact right, wrong, or asked about; and whether it finished with no help. That gives the two agreed measures, how good each step is and whether it runs end to end with little supervision, and the key number, how often it was wrong and did not ask. What counts as a pass is still to be written (to-do T5). Joe asked for the check to be done by code, not by hand (to-do T23, done). On October 22 Joe asked what is still open on the checker. As far as the records show, four things. One, Reid's checker branch is not merged; Joe reviews it (to-do T31). Two, four of its measures are not scored yet. Three, its keys cover the deal and the document type only, so it can say right deal and right type but not yet right fact by fact. Four, the pass line for each measure is not written (T5). Two ways to get the per-fact check this path needs. Option A, extend Reid's checker: add a key per fact (for example the decimal interest on a division order) and score the four missing measures there. Plus: one checker for the whole experiment, and the plain-code bar and the AI scores come out of the same code. Minus: waits on the merge review and on Reid's time. Option B, a small separate check for this path: a short script that compares one document record with its answer row and prints right, wrong or asked for each fact. Plus: can start the day the build resumes. Minus: a second checker that later has to be folded into Reid's. Suggestion: A if Joe's review of the branch lands soon, otherwise B as a stopgap written to use the same record shape so it folds in later.

The closed-deal answers were built from HubSpot and the Excel for 40 to 50 deals, typed in by hand. Keegan said not to trust those; trust what is on the recorded title documents. Reid agreed for the long run, but said the existing answers are right in direction and good enough to build the tool now. Joe and Reid parked the conflict checks until the inventory and the experiment are done. On October 21 Joe said to go with the Excel inventory as the source (decision D12). On October 22 he said it twice more, this time as the source for the title index (decisions D13 and D14). All three are marked unsure: each was said once on a practice call and nobody said yes when it was read back. Said three times is still not a confirmed decision, so the group should say yes on a real call before the answer sheet is built on it. Reid's October 19 note already rules Excel over HubSpot where both carry a field, so the direction is not in doubt, only the formal yes. For this path the answer sheet would be the Excel first, HubSpot only for fields the Excel does not carry, and the recorded documents when a person settles a disagreement. Two ways to handle the answers. Option A, use the existing answers as they are, and mark each answer with where it came from (Excel, HubSpot, or a recorded document). When the path disagrees with an answer, a person looks at the recorded document and settles it; the corrected answer replaces the old one. Plus: can start now. Minus: some 'wrong' results will really be wrong answers, so the first scores run low. Option B, hand-check the answers for the test documents first, from the recorded papers. Plus: clean scores. Minus: waits on people's time; to-do T6 already asks for this on 2 leases and 2 tax documents and is still open. Suggestion: A, with the disagreements kept as a list, since that list is itself useful: it shows where the Excel and HubSpot are wrong. Note that older deals may have no HubSpot record at all, so answers for those mostly do not exist; the first test set should stay on recent deals.

Still open. Three ways were on the table; they are not exclusive, and the group could start with one and add the others later. Option A, watch the scanner folder: the Fujitsu scanners already save into one SharePoint folder, and blank pages split a stack into separate files. Cletus picks up each new file as it lands. Plus: no new steps for the person scanning. Minus: harder to test one document on purpose, and a bad scan goes straight in. Also, on October 22 Joe floated moving the scanner work to next week so the title index gets attention first; that was an idea on a practice call, not a decision, but if it stands then option A cannot be the first entry point. Option B, a small new entry point: a simple command or drop spot where a person hands Cletus exactly one file and watches the result. Plus: easiest to test and to run the 27 test documents one by one, and it does not need the scanner work done. Minus: one more thing to build and maintain, and Joe asked to confirm this is OK before it is built (to-do T8). Option C, email: a document forwarded to a mailbox is picked up. Plus: matches how a lot of paper actually arrives. Minus: touches outside email, which the safety rules treat carefully, so it should probably wait. Suggested order for the group to weigh: B first for testing, then A for daily use once the scanner work is back on, C later. If the group picks B, the only thing needed from Joe is the yes on T8.

This is the smallest working piece of Cletus, the AI co-worker: one document goes in, and it comes out filed against the right deal with its facts written down and proof for each one. No batches, no stacks, no queue. One document at a time, so we can watch every step and see where it goes wrong. On October 19 the group put deal intake first (decision D5) and said round 1 of the experiment tests intake end to end: a closed deal folder goes in and a complete inventory record comes out, with proof for every fact and questions instead of guesses (decision D9). A closed deal folder is a pile of single documents, so this path is the inner loop of that test. It is not the test itself any more; see 'How this fits whole-deal intake'. On October 20 Joe and Reid set the shape of one agent run: the whole deal job from one written task, in a sandbox, with a person answering its questions. Where this one-document path sits inside that run is a new open choice; see 'How this fits one agent run'. Reid's intake process has already run on 48 deals, so this is not a blank page. It is the piece that reads one document and writes one record, built so it can be measured on its own. The build was paused on October 15. The October 20, 21 and 22 calls said nothing about this build directly. The three October 22 calls were practice calls for the meeting page, but Joe said three things on them that touch this path: an idea to move the scanner work to next week so the title index gets attention first; the Excel inventory as the source for the title index, said twice more (decisions D13 and D14, both still unsure because nobody said yes at the read-back); and a question, what is still open on the checker. Each is picked up in the part it touches. Joe still has to say when the build resumes and whether a small new entry point is OK (to-do T8).

New, proposed, from the October 20 run shape, which lists 'what the agent may never do' as one of the eight details of a run, and from the safety rules already decided: a person approves every outside email, every approval shows its proof, and nothing leaves the building during tests. For one document, a short list the group could adopt as the rule for this path. Never send anything outside: no email, no upload, no call to an outside service with the document's contents, during tests. Never change the file: the scan stays where it landed in SharePoint, untouched; the path only reads it. Never overwrite an answer a person gave: once a person has settled a question about a document, that answer stays unless a person changes it. Never guess a deal: if the match is not sure, the record is written with no deal and waits (see 'The path, step by step'), it is never forced onto the nearest deal. Never write a fact without proof: a fact with no page and quote behind it is written as a question, not a fact. Open choice, where the rule lives. Option A, in the written task: the task handed to each arm says the rule in plain words, and we trust the arm to follow it. Plus: nothing to build. Minus: a weak arm can break it, and we only find out from the trail afterwards. Option B, in the sandbox: the run has no way out to email or the internet and no write access to the scan folder, so the rule cannot be broken. Plus: safe by build, not by trust. Minus: the path then cannot reach outside tools either, which may matter for the AI arms that run on a cloud model. Option C, both: the task says it and the sandbox enforces the parts it can. Suggestion: C. This is a proposal for the group, not a decision.

New since October 20. Joe and Reid set the shape of one agent run for the experiment: an agent gets one written task that says here is the job, here is your input (a closed deal folder), and does the whole deal job from there. It runs in a sandbox with the name-and-entity tooling inside, a person answers its questions, exactly one thing changes between any two arms, and the setup is modular so adding an agent is cheap. Eight details of a run are still unanswered (where it runs, what is inside, tools, what it hands back, who answers questions, what it may never do, the test set, and whether plain code is an arm or the pass line). None of this was said about one document, so where this path sits inside a run is open. Three shapes for the group. Option A, this path is a tool the agent calls: the agent opens the folder and hands each file to this path, which reads it, names the type, finds the deal, pulls the facts and writes the record; the agent only does the merge on top. Plus: every arm reads documents the same way, so the per-document scores are the same for all arms and the arms differ only in the merge and the judgement. Minus: the arms are then not really tested on reading documents, which is most of the work. Option B, this path is the written task, not a tool: the task tells the agent to do these seven steps for every file and write one record per file, and each arm does it its own way. Plus: this is what the experiment is meant to compare. Minus: the per-document trail depends on each arm following the steps, and a weak arm may skip the proof. Option C, both: the plain-code version of this path is one arm (or the pass line), and the AI arms get the same task in writing. Plus: matches decision D6, that AI must beat plain code per document type. Minus: two things to keep in step. Suggestion: C, because it gives the plain-code bar for free and still tests each AI arm on reading. Either way, 'When it is unsure' should line up with 'a person answers its questions' from the run shape: the ask goes to the same person, in the same place, whether it comes from this path or from the merge.

Proposed steps for one document. 1. Pick up: the document arrives as one file (see 'Where a document enters'). 2. Read it: turn the pages into text, keeping which page each line came from. 3. Name the type: lease, division order, check stub, tax document, mail, and so on. The 8 types in the test set are the starting list. 4. Find the deal: match the document to a deal we already have, using names, legal descriptions, well or lease names, and dates. This step runs plain code and AI side by side during tests (see 'Beating plain code'). It needs a 'no deal yet' outcome, because Reid noted paperwork like title packets arrives before a deal closes; in that case the record is written with no deal and waits to be matched later, instead of being forced onto the nearest deal. 5. Pull the facts: the fields that type of document carries, for example the decimal interest on a division order. 6. Write the record: one database row, which is the truth, pointing at the file that stays in SharePoint. 7. Check and finish: if everything is sure, mark it done; if anything is unsure, stop and ask a person (see 'When it is unsure'). Every step writes down what it did and why, so a person can look back at any one document and see the whole trail. Out of scope here: combining many document records into one deal record, and the basis allocation; those sit in the merge step described in 'How this fits whole-deal intake'.

Decision D6: for each document type, AI has to beat plain code before we use it on that type. Reid ran thousands of check stubs and other documents through plain code on Friday; code matched check stubs to the right deal 81% of the time, and each type gets its own bar. For this path that means the 'find the deal' step must be able to run two ways on the same document, so the same check scores both. Three shapes for the group. Option A, run both and compare: every document goes through plain code and through AI; both answers are saved with the record; the check reports each type's score side by side. Plus: gives the number D6 asks for with no extra runs. Minus: twice the work per document during tests. Option B, code first, AI only when code is unsure: plain code answers when it can; AI is called only on what code could not match. Plus: cheapest; AI only has to beat code on the hard cases. Minus: we never learn whether AI would have beaten code on the easy ones. Option C, AI only, with code's score as a fixed bar from Reid's Friday run. Plus: simplest build. Minus: the bar and the test are on different document sets, so the comparison is loose. Suggestion: A during the tests, then B for daily use once a type's bar is beaten. Reid also has 3 questions for Joe about the test documents and how far the plain-code version should go (to-do T32); the answers set what 'plain code' means for each step here.