Contents —
Seventeen things we'd tell anyone rebuilding their firm around AI. How the system itself works is in How we analyse every deal with AI.
1. Don't let the AI write your plan
We started planning the build with Claude, and Claude is very happy to plan. Every page went into Notion, long, confident and in the same voice. Nobody on the team could tell you which were decisions, which were drafts and which were a model thinking out loud, including the people who'd asked for them. So we started again on a whiteboard (a Figma one, but still), mapped the whole thing out ourselves, and only brought the AI in once we knew what we were building.
2. Get the judgement out of people's heads first
Every experienced investor has rules they apply without thinking, like when a company is too early, or when a weak market doesn't matter because the founder is that good. None of ours were on paper, and we usually found out partners disagreed only when it came up on a live deal. Getting those rules precise enough for a model to apply meant settling arguments we'd been putting off for years.
Whatever your firm does, the judgement in your people's heads is what the AI needs most, and it's the slowest thing to get out. Start there.
3. Only build what nobody can sell you
We built the parts that hold our judgement ourselves, meaning the criteria every deal is scored against and the steps the AI follows to apply them. Everything else we pay for.
AI products for investors, like Kruncher, will score deals against your thesis, but the system underneath is theirs, and we change ours every week. We also got our own deal dashboard as far as a prototype, then bought Attio instead. Nobody wants to maintain a CRM, and we'd have been maintaining that one for the life of the fund.
4. Start where mistakes are cheap to fix
We started with screening. It's the highest-volume, most repetitive part of the job, and when the AI gets a screen wrong, a partner catches it in Slack and the deal carries on. That made it a safe place to find out what the system gets wrong. The deeper checks for deals that get past a first meeting come later, once we trust what's underneath.
The same goes for any firm. Start with the work you do most often, where a mistake is easy to see and easy to undo. Leave the work where one error costs you a deal, a client or your reputation until the system has earned it.
5. Don't make every step an agent
Right now everything is pitched as "agentic", as if a model should decide every step for itself. Most work doesn't need that. A lot of what gets called an agent is a model making a few tool calls, like searching the web or reading a file, inside a process someone could have written down. We started with agents handing work to each other, and every handoff lost something. Ours now runs the same steps in the same order every time, set in code, and the model makes the judgement calls and the tool calls inside each step.
If you can write down the order the work goes in, do that. Keep the self-directing agents for work where nobody can say in advance what comes next.
6. Everyone makes things up
There are two things that are true in this world. Death and taxes.
- Founders lie.
- AI hallucinates.
"Lie" is harsh. Mostly it's rounding up. A one-off contract becomes recurring revenue, and a pilot becomes a client. Fair enough, it's a pitch. The memo still needs the real number, so the AI checks every claim in the deck against its own research and the founders' work history, and shows both when they disagree.
The AI is harder to catch, because it doesn't know it's doing it. Early on, ours stated made-up facts as confidently as true ones. Now it's told to say when it doesn't know, a second agent checks every memo against the sources, and every claim is tagged verified, claimed or unverified.
Whatever your field, assume the inputs exaggerate and the model invents, and build a check for each.
7. Show your sources, and let a person decide
New material keeps arriving after the first run, like a call transcript, an email thread or the doubts of the partner leading the deal, and at seed none of the sources is reliably right. We tried deciding in advance whether the founder, the research or our own read wins, and couldn't make it work. Now every claim shows where it came from, disagreements stay visible, and a partner decides who to believe.
When sources disagree, the person deciding should see both sides, with where each came from.
8. Give a model a middle option and it takes it
Our first generation of agents rejected far too many companies that should have gone through, and for a seed fund that's the worst mistake there is. Deal tracking will watch the companies we pass on, so we'd find out eventually, but by then someone else would have backed them.
So we rebuilt the agents to be less harsh, and they swung the other way. The recommendation has three options, Proceed, Pass and More Info. Instead of saying no, they started saying More Info, which hands the decision back to a partner without saying which way to lean. Retuning the prompt on deals partners had flagged has pared that back, and it now passes on more of the deals it should.
If you give a model a "not sure" option, keep an eye on how often it picks it.
9. Let a person make every call
Hard rules handle most deals, such as our Australian-only mandate, but the exceptions are too varied to write rules for. An exceptional founder who happens to be based overseas is one. So the AI only ever recommends. A pass goes to a partner in Slack before it's final, and a partner reads every memo before deciding whether to meet the founder.
Let the system recommend, and keep a person on the final call wherever a wrong call would be expensive.
10. A second model isn't a second opinion
We ran the same deals through Claude, GPT, Gemini, DeepSeek and Qwen, and they mostly agreed, because the rubric does most of the work. To get real disagreement, our AI committee argues each deal from four positions on the same model, as a sceptic, an optimist, a futurist and an operator.
11. The newest model isn't automatically the best one for you
Every few weeks there's a new frontier model, usually launched with talk that AGI has arrived. Newer models also tend to think for longer before they answer, and you pay for all of that thinking. One we tested charged only a bit more per token, but a whole deal cost about one and a half times as much to run, and the memos were no better. We waited for the next release, which was.
Run any new model on your own past work before you switch, and look at what a whole job costs, not the price list.
12. Check every change against every kind of case
Before any change goes live, we run it over past deals we've already scored and compare the results with the current version. We check each kind of deal separately, because a change can improve the overall result while quietly making one kind of deal worse. We learned that when a change sharpened the founder analysis, broke the market sizing, and nobody noticed for a week.
Setting up those tests meant deciding what a good output even looks like, writing that down as something we could score, choosing which deals went in the test set and which to hold back, and working out how much a drop on one part of the analysis should matter. None of that is engineering, and none of it is investing either.
13. Someone has to learn to read the traces
Before any of this, none of us had read a trace, the step-by-step record of a run. When a run goes wrong you open it up and it's a tree of nested calls, each one with its inputs, its outputs, its cost, and you have to work out which step made the decision that ruined the rest. Learning to read that, and to tell a bad answer from a bad prompt or a bad input, took weeks.
14. Keep what you know in your own database
Everything about a deal is stored in its record, not in the model. So we can switch models, re-run one stage or rebuild the whole system without losing the rubric, the comparables or the tuning.
15. Don't make anyone open a new tab
Partners already live in Slack, deal records already live in Attio, and people were already editing documents in Notion, so that's where the criteria went. The AI comes to them. A recommended pass arrives as a Slack message with Agree and Disagree buttons, and the memo lands in the company's record in Attio. We didn't build a single new screen for anyone to check.
Every internal tool competes with the inbox. If using the AI means opening something new, people soon stop opening it. Put its output in the tools your team already has open every day.
16. Start on real work early, because that's where the edge cases are
Most of what broke was the thousand different shapes a deal arrives in. Our crude 2023 version had already hit thousands of them on live deals before we started the rebuild. Some of the ones that caught us:
- A founder replied to an email thread with no attachment, and the system treated it as a new deal with no deck, so the run stopped.
- A referral email can mention three companies at once, and each one needs its own record.
- Deals with no website yet crashed the whole run instead of carrying on without one.
- PowerPoint decks went to the PDF reader as if they were PDFs, and came out as garbled text.
- A founder who shared a name with someone else on LinkedIn got the stranger's career in the memo.
- Passed deals were tagged "International" because the word appeared somewhere in the reasoning.
The only way to find them is to run real deals, so the sooner you start, the sooner you're building for the deals you'll actually get. Keep the code that moves data apart from the code that does the analysis, so when something breaks you know which half to look in.
17. Budget for the learning runs
At the peak we were spending hundreds of dollars a day in OpenRouter credits just running deals to see what broke. The receipts landed several times a day, and every one was a small jump scare.
Part of Rebuilding the fund around AI, alongside How we analyse every deal with AI.