Bloomberg went through roughly two million Polymarket addresses active since early 2025. More than 100,000 of them booked losses over $1,000. Retail lost around $131 million in total, while the top 1% of accounts — mostly bots — took more than 80% of the profits.

Here's the part almost nobody quotes. The human traders picked the correct outcome more often than the bots did.

Joshua Della Vedova, the University of San Diego professor involved in the analysis, put it plainly: the bots weren't better forecasters. They were better at getting in. They entered early at good prices while the humans showed up late and paid up.

Being right isn't the edge. Acting on being right, at a price that still leaves room, is the edge.

Two things are now both called "a bot"

I keep coming back to that finding because of what the last year has done to the word. Two years ago a trading bot meant a script — if this condition, then that order.

Now a growing share of what's being sold are agents: language models that read the news, form a view, and decide for themselves what to do next. That sounds like a straight upgrade. It isn't. It's a different tool doing a different job.

A rule-based system is deterministic. Feed it the same six years of candles and it takes the same trades, in the same order, every time you run it. Boring — and that's precisely why the number at the bottom of the test means something.

An agent doesn't behave that way. A 2026 study on the TradingAgents framework measured it directly: hold every input constant, run the pipeline again, and you get different decisions and a different equity curve.

Non-determinism is a design property of these systems, not a bug someone forgot to patch. Dropping the temperature to zero narrows the spread. A fixed seed doesn't close it either.

Why reproducibility isn't a technicality

That repeatability is the whole basis for sitting through a drawdown. When you're down 18% and every instinct is telling you to switch it off, the only thing that keeps you in the seat is knowing this exact shape appeared three times in the test and recovered each time. You can't build that kind of conviction on a process that improvises.

So what is a backtest of an agent actually telling you? One draw from a distribution whose shape you've never seen. Run it ten more times and you'd have ten more equity curves. The one in the marketing deck is, statistically, the best of them.

Picture the live version. Your agent sits flat through two weeks of chop, then reads a headline about an ETF approval and decides this particular setup deserves double size. Nothing in that decision is checkable against history, because it never made that decision in history. You find out whether the judgment was any good the same way you'd find out about a discretionary trader — with money, slowly.

There's a labelling problem sitting on top of all this. One 2026 estimate puts roughly 95% of retail "AI" trading products at ordinary rule-based scripts wearing an AI badge. Which is almost funny: you're paying a premium for reasoning you aren't getting, layered over a strategy you could have tested yourself if anyone had told you what it was.

The mistake underneath both versions is treating adaptability as a synonym for edge. A system that can change its mind only beats one that can't if its judgment is reliably good — and you have no way to establish that, because you can't run the same test twice and get the same answer.

Split the job

Let an agent do research. Summarising filings, flagging funding rates that look wrong, reading things you'd never make time for — reasoning genuinely helps there, and a bad summary costs you ten minutes. Then let a tested, deterministic rule handle entry, sizing and exit, inside limits you set while the market was closed and you were calm.

One question sorts most of this out. Can you say exactly what this thing will do at 3am on a 6% candle? If the honest answer is "depends what it decides," you don't own a system. You own a colleague with account access and no sleep schedule.

We've stayed on the boring side of that line on purpose — fixed rules, fixed sizing, and results you can reproduce candle by candle instead of taking on faith.