Innocent Until the Algorithm Says Otherwise

Something is shifting in how courts decide what to believe. Here is what I think is happening, in plain English.

A woman in Florida spent two days in jail. The evidence against her was a set of text messages her ex-boyfriend had allegedly made with an AI tool. They were not real. It still took eight months for the charges to go away. Afterwards she said the thing that has stayed with me since I read it: nobody checked whether the evidence was genuine.

What put her in a cell was a screenshot. Not a witness, not a confession, not a police officer who saw something happen. A screenshot that looked exactly like every other screenshot anyone has ever seen.

I am not a lawyer. I write about what AI is doing to the institutions we all rely on, and for the past few months I have been reading court decisions from several countries. What I found surprised me. The pressure AI is putting on courts is coming from two directions at once, and the two directions pull opposite ways. In both of them, the person who ends up carrying the extra weight is the ordinary individual.

I want to explain what I have seen, plainly, and then ask people who actually practise law whether I have got it right. Every case I mention is linked, so if you want the full ruling and the technical reasoning, it is one click away.

What "innocent until proven guilty" really depends on

Everybody knows the phrase. Fewer people think about what makes it work.

It works because the person making the accusation has to prove it. You do not have to prove you did not do it. And that arrangement only holds up as long as courts can trust the things used as proof. A photograph. A recording. A bank record. Somebody has to be able to look at the evidence and reach a sensible conclusion about whether it is real.

That last part is the bit that is coming loose.

Direction one: when a machine accuses you

Start with the side almost nobody talks about, because it started long before AI.

In England and Wales there is a rule that courts should assume a computer was working properly. If a computer says money is missing, the court starts from the position that the computer is right. If you disagree, it is on you to show the machine was wrong.

That rule helped produce one of the worst injustices in British history. Faulty accounting software at the Post Office made it look as though cash was missing from local branches. More than 900 branch managers were prosecuted between 1999 and 2015. People lost their savings, their reputations, and in some cases their freedom. Many of them said all along that the software was broken. They were right. But the rule meant they had to prove it, against a company that owned the software, held the data, and employed the experts. In January 2025 the British government finally opened a formal review of whether that assumption should continue.

The same shape shows up elsewhere. Australia ran a scheme that calculated welfare debts automatically and sent people bills. It was later found to be unlawful, partly because it made people prove they did not owe the money rather than making the government prove they did. In the Netherlands, a court shut down a system that scored citizens for likely welfare fraud. In Detroit, a man named Robert Williams was arrested on his own front lawn in front of his wife and daughters because face recognition software matched a blurry shop camera image to his driver's licence photo. The city eventually agreed that police could not arrest anyone on a face match alone, and agreed to go back through years of old cases where the software had been used.

Here is the trend, stated simply. When a computer system accuses you, the accusation tends to arrive already believed. Undoing it takes money, technical knowledge, and access to a system you have never seen. Most people have none of those things. For the Post Office branch managers, this was not a figure of speech. It was how the rule actually worked.

Strip away the legal language, and what this amounts to is simple: when the machine accuses you, you are guilty until you can prove yourself innocent.

Direction two: when a video accuses someone

Now the side everyone talks about, which runs the other way.

Earlier this year, New York's highest court threw out videos in a child abuse case. The videos appeared to show a mother's boyfriend abusing her fourteen-year-old daughter. They had turned up on the computer of a man who was under investigation for trading child abuse material and who said he had hacked into the family's home security cameras. Police went to the house and photographed the living room. It matched the footage down to the couch, the lamp, and small objects visible in the videos. Two lower courts accepted the videos. The highest court did not.

The important thing is why. Nobody proved the videos were fake. Nobody even suggested they might be fake until the case was already on appeal. The word "deepfake" was never spoken at the original hearing. The court's point was different: given how the videos had surfaced, and given that convincing fakes now exist, the people bringing the case had not done enough to show the videos were genuine.

Three of the seven judges disagreed, strongly. One of them warned that from now on, anyone facing damaging video can simply suggest it might be AI-generated, with nothing to back that up, and the other side then has to prove it is not. She asked the questions that stuck with me. Are doorbell cameras and traffic cameras now open to that challenge? What happens when someone accused of a crime produces a video proving they were somewhere else, and the prosecution says it might be fake? Does that person now have to hire an expert, or take the stand and give up their right to stay silent?

You can see the same move being tried elsewhere, usually without success. In a lawsuit over a fatal crash involving Tesla's Autopilot, the company's lawyers suggested that recordings of Elon Musk making public claims about the technology might be fakes and should not count. The judge called that argument "deeply troubling" and ordered him to answer questions under oath, pointing out that famous people cannot say whatever they like in public and then hide behind the possibility that the recording was faked. At the first trial arising from the January 6 attack on the US Capitol, the defence hinted that video of the defendant might be fake, offered nothing to support it, and the jury convicted him anyway.

So judges have mostly been unimpressed, and I would rather report that than leave it out.

But the problem is real in the other direction too. A judge in California spotted a fake video of a witness that had been handed in as genuine testimony, and threw the case out. A judge in Washington State refused to allow video that had been sharpened by AI software in a murder trial, because nobody could explain what the software had actually added to the picture.

The trend on this side: a recording no longer speaks for itself. It used to be treated as close to the last word. Now it is one more thing that has to be vouched for, and vouching for it is expensive.

Worth noticing, too, that the rules have not caught up. In the United States, two proposed changes designed to handle AI evidence have been drafted, argued over, and sent back for more work. Neither is in force. Judges are handling this with the rules they already had, which were written for a different world.

Why this is happening

In the book I have spent the past two years writing, Future Without Consent, I gave this dynamic a name. I call it the falsehood scaling law, and it is simple enough to fit on one line.

S = k / C

S is the amount of falsehood floating around in a society. C is the cost of producing one more lie, counted in money, time, skill, and effort. The k accounts for everything that helps a lie travel once it exists, such as social media reach and automated targeting. All the formula says is this: the cheaper it gets to lie, the more lies there will be. And as the cost gets close to nothing, the amount of falsehood has no ceiling.

For nearly all of human history, lying well was expensive. Forging a document took a skilled forger. Faking a photograph took a darkroom and real talent. Imitating somebody's voice convincingly was, for practical purposes, impossible. That expense was never designed to protect anyone, but it protected us anyway, and a lot of our institutions were quietly built on top of it. Courts included. Rules about evidence were written in a world where fakes were rare and costly, which is why a video with a believable backstory could be treated as probably genuine.

A convincing fake now costs a monthly subscription and a few minutes.

Nothing in the rulebook changed. The world underneath the rulebook did. That is the honest explanation for something that otherwise looks like confusion: judges keep saying the existing rules are good enough, and they also keep throwing out material those same rules would have accepted a decade ago. Both things are true at once.

There is a second cost that moves in the opposite direction, and it is the one that decides who gets hurt. As faking gets cheaper, proving something is not fake gets more expensive, because common sense is no longer enough and you need a specialist. Common sense is free. A specialist is not, and there are very few of them.

A large company defending a lawsuit can afford that specialist. A parent fighting to keep their children, a Post Office branch manager, or a woman sitting in a Florida jail cannot.

What I think this adds up to

Nobody has abolished the presumption of innocence. No court has ruled that you are guilty until proven innocent, and I am not claiming one has.

What is moving is something narrower, and in everyday life more important: the question of who has to prove what about the evidence itself. It is moving in both directions at the same time.

When a system accuses you, its word tends to be taken, and disproving it is beyond most people's reach. When a recording accuses someone with money, that recording can now be met with a shrug and a suggestion that it might be AI, which costs nothing to say and a great deal to answer.

Put those together and you get a pattern worth worrying about. The burden is drifting toward whoever is least equipped to carry it.

Three reasons I might be wrong

I would rather name these myself than have them turn up in the comments as an ambush.

One case is not a trend. The New York decision was close, four judges to three, and the three who disagreed thought it was a serious mistake. It may end up as an oddity that other courts decline to follow. Nobody knows yet.

I may be tying together things that are not connected. The rule about trusting computers and the question of proving a video is real are different corners of the law. Treating them as one story is exactly the kind of thing a non-lawyer does. Perhaps they only look related from the outside.

The existing rules may be enough. That is the view of many of the people whose job it is to write the new ones, and it is not a weak argument. Courts have already blocked fake and AI-altered material without any special AI rule at all.

What I would like to know

If you work in or around the law, I would like to hear from you.

Are you seeing this argument raised in your own cases, and how is it being handled? Has anyone had to hire an expert to prove a recording was real, and what did that cost? Does the New York decision worry you, or does it look to you like the system working as intended?

And underneath all of it, the question I keep coming back to. In a world where any recording can be doubted, and any computer's output tends to be believed, who should have to prove what?

I do not think the answer is obvious. I do think it is being decided right now, one case at a time, mostly out of sight, by people who never asked for the job.

Asim Javed is an AI methodologist and the author of Future Without Consent: How AI, Power, and Profit Are Deciding What Comes Next. The falsehood scaling law comes from Chapter 5, "Truth Dies Quietly."