Outside of law enforcement, this is certainly how shitty customer service policies get enforced. In other words, “Computer says no”.
The British Post office rolled out a hugely buggy piece of software that bankrupted small business owners, got some sentenced to years in prison, and caused thirteen people to commit suicide because “computers can’t be wrong”
Here in Australia they rolled out an automated system to calculate welfare overpayments and issue debts. It didn’t quite work of course and hundreds of thousands of the poorest Australian were issued with false debts, some of whom died or committed suicide before they could be repaid. People still keep floating the idea of automation and AI in our welfare systems…
Oh yeah! I forgot about that!
The case was settled for £58 million, leaving the claimants with £12 million after legal costs.
Ewwww
Law enforcement will seize and use computers and the data they hold as evidence to convict criminals, just like any other tool that they might be warranted to seize.
Courts will examine the evidence of what it did to determine what role it played in the offence and whether it supports the allegation.
Likewise police complaints authorities could do the same in principle against the police; if someone were to give them a warrant and the power to execute it.
If a thing happens in public that was unwarranted and can be traced back to a police force or how they deployed any equipment, they can be judicially reviewed* for any decision to deploy that bit of kit. It’s more a matter of will they actually be JR’d and will that be review be just and timely. * - in my country.
I don’t think it’s much different from how they deploy other tech like clubs and pepper spray, tear gas, tazers or firearms. If they have no fear of acting outwith their authority , that’s a problem.
In some ways it might be easier to have an ‘our word’ vs ‘their word’ defense when they shoot someone, compared to a computer program that might literally document the abuse of power in its code or log files.
“Oops i dropped my notebook”, is maybe easier than, “oops i accidentally deleted my local file and then sent a request to IT - that was approved by my manager - asking them to delete instead of restore any onsite or offsite backups”.
Now where does this thought come from?
Do you not know what a computer is? It’s literally a digital logical accountant! Yeah yeah, we should probably blame the programmers and engineers instead when shit goes sideways, but now I think we need to also hold CEOs accountable when they decide to inject faulty AI into mission critical systems…
There’s a reason why license agreements often stay there there are no warranties express or implied, no guarantees, and no fitness for any particular purpose.
“This software is useless and should not be used by anyone for any purpose” is my favorite part of license agreements.
If a building collapses. You blame the people who built the walls and poured the concrete, or the ones who chose the materials and approved the project?
In any case, often programmers and engineers retain no rights to the software they worked on. So whoever profits from the software should also shoulder the blame.
the AI decided ¯_(ツ)_/¯
Excuse me, you dropped this: \
No, I’m sorry, the AI has decided you don’t need that forearm, but you’re welcome to keep the hand.
IT disagrees. Misbehaving hardware can be taken out back and shot.
Same argument for cars
…or any inanimate objects, really.
This is unironically one of the main drivers of AI. As soon as all crucial social systems are inundated with AI, the built-in bias will be excused as “minor glitches” of the system, but the real reason was always a total lack of accountability.
That’s why cops love using dogs too. Courts have ruled that dogs can’t lie. That means if a dog indicates you have contraband, then a search is warranted, even if nothing is found. This of course ignores that it is entirely possible the dog indicated contraband because the cop trained it to do so on command.
We don’t jail the gun for murder.
