Local Governments Caught Using AI Without Policies or Permissions

Written by Tan Yigitcanlar on January 5, 2025. Posted in Current News

In 2017, the city of Rotterdam in the Netherlands deployed an artificial intelligence (AI) system to determine how likely welfare recipients were to commit fraud.

After analysing the data, the system developed biases: it flagged as “high risk” people who identified as female, young, with kids, and of low proficiency in the Dutch language.

The Rotterdam system was suspended in 2021 after an external ethics review, but it demonstrates what can go wrong when governments adopt AI systems without proper oversight. As more local governments turn to AI in an effort to provide real-time and personalised services for residents, a “smarter” environment and better, safer systems, the risks are rising.

As part of our ongoing research, we studied 170 local governments around the world that use various AI systems.

We found AI is already touching nearly every aspect of public service delivery, and most of the governments didn’t even have a published policy about it.

AI in everyday governance

AI applications are affecting local governance in profound ways. Our international investigation uncovered 262 cases of AI adoption across 170 local councils, spanning a wide array of technologies and services.

cityscape of barcelona buildings — Cityscape of Barcelona. Image credits: Kaspars Upmanis.

We found these technologies are being deployed across five key domains.

1. Administrative services. For example, the VisitMadridGPT tourism chatbot in Madrid, Spain delivers personalised recommendations, real-time support, and cultural insights for visitors.

2. Health care and wellbeing. For example, during the height of the COVID pandemic in 2021, Boston mayor’s office in the United States launched an AI-driven chatbot for contactless food delivery, simultaneously addressing hunger and safety concerns.

. Transportation and urban planning. Logan City in Australia has implemented a real-time AI system that keeps drivers informed where parking is available, reducing congestion and frustration. Meanwhile, AI-driven route optimisation for public transport is being widely adopted to save time and emissions.

4. Environmental management. In Hangzhou, China, an AI system is being used to classify waste more efficiently, boosting recycling rates.

5. Public safety and law enforcement. Chicago in the US has used sensors and AI automation to shape law enforcement strategies. By pinpointing crime hotspots, the city reportedly reduced gun violence by 25% in 2018. However, this technology has also raised ethical concerns about racial profiling.

The double-edged sword of AI

Our study using AI found only 26 had published AI policies as of May 2023 – less than 16%. Most are deploying powerful AI systems with no publicly available framework for public oversight or accountability.

This raises serious concerns about ethical violations, systemic biases and unregulated data use.

Without robust policy, local governments risk deploying powerful AI systems without critical checks or external supervision. Algorithms could unintentionally discriminate against certain populations when allocating resources such as public housing or health services. The stakes may be incredibly high, as in Rotterdam’s welfare fraud risk scores.

Among the councils with AI policies, there was a clear emphasis on collaboration with stakeholders, raising awareness among employees and citizens, and ensuring transparency and regulation.

Among these, Barcelona City Council’s AI policy stands out. Its policy includes principles such as being transparent about AI, making sure AI decisions can be explained, and fair, and sets a benchmark for other municipalities.

Public in the dark

A recent survey our team conducted in Australia, Spain and the US shows a significant gap between public awareness and local government action about AI. More than 75% of respondents were aware of AI technologies and their growing presence in everyday life, but not when it came to local government initiatives.

On average, half of the respondents were unaware their local governments are actively using AI in public services. Even more concerning, 68% said they had no idea local governments have – or could have – policies governing AI use.

This striking lack of awareness raises pressing questions about the transparency and communication of local councils. As AI becomes increasingly embedded in urban management – from traffic monitoring to public safety and environmental sustainability – better informing the public is essential.

Without public understanding and engagement, efforts to build trust, accountability, and ethical oversight for AI in governance may face significant hurdles.

The future we face

There is no doubt AI systems have great potential to improve urban governance. But without policies that prioritise transparency, accountability and ethical use, cities risk unleashing a system that could harm more than it helps.

However, it’s not too late for local governments – and citizens – to avoid this grim future. Local governments can create robust AI policies that ensure fairness, transparency, and the ethical use of data. Citizens can be educated about AI’s role in local governance.

See more here zmescience.com

Please Donate Below To Support Our Ongoing Work To Defend The Scientific Method

PRINCIPIA SCIENTIFIC INTERNATIONAL, legally registered in the UK as a company incorporated for charitable purposes. Head Office: 27 Old Gloucester Street, London WC1N 3AX.

Trackback from your site.

Comments (3)

VOWG

January 5, 2025 at 11:25 am | #

You need AI to do that? Good grief people are really getting dumber and dumber.

Reply
Tom

January 5, 2025 at 12:15 pm | #

That was 7-8 years ago. A/i has jumped leaps and bounds over that time. A/i is now one of the preeminent anti-privacy, spying and surveillance machines, as it reaches its full designed and purposed potential. We will be replaced by A/i and therefore it needs to know everything about humans. All the public needs to know is how to buy a coffin and cemetery plot on the cheap. Hopefully, our A/i masters can guide us there.

Reply
Howdy

January 5, 2025 at 3:57 pm | #

“deployed an artificial intelligence (AI) system”
Read: Used an algorithm…

“There is no doubt AI systems have great potential to improve urban governance.”
Except:
“But without policies that prioritise transparency, accountability and ethical use, cities risk unleashing a system that could harm more than it helps.”
That’s why it’s great for govts, because they have none of those noble attributes. Digital thumbscrews is what they see it as.

AI is great when there is an actual Human lack of intelligence, or sheer laziness. Hence it’s use by governments that have no real intelligence, or otherwise.

“Citizens can be educated about AI’s role in local governance.”
I doubt the majority would understand, or even care.

Nobody comes to read my gas meter any more. You could say, technology, or AI took that job. It’s not about doing or being better, it’s about money and convenience, even after it’s shown to go wrong.
As allways, people cross the (supposedly unknown bad effects) bridge when they come to it, and it’s too late by then.

Reply