Can NYC’s Algorithm Policy Problems Inspire Smarter Government Tech Deals?

New York City was supposed to lead other municipalities around the world into the tech policy future. Its task force on automated decision systems was created to explore how city agencies use algorithmic technologies and devise concrete guidance for determining whether and how to employ those technologies going forward.

Instead, as I reported in CityLab this week, some believe the project suffered from bureaucratic limitations and failed to produce concrete results.

In fact, in response to the city’s official task force report, advocates for civil rights and algorithmic justice went so far as to publish a shadow report detailing what they saw as an opaque process with little community engagement and presenting their own set of guidance.

Its editor, Rashida Richardson, director of policy research at the AI Now Institute at NYU, worried of the flawed effort’s domino effect, suggesting it would set a “dangerous precedent.”

Read about the task force, the city’s official report and the shadow report criticism in Kate’s CityLab story.

As RedTail was first to report back in April, problems with the initiative were made public months ago. Members and engaged community stakeholders complained of a lack of transparency around process. And perhaps more alarming, nearly a year after the effort was announced, there was still no inventory of automated decision systems used by city agencies.

In the end, task force members never got that list.

So, now what? For Richardson, NYC’s effort could inspire cities to take more control when negotiating tech contracts. Here’s what she said in the CityLab story:

“New York City should use its purchasing power to only contract with firms willing to disclose the details of their technology, how they will be used, and who could potentially be harmed. Any firm not willing to be transparent about their technology should be left off the NYC payroll.”

As noted in that story, there are others working on thoughtful approaches to help guide governments as they navigate the murky waters of decision-making technologies that affect policy.

Will they pay attention?