Wyden: Stopping algorithm discrimination

Oregon Senator Ron Wyden,

Sen. Ron Wyden, D-Ore., Sen. Cory Booker, D-N.J., and Rep. Yvette D. Clarke, D-N.Y., introduced the Algorithmic Accountability Act, which requires companies to study and fix flawed computer algorithms that result in inaccurate, unfair, biased or discriminatory decisions impacting Americans.

“Computers are increasingly involved in the most important decisions affecting Americans’ lives –whether or not someone can buy a home, get a job or even go to jail. But instead of eliminating bias, too often these algorithms depend on biased assumptions or data that can actually reinforce discrimination against women and people of color,” Wyden said. “Our bill requires companies to study the algorithms they use, identify bias in these systems and fix any discrimination or bias they find.”

“50 years ago my parents encountered a practice called ‘real estate steering’ where black couples were steered away from certain neighborhoods in New Jersey. With the help of local advocates and the backing of federal legislation they prevailed. However, the discrimination that my family faced in 1969 can be significantly harder to detect in 2019: houses that you never know are for sale, job opportunities that never present themselves, and financing that you never become aware of—all due to biased algorithms,” Booker said. “This bill requires companies to regularly evaluate their tools for accuracy, fairness, bias, and discrimination. It’s a key step toward ensuring more accountability from the entities using software to make decisions that can change lives.”

“Algorithms shouldn’t have an exemption from our anti-discrimination laws. Our bill recognizes that algorithms have authors, and without diligent oversight, they can reflect the biases of those behind the keyboard,” Clarke said. “By requiring large companies to not turn a blind eye towards unintended impacts of their automated systems, the Algorithmic Accountability Act ensures 21st Century technologies are tools of empowerment, rather than marginalization, while also bolstering the security and privacy of all consumers.”

The bill follows numerous reports of computer algorithms creating biased and discriminatory results. Earlier this month, the Department of Housing and Urban Development charged Facebook with violating the Fair Housing Act by allowing advertisers to discriminate based on race, religion and disability status. Last year, for example, Reuters reported that Amazon shuttered an automated recruiting tool that was biased against women.

The Algorithmic Accountability Act would:

– Authorize the Federal Trade Commission (FTC) to create regulations requiring companies under its jurisdiction to conduct impact assessments of highly sensitive automated decision systems. This requirement would apply both to new and existing systems.
– Require companies to assess their use of automated decision systems, including training data, for impacts on accuracy, fairness, bias, discrimination, privacy and security.
– Require companies to evaluate how their information systems protect the privacy and security of consumers’ personal information.
– Require companies to correct any issues they discover during the impact assessments.

Flexibility for Small Businesses

– The obligations in the bill only apply to companies that are already regulated by the FTC and that make more than $50 million per year.

– However, data brokers or companies that have data on more than 1 million consumers or consumer devices are covered, regardless of their revenue.

The bill is endorsed by tech and civil rights groups, including Data for Black Lives, the Center on Privacy and Technology at Georgetown Law and the National Hispanic Media Coalition

“At Data for Black Lives we are building a movement of organizers, scientists and technologists using data science to create concrete and measurable change in the lives of Black people. The Algorithmic Accountability Act of 2019 helps advance this work by putting accountability and public education front and center. We know first-hand the harmful impact that automated decision systems have on parents fighting for access to quality education, black mothers engaging health systems in how to provide care that protects their newborns, and activists fighting against community disinvestment and deprivation. Beyond regulation, we are hopeful that this legislation will lead to a broader discussion about the tremendous potential for data systems, if used ethically, to uplift, empower, and democratize our communities.” – Data for Black Lives

“Automated decisions are not neutral decisions. They turn on human data. As long as humans are biased, algorithms will be biased too. This bill will force companies to reckon with that reality.” – Center on Privacy & Technology at Georgetown Law

“Algorithms control various aspects of a digital economy. They determine which candidates will be interviewed and how much they will be paid; who will be targeted for or excluded from advertisements; and how much consumers will pay for goods and shipping online. Still, the public is largely in the dark about how personal data is collected, manipulated, shared, and stored. Especially for Latinos and other marginalized communities who have been frequent subjects of data harms, the Algorithmic Accountability Act of 2019 injects much-needed transparency and accountability into data frameworks that codify unconscious bias and institutional discrimination into code.” – Francella Ochillo, VP of Policy and General Counsel, National Hispanic Media Coalition