Automating Inequality

The AFST is run on every member of a household, not only on the parent or child reported to the hotline. Under the new regime of prediction, you are impacted not only by your own actions, but by the actions of your lovers, housemates, relatives, and neighbors. Prediction, unlike classification, is intergenerational. Angel and Patrick’s actions will affect Harriette’s future AFST score. Their use of public resources drives Harriette’s score up. Patrick’s run-ins with CYF when Tabatha was a child will raise Harriette’s score as an adult. Angel and Patrick’s actions today may limit Harriette’s future, and her children’s future. The impacts of predictive models are thus exponential. Because prediction relies on networks and spans generations, its harm has the potential to spread like a contagion, from the initial point of contact to relatives and friends, to friends’ networks, rushing through whole communities like a virus.