Yonatan’s Death Spurs Deep Look at How L.A. Gauges Child Abuse Risk

Last month, in response to news reports about the tortured life and death of a little boy, the Los Angeles County Board of Supervisors waded into the thorny question of how to predict child abuse.

The board’s first substantive action, in the wake of 11-year-old Yonatan Aguilar’s death, was to pass a motion questioning whether the county’s long-used system for measuring the risk of child abuse was working.

“Who was there to catch that failure in the system, to protect that young child, save that life?” asked one of the motion’s authors, Supervisor Mike Antonovich, during a September 20 board meeting. “Should that child should have been removed from the family sooner?”

In August, police found Yonatan’s 34-pound body, wrapped in a blanket, in the closet of his mother’s apartment in Echo Park.

Just last week, The Los Angeles Times published new details about Yonatan’s case, again noting that Aguilar’s family was known to the county’s child protection agency, the Department of Children and Family Services (DCFS).

Using a risk-assessment protocol called Structured Decision Making (SDM), child abuse investigators deemed the family at high risk for subsequent abuse four times from 2009 to 2012.

But Times reporters Richard Winton and Hailey Branson-Potts neglected to compare those “risk” scores to also important “safety assessments.”

Case files show that the four safety assessments conducted alongside the “risk assessments” all deemed that “no safety threats” were present. DCFS maintains that its workers cannot remove a child if there is no threat to his or her safety.

During the September 20 board meeting, supervisors gave the county’s Office of Child Protection and DCFS 30 days to complete a report diving into those issues. That deadline has come and passed, and the report’s timing remains unclear.

Ultimately, blame will be hard to ascribe in Yonatan’s case. It may be an instance where tragedy escapes the reach of a system unable to prevent it.

It will, however, be a chance for the county to take a hard look at the “front door” of child protection — that murky space between a caller reporting child abuse and the moment a social worker either removes the child, or walks out of that child’s life.

The implications are significant. More than half the states in the country use SDM, making what happens here an issue of national importance.

What is Structured Decision Making?

Structured Decision Making is “a suite of assessment instruments that promote safety and well-being for those most at risk— from children in the foster care system to vulnerable adults,” according to its developer, the non-profit National Council on Crime and Delinquency (NCCD).

When it comes to child protection, that “suite of instruments” is really a set of six different questionnaires that help social workers make decisions about what to do with referrals of child abuse like those involving Yonatan Aguilar. The system is very similar to the “actuarial” questionnaires that insurer’s use to determine rates.

At every step of the case ­– from the moment the call comes into the child abuse hotline to the reunification with parents after children have been removed from their family homes – NCCD’s suite of child protection tools help social workers decide what action to take.

For years, SDM has enjoyed primacy in the risk-assessment market.

Evaluations, many of which were commissioned by NCCD, or conducted by former contractors, indicate that SDM is a better tool for stratifying risk than a worker’s gut instincts or the consensus of a worker and his or her peers and supervisors.

In a 2012 report submitted to England’s Department of Education, which reviewed a wide range of risk assessment tools, researchers concluded that SDM is essentially better than anything else out there, but still far from perfect.

“Our review found that although there is evidence favouring the validity, reliability and impact of one actuarial SDM risk assessment tool (California Family Risk Assessment tool), evaluation of its implementation in other contexts highlighted a range of significant problems,” Jane Barlow and her colleagues wrote.

What the British research team was saying is that there is significant variability in SDM’s accuracy depending on where it is deployed, and who is using it.

Here in Los Angeles, SDM featured prominently in a scathing report submitted to county officials in 2012. Conducted by the county’s Children’s Special Investigative Unit, the report looked at 14 child fatalities and one critical injury that were caused by severe abuse. In 11 of these cases, a social worker either failed to use SDM, or misused it.

This type of user error was central to the explanation given by NCCD CEO Kathy Park for Yonatan’s death.

“The Board of Supervisors needs to find out why the high risk level was repeatedly not acted upon,” Park said during public comment at the September 20 board meeting. “It needs to understand whether this is happening in other cases; children in Los Angeles deserve this. However, the most crucial issues go beyond Structured Decision Making to the organizational and systemic. The Board of Supervisors needs to ask: is the Department of Children and Family Services’ leadership supportive of using research in child protection?”

A week later, Park submitted an op-ed to The Chronicle of Social Change and went further, writing that the board needed to ask: “Is the Department of Children and Family Services’ leadership establishing a culture that values and respects the life of every child?”

DCFS’ top official, Director Philip Browning, said that his workers followed the protocols outlined by NCCD when investigating Yonatan’s case, and suggested that Park’s comments were more about retaining SDM’s hold on the child abuse prediction market than genuine outrage.

“I think any organization that gets paid to do work is going to defend their work and try to rationalize their tool as being correct and say that someone else has misused it,” Browning said.

He added that the county is exploring alternatives to SDM.

Most notably, DCFS experimented with the use of predictive analytics to measure risk in 2014. Results from that experiment suggested that using algorithms that mine vast data sets to ascertain risk could be useful in trying to predict serious child maltreatment.

Browning also said that he is in talks with the California Department of Social Services to develop a predictive analytics system that would be applicable across all 58 counties, most of which currently use SDM.

“I think they [NCCD] know we’re interested in a predictive analytics tool, and I think they see that as competition, and they are concerned about that,” Browning said. “And I think they have a proprietary interest in continuing what they’re doing.”

According to tax filings from 2012 through 2014, NCCD’s annual revenue ranged from roughly $15.5 million to $20.5 million. Ninety-seven percent of that money was derived through government contracts, not charitable donations.

A 2012 audit showed that NCCD brought in $7.7 million in child welfare program revenues, which includes some money from SDM.

In a series of emails, NCCD Communications Director Erin Hanusa maintained that her agency is “not a commercial vendor whose goal is to sell a product.”

When asked to provide details on overall revenue from SDM, Hanusa wrote, “Compiling an estimate would require many hours, as contracts often contain multiple pieces of work, some SDM-related, some not. Given the extensive time it would require to calculate these numbers, it isn’t feasible to provide more detailed information.”

But she did point to DCFS’ payment schedule with NCCD from 2015-2017, which authorizes annual expenditures ranging from $33,570 to $55,570.

How was Structured Decision Making Used in Yonatan’s Case?

Los Angeles County uses three SDM tools when trying to determine what to do with a referral like those lodged against Yonatan’s mother, Veronica Aguilar.

First, workers at the child abuse hotline use something called the “intake assessment” to determine whether or not they should investigate a call. If they decide to go out, the tool helps them decide how fast they need to complete their investigation.

Once at the family’s home, workers conduct a “safety assessment.”

DCFS investigators conducting safety assessments in Yonatan’s case from October 2009 to March 2012 always checked “no” to prompts like the “caregiver caused serious physical harm to the child” and “caregiver describes child in predominantly negative terms.”

SDM automatically spat out this phrase every time:

“No safety threats were identified at this time. Based on currently available information, there are no children likely to be in immediate danger of serious harm.”

If a child is deemed in imminent risk of harm, the worker will either detain them on the spot or set up a “safety plan” to mitigate safety threats until they can get a warrant to remove the child or children.

The third step in SDM’s initial child protection protocol is called a risk assessment. This is what that the Times focused on.

As reported in The Times, Yonatan’s risk for subsequent abuse was rated at high risk in four investigations from 2009 to 2012.

Despite the SDM generating a recommendation that the case be “promoted” each time, the workers used the tool’s override function to close the case.

So, Was it the User or the Tool?

“Now in this particular case, from what I know about it so far, I don’t know that it is necessarily a case where the tool, the risk assessment tool, Structured Decision Making, is called into question,” said Office of Child Protection Director Michael Nash during the September 20 board meeting.

“However,” Nash continued, “I have questions about the implementation of that tool.”

While Yonatan’s is not likely the test case for the failure of DCFS’ current risk assessment system, it has served as a prompt for the county to address one of the most fundamental issues within child welfare today.

Ever since that 2012 report conducted by the Children’s Special Investigative Unit, there has been an urgent need to better understand how the child protection system responds to allegations of abuse.

Now, under these terrible circumstances, it will.

Holden Slattery contributed to this story.