How to Use The Early Warning Project's Statistical Risk Assessment

Earlier this year, we shared the results of our Early Warning Project’s latest Statistical Risk Assessment (SRA)—a list of 166 countries ranked by their risk for the onset of mass killing. The results provide an overview of worldwide risk and are presented as a ranking so policymakers, NGOs, and governments can prioritize and distribute resources effectively.

This assessment seeks to identify countries that are most likely to experience a new mass killing. We consider a mass killing to have occurred when an armed group in a country deliberately kills more than 1,000 noncombatant civilians (in the same country), over a 12-month period, because of their membership in a particular group. As this definition indicates, our assessment is only able to forecast mass killings within countries (i.e., in which the perpetrator group and the targeted civilian group reside in the same country). 

But how does it work? Using sophisticated machine-learning techniques, our model learns from the past in an attempt to forecast the future risk of mass killing. The model looks for patterns in historical cases of mass killing and identifies how variables or potential “risk factors” are related to the onset of mass killing. Simply put, the model is asking: Which countries today look most like countries that have experienced a mass killing in the past in the year or two before those mass killings began? 

The results of this risk assessment should be a starting point for discussion and further analysis of opportunities for preventive action. We see it as an interactive process, combining statistical results with other kinds of analysis to provide the most useful assessments. For example, the US Atrocity Risk Assessment Framework, updated in 2022, sets out guidance for the kind of in-depth analysis that should be conducted on countries near the top of our SRA.

So how can one use the risk assessment to identify countries to prioritize?

When we consider the results of the SRA, we first look at four types of countries:

1. Highest risk

2. Consistently high-risk

3. Significant shifts in risk

4. Unexpected results

For countries in each of these categories, we recommend asking certain key questions to gain a fuller understanding of the risks and adequacy of policy response and to identify additional useful lines of inquiry. 

Highest risks

As with any country ranking, we begin by looking at countries at the top. In our latest assessment Afghanistan, Pakistan, and Yemen top our list. For these countries, we want to ask: Is there adequate attention to the risk of mass atrocities in the countries facing the very highest risks of mass killing?  

While we consider all countries in the top 30 to be high risk, “high” in this assessment is relative. For example, the highest-risk country in our most recent assessment, Afghanistan, has an estimated 6.5% risk, or a 1 out of 15 chance, of experiencing a new mass killing. Although mass killings are low-likelihood events, they are so destructive that it is important to take steps to prevent them.

In Afghanistan, many groups, including women, Hazara, and other ethnic and religious communities, are vulnerable to targeted violence. These groups face discrimination and attacks by the Taliban, the de facto authorities, and a nonstate actor, the Islamic State of Khorasan Province. To help understand how these risks could lead to mass killing, we want to ask: What might spark large-scale violence by the government or non-state actors against a particular community? (For additional information on risks in Afghanistan, see here).

Consistently High-Risk

If a country ranks relatively high for multiple years running, we take this as a strong indication that policy makers should devote significant attention to understanding and addressing mass atrocity risks. In our most recent report, we highlight countries like Sudan, Ethiopia, and India.

Sudan, for example, has consistently ranked within the 15 highest-risk countries in our assessment. On April 15, 2023, fighting broke out in Sudan between the Sudanese Armed Forces (SAF) and the Rapid Support Forces (RSF). From the start of the conflict both the RSF and the SAF have targeted civilians in a bid for control and power, with spiraling violence fed by cycles of grievance and impunity. As war spreads and impunity persists, civilians continue to face escalating risks. In January we published a policy brief that assesses the current and future risks of mass atrocities in Sudan and provides policy options to mitigate these risks. This is just one example of how the SRA can draw attention to issues in a country that is consistently at high risk. (For additional information on risks in Sudan, see here).

Significant shifts in risk

We recommend that people using our data pay special attention to countries that have moved significantly in the rankings from the previous year or years. In our latest report we highlight Tajikistan and Burkina Faso as countries that moved up in our rankings substantially, and Chad, as a country that moved down in our rankings. 

While some of the variables included in the model, such as infant mortality or ethnic fractionalization, stay relatively consistent year to year, others, such as a coup attempt, can cause a country to jump in the rankings. Key questions for countries with significant shifts in risk are: What events or changes explain the big shift in estimated risk? Have there been additional events or changes, not yet reflected in the data, which are likely to further shift the risk? Does the increase or decrease indicate an ongoing trend?

Burkina Faso, for example, moved up to the top 30 highest-risk countries in our risk assessment from the 40s in 2022–23. In recent years, the country has experienced growing instability and violence, including two coups in 2022. Since then, civilians have faced increased attacks by state security forces and Islamist nonstate armed groups. For countries like Burkina Faso, we want to understand why there was a shift in risk and what might lead to a continuing trend upwards in risk.

Unexpected results

The top countries in our SRA tend to confirm people’s intuitions: mass killings are most likely in countries with ongoing civil wars and other violent crises. But sometimes certain countries' results may come as a surprise. This year we highlight South Sudan and Indonesia. 

In cases where our statistical results differ substantially from expectations, we recommend conducting deeper analysis and revisiting assumptions. The purpose of the SRA is not to pit qualitative analysts and statistical models against one another, but rather to suggest opportunities where we can deepen our understanding of risk in the country in question. For countries with unexpected results we should ask: What accounts for the discrepancy between the statistical results and experts’ expectations? What additional analysis would help shed light on the level and nature of atrocity risk in the country?

In this year’s results, Indonesia’s high ranking of 14 may come as a surprise, given that it is one of the world’s largest democracies and plays a major political role on the international stage. However, discussions with country experts emphasized the increasing tension and violence between Indigenous Papuan supporters of Papua’s long-standing independence movement and the Indonesian government. In 2022, the Museum’s Early Warning Project published a report assessing the risks of mass atrocities in the Papua region. 

In sum, the SRA is a useful tool for identifying countries at high or rising risk of mass killing. Used together with other kinds of analysis, the SRA can help generate atrocity risk assessments and warnings necessary for effective preventive action. If you have examples about how you have used the SRA, or questions about how you might, please contact us at ewp@ushmm.org.

To learn more about the factors that contributed to the high-risk estimate of any particular country, visit the country pages on our website and use the “select a country” dropdown in the top right corner.