Abstract
In recent years, the use of automated systems and artificial intelligence in emergency management has expanded rapidly, offering tools for faster decision-making, predictive modelling, and real-time data analysis. However, this technological shift also raises urgent ethical and social concerns. Automated systems are not neutral; they are shaped by the data they are trained on and the assumptions embedded in their design. As such, they risk reproducing and amplifying existing inequalities, especially when applied to vulnerable populations such as migrants, the elderly, or persons with disabilities.
This article critically explores the intersection of algorithmic bias, equity, and ethics in emergency management. It argues that the integration of AI must be accompanied by transparent governance frameworks, participatory design processes, and ongoing ethical audits. Drawing from real-world examples and recent emergencies, the paper highlights how flawed data or opaque decision-making can lead to exclusionary outcomes, even when technological tools are deployed with good intentions.
The article concludes by outlining a set of concrete recommendations for building an ethical governance model for automation in crisis response. These include enhancing transparency, fostering inclusive participation, and institutionalizing critical oversight mechanisms. Ensuring equity in emergency technologies is not merely a technical challenge—it is a societal imperative.
Over the past twenty years, the way we address emergencies has changed radically. The increasing availability of real-time data, the spread of predictive systems, and the integration of artificial intelligence in crisis management have transformed the operational machinery of civil protection. Today, thanks to automated dashboards, decision-support algorithms, and intelligent surveillance tools, it is possible to anticipate scenarios, distribute resources more efficiently, and, in some cases, save lives that would previously have been lost.
This transformation is, in all respects, one of the most significant in contemporary emergency management.
Yet, this technological push is matched by an increasingly evident ethical urgency, which is too often underestimated. If the algorithms that guide critical decisions in emergencies are built on partial data, if they reflect structural biases, or if they operate opaquely, the risk is that rather than reducing social divides, they will worsen them. The binary logic of machines is ill-suited to the complexity of human vulnerabilities. Technologies are not neutral: they incorporate worldviews, operational priorities, and implicit assumptions about what matters and who matters. And in high-pressure situations like a flood, an earthquake,
or a pandemic, these choices translate into immediate, sometimes irreversible, material consequences.
Consider, for example, the predictive models used to determine who to evacuate first, or which neighbourhoods to police during a power outage. If the historical data on which they are based is lacking, or if the risk parameters do not consider cultural or socioeconomic specificities, entire communities can end up excluded from decisions, invisible to the machine. This often happens with irregular migrants, isolated elderly people, people who do not speak the country’s official language or who lack access to digital technologies. All of these individuals not only risk being forgotten, but are also often hyper-surveyed in the name of public safety, in a paradox that mixes invisibility and control.
In this article, we therefore address a central question for the future of emergency management: how can we ensure that automation operates in compliance with the principles of equity, justice, and inclusion? The issue isn’t technical, but political and cultural. It doesn’t just concern data engineers or software designers, but everyone working in the civil protection system, from policymakers to volunteers, from public officials to affected communities.
We offer a critical reflection on these issues, accompanied by examples and guidelines, in the belief that technological innovation can (and should) be an ally of social justice, not its enemy.
Algorithmic Bias and Data Blindness
The idea that a machine can “decide better” than a human is one of the most widespread—and most dangerous—narratives when it comes to artificial intelligence in emergency management. Automation is often presented as a guarantee of objectivity, neutrality, and impartiality. But in reality, algorithms are nothing more than extensions of the logic and data on which they are trained. And data, no matter how abundant, is always partial, always selective, often distorted by structural gaps or systemic errors. The right question to ask is not “what does the data tell us?” but “who is included in the data, who is left out, and why?”
A first level of bias arises precisely from the unequal representation of populations in datasets. People living in precarious situations, outside of official channels, who speak different languages, or who don’t use digital tools are often not registered and leave no trace in information flows. This applies to irregular migrants, the homeless, people with cognitive disabilities, but also to many elderly people living alone. It’s not just a matter of “statistical absence”: in predictive systems, what isn’t in the data simply doesn’t exist. And therefore, it receives no attention, resources, or protection.
A second level concerns the way models interpret and classify risk. Algorithms learn from past events, but past events are never neutral. If a certain urban area lacked significant interventions during previous emergencies, the system could “learn” that that area is less vulnerable. But perhaps interventions were lacking there precisely because those people didn’t know how to ask for help, or because they were too marginalized to receive it. Thus, the lack of assistance confirms the “non-priority,” and the vicious cycle repeats itself.
A prime example of this phenomenon was seen during the COVID-19 pandemic, when several automated triage systems in emergency rooms or community services assigned lower priority scores to elderly patients or those with comorbidities, based on purely probabilistic survival models. From an efficiency perspective, the logic was clear. But from an ethical perspective, it raised dramatic questions: who decides what is more valuable in a human life? Who draws the line between efficiency and discrimination?
Finally, the risk of biases embedded in intelligent surveillance systems, such as those based on cameras, facial recognition, or social media analytics, cannot be ignored. These tools, often used to monitor “abnormal” behaviour or predict dangerous situations, tend to reflect implicit biases about what is “normal” and who is “at risk.” In some cases, entire ethnic groups or working-class neighbourhoods are subjected to more intense surveillance not for objective reasons, but because they have historically been associated with greater “attention” from law enforcement. It’s easy to imagine how, in an emergency, this could translate into selective checks, forced evacuations, or even the failure to activate alarms in certain areas.
The key point is that no algorithm is neutral. Every model reflects a worldview, whether explicit or implicit. This is why it’s essential that the design and use of technologies in emergencies be accompanied by tools for critical control, transparency, and accountability. Only in this way can we prevent protection from becoming exclusion, and innovation from turning into a new form of injustice.
Toward an Ethical Governance of Emergency Automation
If we accept that technology is not neutral, and that its choices can have material consequences on real lives, then simply “using algorithms better” is not enough: we need a framework of collective responsibility. We need ethical governance. Not as a document of good intentions, but as a daily practice, shared among public bodies, technicians, operators, and citizens. In particular, in emergency management, this governance should be based on three fundamental pillars: transparency, participation, and critical oversight.
Transparency is not just about publishing codes or datasets, which often remain inaccessible for security reasons or technical complexity. Rather, it is a transparency of process: knowing who built a system, on which data, and for what purposes. In emergencies, where decisions must be made quickly, it is essential that those making them know how the tools they rely on work. It is not enough to “trust the machine”: we must understand it, question it, and question it. This is even more true for public decision-makers, who have a duty to be accountable for the choices they make, even when they are mediated by an algorithm.
Participation is perhaps the most undervalued and most urgent dimension. Communities experiencing risk first-hand must also have a voice in the design of the tools that are supposed to protect them. This doesn’t mean turning every citizen into an AI expert, but rather building spaces where local experience, situated knowledge, and everyday fragilities can be incorporated into models, even indirectly. Consider, for example, the possibility of co-designing personalized alert systems with people with disabilities, or involving migrant associations in defining logistical priorities during an evacuation. Only in this way can we prevent technology from becoming an abstract and distant filter, and instead transform it into a tool for empowerment.
Finally, critical surveillance requires that automated systems be subjected to continuous verification, even after their implementation. Too often, we limit ourselves to validating models once, at the outset, as if they were stable and definitive structures. In reality, contexts change, populations evolve, and even algorithms “learn” over time—sometimes in unexpected directions. This requires a permanent ethical audit function: an independent, cross-sectoral review involving various figures—technical, social, and legal—capable of assessing whether a system is producing undesirable effects, excluding individuals, or reinforcing inequalities.
Ethical governance is therefore not a utopia, but a concrete necessity. It is a prerequisite for any innovation operating in the emergency field, where lives and vulnerabilities are affected every day. And it’s not just about avoiding harm: a fair technological system is also a more effective, more stable system, better able to generate trust. In an era where crises are increasingly frequent, complex, and interconnected, this trust is the most precious capital that institutions can build.
For Fair Technology in Emergencies
Automation is neither good nor bad in itself: it is an accelerator. It accelerates the development of what already exists in the system. If the system is inclusive, so will the technology. But if it is built on inequalities, structural blindness, or implicit hierarchies of human value, then automated systems will only amplify these imbalances, making them less visible but more pervasive.
In emergency management, this risk is particularly acute. At stake are not only abstract rights, but concrete choices about who receives help, when, and with what tools. This is why a paradigm shift is needed today: we can no longer think of technology as a simple technical “add-on” to the emergency system. It is a political dimension, which must be addressed with the same care, responsibility, and transparency with which an evacuation plan, a fund distribution, or a relief strategy are decided.
The operational recommendations that emerge from this reflection are simple, but not trivial. First: build transparent systems, not just in their codes, but in their processes and responsibilities. Second: actively involve vulnerable communities in designing solutions, starting from the assumption that those who experience risk know dimensions that models cannot predict. Third: establish permanent ethical audit mechanisms capable of assessing the real impact of technologies, not just their technical precision. Lastly, but perhaps most importantly: train those working in emergency management to critically read technological tools, not as oracles, but as tools to be questioned and, if necessary, challenged.
Emergency management of the future will inevitably be increasingly technological. But that doesn’t mean it has to be increasingly inhumane. It’s up to us to decide in which direction to make it evolve. The challenge is open, and it concerns everyone: technologists, decision-makers, and citizens. Because protection, if it’s not equitable, isn’t protection at all. It’s selection.
And this is a choice no algorithm can make for us.
Manuel Carta
Disaster Management Consultant
#civilprotection #disastermanagement #mctraining #esculapiani #manuelcarta #AI #populations
