Chloe Wallace is the founder of OneSourceHR, a trauma-informed HR practitioner, and a female tech founder, as the creator of Scenari, a practice based AI powered workplace conduct platform.
AI is often described as a transformative technology, but transformation is not automatically positive.
The systems we build reflect the assumptions, blind spots and priorities of the people who design them. When those perspectives are narrow, the technology can be too. If we want AI that genuinely supports women’s safety and wellbeing, women need to be involved long before the product reaches the market.
This is not a theoretical concern. UNESCO’s recent Gender and AI report highlights how gendered inequalities are already being reproduced in AI systems. Women are under‑represented in AI development, under‑represented in the datasets used to train models, and under‑represented in the leadership roles that shape how AI is deployed. When the people most affected by gendered harm are not present in the design process, the technology is far less likely to anticipate or prevent it.
We are also seeing the consequences of this gap in real time. Deepfake technology has accelerated faster than regulation, and women and girls are disproportionately targeted. The rise of AI‑generated sexualised deepfakes, harassment, blackmail and stalking and has been well documented. One example is that the Oxford Internet Institute has identified nearly 35,000 publicly downloadable deepfake model variants on one platform alone, downloaded almost 15 million times since late 2022. Of the models targeting identifiable people, 96% targeted women. These are not abstract harms, but real people experiencing real violations.
When general‑purpose AI tools can be manipulated into producing sexualised abuse, the fact that abuse was not the intended purpose offers little comfort. It raises a harder question: who is responsible for anticipating misuse? And how early in the design process should that responsibility begin?
This is something I have had to confront directly. I am the founder of Scenari, a female‑created AI learning platform that allows people to practise difficult workplace conversations through simulation. Our predominant use is sexual harassment prevention, but we also simulate disclosures of domestic abuse, pregnancy loss and other situations where somebody’s response can have a profound impact on the person speaking to them.
When we began developing Scenari, we faced a practical version of the responsible‑AI debate. If we create a fictional employee who discloses sexual harassment, what should happen if a learner deliberately starts abusing her?
A training simulation has to allow people to get things wrong. Learners need to be able to ask an inappropriate question, make a victim‑blaming assumption or respond badly enough for the experience to expose why their response was harmful. If every problematic response is blocked, the simulation becomes artificial and loses much of its educational value.
But there is a difference between allowing someone to make a harmful response so they can learn from it, and giving them an unrestricted generative environment in which they can repeatedly degrade, blame or abuse a simulated victim. We have deliberately closed loopholes within Scenari to prevent that. A learner cannot simply escalate abusive behaviour indefinitely to see what the technology will permit. Those decisions required us to think not only about the intended learner journey, but also about how the product could be used by somebody engaging with it in bad faith.
This is what safety by design should mean. Responsible AI is often discussed through frameworks, acceptable‑use policies and statements of principle. Those things have value, but product development eventually requires specific decisions about what a system will allow. Where could this be exploited? What behaviour could the technology unintentionally reinforce? What happens when somebody pushes against the purpose for which it was created? Where should the technology itself draw a boundary?
Women’s perspectives are essential to answering those questions. Gendered harm is not hypothetical for many women, it is lived experience. When women are involved in designing AI systems, they bring an understanding of how harm manifests, how it escalates, and how it can be prevented. They are more likely to ask different questions, anticipate different risks and challenge assumptions that might otherwise go unexamined.
This is not about creating “women’s AI”. It is about ensuring that the people most affected by gendered misuse are present in the room where decisions are made. Representation is not a diversity metric, it is a safety mechanism.
AI has extraordinary potential for good. It can support learning, improve access to information, and help people practise difficult conversations in ways that were previously impossible. But a worthy purpose does not automatically make a technology responsible. The questions have to extend to how it has been built, the behaviours it enables, the harms its creators have anticipated and the restrictions they have deliberately chosen to impose.
If we want safer AI, women need to shape it before it is built, not after harm has already occurred.




