Human in the Loop: Why Inclusive AI Must Keep People with Disabilities at the Centre
Artificial intelligence is increasingly being used to support communication, mobility, education, employment, healthcare and independent living. For people with disabilities, these developments can create significant opportunities. AI can describe images, generate accessible text, interpret speech, predict words, control devices, personalise learning and help people navigate complex information.
However, AI systems also make mistakes. They can misunderstand speech, misinterpret behaviour, reproduce bias, provide inaccurate information or make decisions based on incomplete data. When these systems affect disabled people, the consequences may be more than inconvenient. They can restrict access, reduce independence, reinforce discrimination or place someone at risk.
This is why the concept of “human in the loop” is so important.
Human in the loop means that people remain actively involved in the design, operation, review and oversight of an AI system. AI may analyse information, suggest an action or automate part of a process, but it does not operate without meaningful human judgement and accountability.
For inclusive AI, the human in the loop must include people with disabilities themselves.
More Than Human Supervision
Human in the loop is sometimes understood narrowly as having a staff member check an AI system’s decisions. This may be necessary, but it is not enough.
Inclusive human involvement should begin before the technology is developed. Disabled people should participate in identifying the problem, defining what success means, testing prototypes, identifying risks and deciding when human intervention is required.
This is fundamentally different from designing a system first and asking disabled people to test it at the end.
A genuinely inclusive approach recognises that disabled people are not simply users or beneficiaries. They are experts in their own lives, access requirements and experiences of exclusion. Their knowledge can reveal problems that may not be visible to developers, policymakers or service providers.
For example, an AI-powered recruitment tool may appear efficient because it can screen hundreds of applications quickly. However, disabled applicants may identify that the system penalises employment gaps, non-standard career paths, alternative communication styles or assistive technology use. Without their involvement, these patterns may remain hidden.
Why AI Cannot Always Understand Context
AI systems are built by identifying patterns in data. They do not necessarily understand the social, cultural or personal context behind those patterns.
Disability is particularly difficult to represent through data alone. Two people with the same diagnosis may have very different abilities, preferences, communication methods and support requirements. A person’s access needs may also change according to fatigue, pain, environment, stress or the nature of a particular task.
An automated system may interpret difference as error or risk.
Speech recognition may struggle with dysarthric speech. Facial analysis may misinterpret the expressions of someone with facial paralysis. Behaviour-monitoring software may classify autistic movement as inattention. A healthcare algorithm may underestimate the quality of life of a disabled person because its training data reflects negative assumptions about disability.
A human in the loop can consider context, question the system’s assumptions and recognise when the output does not reflect the person’s actual circumstances.
However, that human must also be appropriately trained. Human oversight is not effective if the reviewer simply accepts the AI recommendation or holds the same biases embedded in the system.
Supporting Choice Without Removing Control
One of the most valuable roles of AI is to support decision-making. It can organise information, offer predictions, suggest communication, identify options or reduce repetitive tasks.
But support should not become control.
For a person using augmentative and alternative communication, AI may predict words or generate possible sentences. This can make communication faster, particularly for people who type slowly or use eye gaze, switch access or other alternative controls.
Yet the system must not begin speaking on the person’s behalf without clear approval. A suggested sentence may be grammatically correct but fail to reflect the person’s intended meaning, humour, identity or emotional tone.
The user should be able to accept, reject, edit or ignore the AI’s suggestions. They should also be able to understand why a particular suggestion has been made and adjust how the system works.
The human in the loop should therefore be the disabled person wherever possible. The goal is not merely to have a professional supervising the technology. It is to ensure that the individual retains authority over decisions affecting their own life.
Human Oversight in High-Stakes Decisions
The need for human involvement becomes even more important when AI is used in areas such as healthcare, social protection, education, employment, finance or access to public services.
An AI system may help assess eligibility for support, identify potential health risks or recommend educational interventions. These tools can assist professionals, but they should not become unchallengeable decision-makers.
People must have the right to know when AI has influenced a decision. They should be able to request an explanation, correct inaccurate information and appeal to a person with the authority to change the outcome.
Get David Banes’s stories in your inbox
Join Medium for free to get updates from this writer.
Human oversight should be accessible. An appeals process is not genuinely inclusive if it is only available through complex written forms, inaccessible websites or telephone systems that exclude people with communication disabilities.
The process may need to provide Easy Read information, sign language, captioning, text-based communication, advocacy support or additional time.
A human in the loop is meaningful only when the person affected can reach that human and be heard.
Avoiding Over-Reliance on Automation
One risk of AI is automation bias: the tendency to trust a computer-generated recommendation because it appears objective or scientific.
Staff may assume that an AI system is more accurate than their own judgement. Organisations may rely on automation to reduce costs. Over time, human review can become a formality rather than a meaningful safeguard.
Inclusive AI requires organisations to define clearly when an automated decision must be reviewed. This may include situations where the system has low confidence, where the user disputes the outcome, where the consequences are significant or where disability-related factors may affect accuracy.
Organisations should also monitor whether reviewers are regularly overturning particular types of recommendation. This may indicate that the system is poorly designed or systematically disadvantaging certain groups.
The purpose of human involvement is not to protect a flawed system indefinitely. It is also to generate evidence for improvement.
Co-Design as a Continuous Process
Human in the loop should continue throughout the full life of an AI system.
Disabled people should be involved in initial design, accessibility testing, implementation reviews and ongoing evaluation. Their feedback should influence updates, policies and decisions about whether the system should continue to be used.
This is important because AI systems can change after deployment. Data changes, user behaviour changes and new risks emerge. A system that appeared accessible during a controlled pilot may create barriers in real-world settings.
Organisations should establish accessible ways for people to report errors, challenge harmful outputs and suggest improvements. They should also compensate disabled people fairly for their expertise rather than relying on unpaid consultation.
Participation must be diverse. Disability is not a single experience. Design processes should include people with physical, sensory, cognitive, communication, psychosocial and multiple disabilities, including those who may need support to participate.
Preserving Accountability
Human in the loop is ultimately about accountability.
When an AI system causes harm, responsibility cannot be transferred to the algorithm. Developers, purchasers, service providers and public authorities remain responsible for how the system is designed and used.
A human-centred approach asks important questions:
Who can stop the system?
Who reviews its decisions?
Who is responsible when it fails?
Can the person affected challenge the outcome?
Are disabled people involved in deciding how the system operates?
These questions should be answered before deployment, not after harm occurs.
Keeping Inclusion at the Centre
AI can support independence and participation, but only when it respects human rights, choice and diversity.
Human in the loop does not mean resisting automation or requiring a person to approve every minor action. It means designing the right balance between automation and human judgement. Routine tasks may be automated, while significant decisions remain open to review and challenge.
Most importantly, disabled people must not be positioned outside the loop while professionals and developers make decisions on their behalf.
Inclusive AI should be designed with people with disabilities, controlled by them wherever possible and accountable to them when things go wrong.
The future of accessible technology should not be based on removing humans from the process. It should be based on using AI to strengthen human agency, relationships, expertise and choice.
