Greifswald computer science professor warns against AI use in German asylum procedures
Editorial responsibility
- No named human review is recorded for this page.
- Source reporting is collected, normalized, translated or condensed automatically when needed.
- Automatically published source-backed update
In an interview with taz, computer science professor Alexander Steen of the University of Greifswald criticizes a German government draft law that would deploy AI systems in asylum procedures. Steen argues the bill gives authorities a blank check for unreflective technology use, lacking a unified system architecture, mandated usage processes, or training for decision-makers. He warns of automation bias, where officials may uncritically approve AI recommendations even when systems flag errors. Steen also cautions that AI could reproduce statistically biased correlations—such as linking certain nationalities to forged documents—leading to prejudiced decisions against individuals. He says verifying personal data online via AI is about as reliable as randomly asking people on the street, citing risks of misinterpretation and data protection problems that could expose asylum authorities' data flows to repressive regimes. Steen concludes the risks likely outweigh benefits and calls for precise rules keeping humans involved throughout all sub-processes.
Source report
The German federal government plans to use AI in asylum procedures. Computer science professor Alexander Steen sees few benefits—and many risks.
There is a risk of statistical bias and automation bias: AI programs are set to be used in asylum procedures in the future. Photo: Jochen Eckel/imago
Interview by Frederik Eikmanns
taz: Professor Steen, the federal government wants to deploy AI programs in asylum procedures. A corresponding draft law is currently being debated in the Bundestag. Is this a good idea?
Alexander Steen: It is a good idea to speed up procedures and digitalize them sensibly. But what is in the draft law seems poorly thought out and gives the authorities involved a blank check for the unreflective use of technology.
About the interviewee: Alexander Steen, born in 1990, is a professor of computer science at the University of Greifswald and spokesperson for the AI division of the German Informatics Society (Gesellschaft für Informatik).
taz: What exactly bothers you?
Steen: Even the choice of words is remarkable. An important area of application is described in the draft as "monitoring." That is probably more of a euphemism. Monitoring implies that it is about passive observation in relation to asylum decisions. What is actually meant, however, appears to be an active intervention in asylum procedures.
taz: Specifically, the draft law provides that AI should analyze completed asylum procedures and make the results available to employees of the Federal Office for Migration and Refugees (Bamf). What is so bad about that?
Steen: People sometimes trust recommendations from automated systems more than their own judgment. The technical term for this phenomenon is "automation bias." Scientific studies have also demonstrated this for human-AI interactions. Even if humans are still formally the ones deciding on asylum applications at the end, a situation threatens in which AI recommendations are increasingly rubber-stamped without being critically questioned.
taz: Surely this can be counteracted?
Steen: Automation bias can be observed even when AI systems explicitly point out possible sources of error or provide explanations when outputting data. The draft law provides no protective mechanisms whatsoever in this regard.
taz: Where do you read that from?
Steen: No uniform architecture for such systems is established, no usage process is prescribed. Each authority is to decide autonomously how it uses AI. The draft also ignores the costs of necessary training for the decision-makers who are supposed to interact with the systems.
taz: As long as the data the AI outputs is correct, it's not that bad, is it?
Steen: Even if mere facts should be correct, that does not mean that all details of a case are appreciated and weighed. AI models could recognize and reproduce statistical correlations that have no causal basis without anyone noticing. People attribute an objectivity to automatically collected data that it does not have to that extent.
taz: How should one imagine this in practice?
Steen: One example would be AI detecting a correlation in the files showing that forged documents were submitted at an above-average rate by a particular group of people.
taz: So, for example, persons of a particular nationality, region, or ethnic group.
Steen: If someone from such a group submits an application and the AI system points to the correlation in the data, then suspicion would naturally immediately fall on that person. In doubt, this could lead to decision-makers no longer examining the application objectively, consciously or unconsciously. But such a calculated correlation is not a probability that says anything about an individual.
taz: But such data isn't entirely meaningless either, is it?
Steen: AI systems make mistakes. And datasets can exhibit statistical biases from which irrelevant or false correlations can be derived. Another factor is that AI systems are trained on data that may itself already be tendentious. Detecting and preventing this is expensive and laborious.
taz: The draft law is also supposed to allow authorities to automatically verify questionable information that asylum seekers provide during proceedings on the internet. At least that makes sense, doesn't it?
Steen: Even for humans, the following applies: trying to verify personal data on the internet is about as reliable as randomly asking people on the street about it. It was already easy 20 years ago to fake photos, texts, and profiles online. With today's technical possibilities, it is much easier.
taz: Isn't that precisely an argument for using AI?
Steen: If someone takes photos from my website and uses them to create a fake Facebook profile of me, then an AI system cannot reliably detect that it isn't me behind it. I could have created a profile myself. Moreover, there is a risk with AI systems that they misinterpret data. Recently, there was the case of a chief physician from the Rhineland whom Google's AI mode falsely accused of being a sex offender. The program had mixed up the chief physician's data with that of another employee at the clinic.
taz: Surely something like that would be noticed?
Steen: Much smaller errors in interpreting data are enough. If the system falsely claims that I went on vacation to my home country, that alone can sometimes jeopardize my asylum application. Not to mention the data protection problems.
taz: What do you mean?
Steen: AI systems use search engines, just like people do. The data streams that go to the search engine operators in the process allow conclusions to be drawn that they originate from the asylum authorities. And that in turn makes the data interesting, for example, for repressive regimes that want to persecute people even after they have fled to Germany.
taz: AI programs must have some benefit in asylum procedures, surely.
Steen: In doubt, the risk is greater than the benefit. In any case, precise specifications are needed for how and for what purpose they are to be used. The goal must not be for machines to do the work and for humans to only come in at the very end of the process and then make a decision. Humans must be involved in all sub-processes and therefore forced to engage substantively with the data and the decision.
taz: Is that realistic?
Steen: Currently, I see little potential there, but I also don't know the specific specialized procedures of the asylum administration. But digitalization doesn't always have to mean "AI." What matters is finding sensible and safe optimization possibilities and implementing them.
taz: Can AI programs at least be used sensibly in other authorities?
Steen: Where vulnerable groups of people are concerned, the unreflective use of AI can be very dangerous. But there are also areas where AI systems can be used sensibly. Especially where work follows clear schemas and is less about discretionary decisions. "AI" doesn't necessarily have to mean statistical learning methods, which are currently so much talked about. Fixed, logical instructions for action can also be used. Current research particularly addresses the combination of both approaches. But even there, the bottom line is: humans must remain involved.
Related articles:
- Common European Asylum System: The Reform Has Failed — Commentary by Frederik Eikmanns
- Legislative Proposal in the Bundestag: AI to Indirectly Co-Decide in Asylum Procedures — By Frederik Eikmanns
- Digital Assistants at BAMF: Software to Recognize Dialects — By Jonas Schönfelder
Source
taz.de - taz.deNeutral / independent