Paul Murphy on Irish role, AI harms and 'hallucinations' in law

And thanks a lot to all the speakers. Maybe just to follow up with that with Dr. Jakubowska, how is the Irish government seen at a European level? Is it seen as being one of those that is closer to big tech and big tech lobbying? So is it seen as pushing back against this deregulation drive or seen as being more in line with the deregulation drive? To the best of my knowledge, the Irish government is not one of the governments that has been most aggressively pushing this agenda at EU level. We've really seen more governments of Germany and France, for example, being some of the strongest champions of the deregulation agenda. And of course, we also understand that with how incredibly fast the process is moving, many countries do not yet have stable positions on the file. I think speaking more broadly, we see a lot of appetite for better enforcement, for big tech accountability coming from the Irish DPC. But I think also given Ireland's outsized role in the enforcement of the EU rulebook, there's always space for improvement, for increased resourcing to ensure that that job that Ireland is actually having to shoulder on behalf of the whole union, that the regulators are properly equipped. And in this climate where regulation is being undermined, where enforcement agencies and authorities, and of course, civil society, academics, and other independent experts are having funding cut and resources taken away, that closes the civic space and makes it even harder for us to get accountability on these topics. So I think there's always room for improvement there. And then maybe to go into some more detail with Dr. Dominguez in relation to what they're trying to do with the digital omnibus. Effectively, we could be moving towards more fully automated decisions by AI. Have we evidence about what the likely impact of that would be on different groups in society? Thanks so much for the question. Yes, the article that has to do with automated decision making is one of the strongest when it comes to GDPR especially. And it's one that the AI Act did not achieve everything that civil society wanted from it. Actually, we're seeing more appetite even to weaken the AI Act. All of these changes are not going to go very technical into it, but indicate a green line for companies to use more automated decision making. I think you've seen many, many examples across Europe of how, you know, this has, this can have incredibly negative impacts. And as Dr. Berhan said, this bears the bronze, especially when it comes to marginalized communities, when it comes to welfare, when it comes to, as she said, credit scoring, when it comes to do with their survival, and it comes to the use of AI, right? Because AI is also to make a decision maker, no matter how we want, how you want to call it. And what we're seeing is this indication that instead of going from a prohibition that had certain exceptions to allowing this activity with under certain criteria, especially, you know, proving that there's a necessity in order to do that, and also undermining some of the safeguards, right? You know, we have, for example, human intervention, meaningfully human intervention as one of the most important safeguards, but you have other safeguards, right? For example, the right of the subject to know what data is in the system and what data is being used in order to actually make this decision, the right of any person to be explained about how this automated decision making works, not just afterwards when you've asked the company, but before, during the process and even before the decision is made. So I think it's really important to understand that this actual attempt at undermining Article 22 of GDPR, which is automated decision making, is clearly also following the line of allowing for more AI, for more data into this AI, and allowing companies to use automated decision making without any kind of safeguards, without any kind of qualms, I have to say, and without any kind of regard for the human rights impact that these decisions are going to have on the whole of society, and also very specifically on some collectives. Thanks a lot. Just to turn to Professor Binns briefly, I mean, I thought, like in a way your main point is a very simple point, but I have to say I hadn't really thought of it as someone who is certainly one of the most critical members of this committee on AI. The basic point that hallucinations aren't kind of a bug, the whole thing is a hallucination, is a very strong point. It's just predicting the next word. Sometimes that coincides with reality, but often it doesn't, and it cannot be eliminated because the nature of the whole thing is just to predict the next word. It's a very, it's got a simple but profound point, and it has like very widespread and deep implications in terms of our ability as a society to debate with each other, to make democratic decisions, and so on. Could you explain like the impact of that, and the like the relationship to a post-truth world effectively, and like if we could, would it be better to ban AI in many circumstances? Sure, yeah. Maybe a good example here is the legal system. So there are, there's a website called AI Hallucination Tracker, which tracks cases where files have been entered into the docket in court systems around the world that have been subsequently found by judges to be AI hallucinations, and the consequences that have been felt for the lawyers who've done that. I think that's a really good example, because the legal system depends on not having to constantly check every single thing. You have to take a certain amount of faith. But if you allow hallucinated cases into the records, you're undermining the the faith that people place in that system. And there's some of the responses from lawyers who've been caught doing this have been, well, okay, I've got the citation wrong, but the gist of the argument is correct. Well, that's not good enough. That's not how the law is supposed to work, right? It's supposed to be, you build on real argumentative processes that are going on in people's minds and being communicated to each other and being accepted as valid. And if you break that connection, I think you break the whole purpose of the legal system. So that's just one area where I think there should be heavy punishments for anyone who's engaging in that. But I think the same goes in lots of other contexts as well. As an academic, I'm dealing with students that are potentially using these systems. And so that also undermines the integrity of the learning process, the education process. So yeah, I think there are definitely cases where we should be banning use of these systems. Thanks, Professor.