Legal Limits of Data Input into AI Systems

Transkript

Intro

Okay, to keep with the time, let us commence. Fun time is over, let us talk about legal things. When Florian approached me, he said, oh, the people here would love to hear something coming from a lawyer. That is the exact moment you know someone is lying. But nevertheless I am here, so let us have some fun.

One quick intro to me. Patrick Munro is my name. I am Of Counsel at Planit Legal, and I specialise in classic IT law, AI, cybersecurity and data protection. So I am kind of at home in these fields. I am also a big fan of Claude and Claude Code, which has become my mainstay LLM for all kinds of legal work.

Today we are asking: what kind of data, what kind of inputs can I actually put into LLMs? And what kind of pitfalls are waiting for me when I handle different kinds of data? The issue is that a lot of people have no idea what secrecy requirements, personal-data requirements or sector-specific duties are attached to a piece of data. And the tricky part is that the requirements follow the data. If a piece of data has a specific secrecy requirement, then that requirement travels, and you will see in a bit why that is an issue.

Data protection

Let us start classic: data protection. I know it is old, it is boring, and still very expensive if you get it wrong, so it is useful to talk about for a quick second.

Personal data, as most of you know, is information that makes a person identifiable. What a lot of people do not know is that it often hides in code, in logs, or in screenshots that are shared around the web. These are pitfalls where people think, well, I just took a screenshot, how can this be an issue, and they put it into a consumer-grade LLM. Right then and there you already have unlawful processing, and often a reportable data breach on top, which theoretically also carries a fine.

For our purposes, most companies use a legal basis under Article 6, often legitimate interest if there is no consent available. But one of the most important things for you builders: when it comes to special categories of personal data, health, biometrics, beliefs, by default keep them out of your AI. Unlike standard personal data, you cannot rely on legitimate interest for any of the special categories under Article 9. If you use health data in your LLM, you need one of the narrow exceptions in Article 9(2), and outside of things like employment law or medical treatment that realistically leaves explicit consent. Legitimate interest is not on that list, and getting it wrong carries a fine.

Another thing many companies do not understand: your data processing agreement is not your secrecy obligation. A lot of clients tell me, well, we have a DPA with our processors, everything is fine. It is not. The DPA sets out how personal data is handled. It says nothing about secrecy, nothing about trade secrets. It only shows the regulator that you have a handle on how you process personal data. So there is a huge gap and a huge risk in that direction.

Highly simplified: if the source is public, go for it. If it is internal, make sure your LLM is enterprise tier, and check that EU data residency and the other safeguards are actually contracted, because they do not come with the tier automatically. If it is highly confidential, you should have data protection agreements and no-training clauses, so your inputs are not used to train the next models. And minimise use. Ask yourself: do I really need this entire contract, this entire project dumped into the LLM? Maybe I only need 20 % of this.

The reason I say this is not to make life harder for you. When courts or regulators judge what you did, if you can demonstrate that you took special care not to overdo it and only inputted what you actually needed for work, then the regulators and courts will look very favourably on that. As a rule of thumb, high fines come when people do not take the necessary steps, not because things happen. In our world there are always going to be data breaches and people putting information into LLMs that they should not. But where regulators get mean-spirited and hand out the big fines is when they come to you and your company says, I did not really worry about that, it is fine. That is when they say, get out the checkbook.

Confidentiality and trade secrets

Something probably more relevant to you: confidentiality and trade secrets. Personal-data protection is fairly well known, but for builders and people in IT, the Geschäftsgeheimnisgesetz (trade secrets), professional secrecy and contract confidentiality are what you will encounter most throughout your professional life.

There is a fun little loophole many people do not know: the trade-secret law only applies if you have taken proper measures to protect your trade secrets. I had a client who said, well, there is a trade-secret law, I do not really do anything with it, it is a trade secret, it is economically important, that is enough, right? But if you do not encrypt, protect, and use appropriate measures for your trade secrets, they are not protected. Coming back to LLMs: if you dump your commercially sensitive data into a consumer-tier model without further protection, you are very likely to forfeit your rights under the Trade Secret Act. In that moment, the new IP code you worked on for half a year is suddenly no longer a protected trade secret and you cannot enforce it as one. Dangerous and expensive.

Professional secrecy (§203 StGB)

The next part is professional secrecy. Those of us who are doctors, lawyers or tax advisors fall under professional secrecy laws that also carry criminal offences if we violate them. I know you are all saying, thank God I am not a lawyer, but do not breathe a sigh of relief yet. And of course the classic NDAs: if you sign an NDA agreeing not to put data into an LLM, do not do it. That is pretty cut and dry.

§203 StGB carries punishment, monetary or jail time, for lawyers. But as soon as IT service providers work with a bound professional and have access to that confidential information, they also become bound by this criminal statute. If you provide IT services to a lawyer, evaluating their cases or reviewing their work, and you get into contact with confidential client information, you are inside that criminal statute from that moment, with or without a confidentiality agreement. The agreement is what the lawyer has to put in place; its absence is the lawyer's offence, not your defence. I do not want to say stop working with lawyers, but I do want to say take a step back and think.

§203 also has a necessity test. A lawyer may open up secrets to a service provider as far as that is necessary in order to make use of the service, and §43e BRAO says the same for us. So if you are a service provider, like a hyperscaler, whose service the lawyer actually needs, then there is a good chance the lawyer and the doctor may use you, but you still need that extra secrecy layer attached. There is a big ongoing discussion in the legal community about whether LLMs clear that bar. Currently the view is that they do not for our work; I personally disagree. But if something does not clear it, and the contract you have with the lawyer gets audited, that audit might reveal the lawyer is not in compliance with our professional statutes, which may mean the contract has to be terminated. Luckily that is the lawyer's problem, not yours, although you will not get paid, so that may also be an issue.

How liability stacks up

A small overview of how risk and liability stack up. When we look at AI use, the AI Act, or GDPR compliance, the risk typically sits with the company. If the company you work at makes a mistake, it is their problem. Now, if you are a leader, the next two get interesting. In financial services, where I originate from, you have the Digital Operational Resilience Act (DORA), and, as many of you have heard, NIS2, in Germany for non-financial cybersecurity. Here we have the entity and the management board, and for NIS2 or the BSIG in Germany there is even explicit liability: if you are responsible as part of the management board and you do not take care of proper data-access management, for example for your LLMs, you have personal liability towards your company.

The harshest, as we said, is if you provide services to a lawyer, are bound by §203 StGB secrecy, and violate it. That is always on you personally. You cannot step back and say, well, I was doing my job, so this is the company's issue. It is not, it is a criminal matter, and it becomes your issue.

Classified public-sector data

One important thing on LLM and AI data access: some of you have worked with public contracts and public authorities. Sometimes they have the classification Verschlusssache nur für den Dienstgebrauch, VS-NfD. That kind of data should never go into any LLM. You may only move it electronically, email included, through an encryption product that the BSI has approved for that purpose, and no LLM is on that list. If you handle that kind of data, please do not put it into any LLM at any point in time.

For DORA and the like, the pattern is clear: the more regulated you are, the more risk you face. A quick AI Act check-in: the omnibus is probably going to be passed, so high-risk deployer obligations will come in 2027. I do not think this is anything new for you. I believe in the good of people.

Export control and sovereignty

We had a glorious few days of the latest model release eliminating my weekly usage limits, which was beautiful. But we also had a brush with US export-control laws, and a wake-up call as to how important access to sovereign European AI is. As a huge Anthropic and Claude fan, I can promise you this is legally very difficult and could fill another three hours. Essentially, the US administration classified two frontier models, Claude Fable 5 and Claude Mythos 5, as too dangerous to be had by foreign nationals and ordered Anthropic to stop serving them. Anthropic had no reliable way to check nationality in real time, so it shut both models off completely, which in my opinion is the right thing to do. The government's explanation of what the concrete issue was stayed rather thin at the time. The risk is: if you rely on a tool and from one day to the next lose access, it could disrupt your entire supply chain. We should all be prepared to have alternatives ready. Easier said than done.

Sovereignty is a huge buzzword, and many of you have heard of the Cloud Act. A quick wake-up call: the Cloud Act is extraterritorial. If a US provider holds data in Europe through a subsidiary it controls, that provider can be compelled to surrender it, wherever the servers stand. So for true sovereignty and true data protection, you need a provider completely distinct from the US, which, as we all know, is difficult to have today.

The point is: whether my data can lawfully go into a tool is only one of the gates. The second gate is whether we can actually rely on that tool going forward. Regulated entities especially need their operations to stay stable. A banking or fintech provider could be criticised by the regulator for hedging all its bets on an American model that could be shut off tomorrow. The regulator will say, there are European options, why did you not use those?

The new EU e-evidence regulation

Before I close, one more thing I promised: the new European e-evidence regulation applies from 18 August 2026, and it is interesting for most of us. It now allows law enforcement from any EU country except Denmark to directly approach and subpoena IT service providers. Communication services are the obvious case, but it also catches hosting and storage services where storing the data is a defining part of what you sell. If you have a platform you sell to a company that enables communication between different users, that is already in scope. Normally France would ask a German court, the court would talk to the police, and then they would come to you. Under this regulation, the French authorities can approach you directly.

The reason so many are not ready, and why it is dangerous, is that it also carries penalties of up to 2 % of annual worldwide turnover. In the most extreme cases you need to respond within eight hours, typically within ten days, and almost nobody is ready for that response time. One hot tip: in the regulation itself, at the bottom, there are two official templates. The regulation lets law enforcement order you either to hand over data or to freeze data to be handed over later, and the official request templates are in the text, as Annex I and Annex II. If you are subject to this regulation, look at these templates now, see what the authorities can request, and do an internal check on whether you could respond when a request comes flying onto your desk.

Closing

Closing out: classify, and think before you input. Have written rules for how you handle data; it is almost more important than what you actually do with the data. That is not entirely true, but you know what I mean. Take the time in your companies, speak with your data protection officers and your CISOs, and make sure there is a policy in place so that everybody knows how they are allowed to use data, because only if people know how they are allowed to use data can they use it correctly. Thank you very much.