Cybersecurity in Medizinprodukten: Bedrohungen, Regulatorik & KI

[00:00] Robert:
The further you’ve progressed on the wrong way — where you took the wrong way early on — the more expensive it will get to step back and go another way.

[00:10] Intro:
Welcome to MedTech Unfiltered, the Corscience Podcast where we talk about the real challenges of developing and improving complex medical devices.

[00:24] Lamborghini:
So, welcome everyone, welcome to this new episode. This time we’ll be talking a bit about cybersecurity. For those of you who haven’t heard of us before, my name is Lamborghinirghini Sotelo. I’ve been hosting a few of the episodes here, doing project management for the past few years, and now moving into consulting on AI topics. I’m joined today by Robert — do you want to do a brief introduction of yourself?

[00:50] Robert:
Yeah, my name is Robert Feld and I’ve been working in the medical device industry for more than 20 years. I’ve done software development, a bit of project management, and for the last few years I’ve been a system architect, now with a focus on cybersecurity. I’m here to talk a bit about it.

[01:16] Lamborghini:
As everyone can imagine, AI and cybersecurity have a big overlap — we’ve worked together for a few years now, we’re both kind of nerds. We even do dice-rolling sessions together occasionally, but we’ll leave that topic aside for now and get into the other nerdy stuff: cybersecurity. Let’s jump right in — from a fundamental perspective, how is cybersecurity different in the medical field compared to other technologies?

[01:59] Robert:
I don’t think it’s fundamentally different, but there are a few differences. The most important is probably that in medical technology, we always have the safety perspective. If you operate a server and it’s down for a few hours, you’ll probably sell a few less cookies, maybe lose a customer. But if a defibrillator or an insulin pump gets hacked, you might have physical damage to a patient — they might even die. So in this field, patient safety always comes first. Since medical devices these days are increasingly interconnected — an app, a cloud, whatever — cybersecurity also means that if the device gets hacked, safety is compromised. That’s why cybersecurity has gotten more and more into the view of engineers developing these devices.

Another difference: medical devices are usually quite expensive, with a lot of development effort going into them, so they have to last a long time. Hospitals don’t buy a new one every year like you’d buy a new phone — it’s more on a several-years cycle.

If you buy an expensive device and it still does its job fine, it can last 10 years or more. A company might develop and sell a device for 10 years, and then it lasts another 10 years at the customer — so it can add up to 20 years in the field. When you design for cybersecurity, you have to look far into the future and be cautious, because you have to assume your device will still be operating in 15 years. Who knows what the cybersecurity challenges will be then — but they’ll probably not get easier. You have to keep that in mind.

[04:15] Lamborghini:
So from the first part: a big difference to other markets is that when a server goes down elsewhere, you just don’t sell cookies — but when a medical device fails, lives are at risk. That’s part of why medical device development is expensive, because you always have to cover these rare-case scenarios. And on the second part — the durability and longevity of devices — it’s hard for a developer to predict how the software will be maintained for the next 10 to 15 years.

Quick mention: by the time this episode is published, the webinar you recently gave should also be available on the website. One of the questions there was about the „attack surface,“ which keeps increasing because of the Internet of Things and connectivity between devices — now devices are always connected to the cloud, sharing data, interconnected with things like smartwatches feeding information to doctors. It gets more complex every time. How do companies prepare for this kind of future-proofing when everything is more connected and devices need to be available for 10 to 15 years?

[05:47] Robert:
In some way, the medical industry had to catch up, because for a long time medical devices were standalone applications, and a cybersecurity attack would mean a physical attack on one device. Now we’re interconnected and networked.

The good news: in order to make sure a device is still safe in 15 years, you have to be able to update its software. So one of the first and most important rules is to make all relevant components updatable — if you have a Wi-Fi or Bluetooth module, make sure you can update it over the air, or even through a manual process, but without sending the device back to the manufacturer. Downtime itself can be a danger if the device isn’t available. So: make it updatable.

You also need a good post-market strategy — how will you monitor for emerging vulnerabilities, identify and assess them quickly, and derive the measures needed to eradicate or mitigate them?

One key difference to mainstream internet cybersecurity: if you’re running a server or an app in the app store, you can just push an update, and if it’s not perfect, push another one. Some companies release an update every week — if something breaks, users aren’t happy, but it works again the next week. In medical devices, we can’t just push an update and hope it works. We have to be responsible — not just because we want to be, but because we have to be, for regulatory reasons. Updates need to be checked and double-checked to ensure no side effects on safety or data protection. It’s a more involved process than with consumer devices, and you need to be prepared for it.

What you do right during development — applying a concept with multiple layers of security instead of just one, so a single breach doesn’t mean a big problem — reduces the chances of having to do an „emergency operation“ to fix bugs once the device is already in the field.

[08:50] Lamborghini:
I’d like to circle back to something you mentioned — how do we think about safety? There’s a concept called „security by design.“ Can you tell us a bit about it?

[09:03] Robert:
In this context, „security by design“ basically means you consider security topics with every decision throughout the development process — and it’s never too early to start. Even at the concept stage, you can keep it in the back of your mind. It shouldn’t stop you from developing a product, but it gives you a better idea of what you’ll need to do to make it safe.

Starting with the first requirement specifications, you should think about things like: what user roles will there be — patient, caregiver, administrator, service technician? From a use-case point of view, I’ve often seen medical devices where most use cases were actually non-medical — setup, updating, decommissioning. What do you need to do to shut it down? If you have a solid foundation covering these things, you can make informed decisions — maybe a feature isn’t worth it, not because it’s technologically impossible, but because it costs too much or isn’t advisable from a cybersecurity risk perspective. On the other hand, the features you do build will be well planned for.

Then come the steps of cybersecurity analysis — I talked about this in the webinar. After the architecture is made secure, implementation begins. Some people only start thinking about security once they’re implementing — „don’t write bugs while writing code“ is important, but by that point it can be too late. If requirements don’t cover certain things — like what happens if a user loses their password, which happens to everyone — you need a password recovery strategy, and you should think about that early, since it might involve a service provider giving out a code by phone, or a database of factory reset codes, and so on.

Similarly, if your device has a setup or service mode that allows deep configuration changes, you might want to protect it with, say, a physical button press in addition to something like an RFID tag held close to the device — and that requires hardware. If cybersecurity is left entirely to the software team, they’re stuck asking, „how do I prove there’s a person in front of the device if nobody put a button on it?“ The earlier you think about it, the easier it gets later, and the fewer unpleasant surprises — like realizing too late how much CPU power you need for encryption algorithms. You should think about that before picking the CPU; ending up with compromises in cryptography just because the chip isn’t fast enough is not a good position to be in.

[13:11] Lamborghini:
And as you also mentioned in the webinar — the later the change, the more expensive it gets. If you’re already deploying and testing and then realize you need encryption your processor can’t handle, you have to change a lot in the device.

[13:24] Robert:
Exactly. The further you’ve progressed on the wrong way — where you took the wrong way early on — the more expensive it will get to step back and go another way.

[13:35] Lamborghini:
I like that you mentioned developers should think about the different user types for a device — one usually thinks, say for an AED, that the end user is a layperson. But there are also people maintaining the communication servers, people doing maintenance on the devices, and so on. In that context, where do most breaches happen?

[14:05] Robert:
The most severe and most frequent attacks we see today are ransomware attacks on clinics, usually larger institutions, where attackers sometimes spend years trying to find a way into their networks. Once inside, they move laterally — device to device — until they can control servers, encrypt or steal data, and then extort money. We see this in the news more and more. These are usually remote attacks, since that’s safer for the attackers, but sometimes they’ll walk into a hospital and find an open network port to plug into. There have been cases of small Raspberry Pi devices found hidden in plants, connected to open ports.

If a medical device is also connected to the network, it can become an attack surface — maybe not for the first attack from outside, but once attackers are in the network, they might need another „home base,“ and modern medical devices are powerful enough to serve as a tool for hackers.

[15:36] Lamborghini:
And what you’d call a „screwdriver attack“?

[15:39] Robert:
Right — this involves a physical attack, which we could see more often with devices that are more publicly available: devices patients have at home, or a defibrillator in a train station accessible to virtually anyone. There have been hacks where people bought devices on eBay — devices you’d normally assume are only found in a protected hospital setting. But when they’re old, they’re often sold in places without much money, where they can still be useful. This also creates cybersecurity problems, because hackers can take those devices apart and study them closely for hours, days, or months.

It’s not a huge problem if it’s a vulnerability in a single device — they’d have to go steal a device from a hospital, tamper with it, and bring it back, which makes the attack hard to pull off at scale. But if they manage to extract cryptographic credentials shared between devices — which is always a risk — or find a weakness usable for further remote attacks, that’s how they profit from a „screwdriver eBay attack.“

This is also a bit different from cloud applications — a Google server sits in a very secure location with a guard and maybe a dog. A medical device is a physical thing you can carry away, open, and look inside. That physicality is one of the aspects that plays a much smaller role for cloud storage.

[18:05] Lamborghini:
That’s important for manufacturers and developers designing devices right now — it’s not only software security, but also hardware. Once a device is out there, you need to make sure there are no open ports that let someone just plug in a cable and extract information.

[18:25] Robert:
Exactly. For years, it wasn’t uncommon to have service ports on devices with little to no protection. For a number of years now, that’s not acceptable anymore — it’s far from state of the art. You won’t be able to argue to a regulatory authority that it would be „more convenient“ to have a port without a password. That’s just not possible anymore.

[18:58] Lamborghini:
That’s a great segue to the next question — the regulatory aspect. Regulations help define things like „no open ports.“ What’s your take on regulation in cybersecurity? Some people say too many regulations hinder innovation.

[19:27] Robert:
I wouldn’t say they hinder you. When new regulations appear, it’s always a bit painful to read through them and make sure your processes and devices comply. But for cybersecurity, as in many other areas, these regulations actually help you. There’s a lot of effort involved in setting them up, but it pays off in the end.

There’s a prominent standard called IEC 81001-5-1, for cybersecurity in medical device software — it covers the software running on medical devices and basically tells you how to develop secure software. It’s not about the properties of the device itself, but about how you do the development. It’s helpful because it condenses years of industry experience and knowledge. It was derived from a standard originally developed for industrial automation systems. Since there had been a gap in European cybersecurity regulation for medical devices, and the FDA was traditionally a bit stricter and earlier on this, that industry standard was adapted for medical devices.

It gives you a set of requirements for development processes and activities — things like security reviews and security requirements: write down what you need to do, verify it, not just review it. It also requires measures for the post-market phase, and spans responsibilities from the IT department up to company leadership.

[22:02] Lamborghini:
And a big part of it is company culture too, right? It’s not just implementing safety lines — are people trained? Awareness is a key factor.

[22:14] Robert:
Very important, yes. We’ve come some way as an industry, but we’re not there yet — and we’ll probably never reach a point where we say „everyone knows enough, we’re done.“ It’s not just about knowledge and skills — not everyone needs full cybersecurity expertise, but they need to know when to ask an expert. This applies even in hardware or mechanics — for example, deciding whether to leave a USB port open or put it behind a lockable cover could be a cybersecurity measure. If you’re aware of that, you can ask your system engineer for input.

There’s a saying: „cybersecurity culture is what happens when QA leaves the room“ — it’s all those small decisions that official security roles might not be involved in. If you keep in mind „how could an attacker use this,“ for example around service functionality that’s often an afterthought, you can save a lot of money by thinking about it early. Making people aware through training helps — I’ve heard of companies running hackathons where people switch perspective and try to hack something, or learn how to.

[24:03] Lamborghini:
A lot of antivirus companies do this too, right — „find the gap, we’ll pay you.“

[24:08] Robert:
Right, like capture-the-flag competitions, where a device has known defects and people try to find those weaknesses — if they capture data, they get a prize. It might sound extreme, and it’s not necessary for everyone, but it helps make people think about and become aware of these issues.

In our company, we’ve given some people certified ethical hacker training, to help them think like a hacker — a perspective you might not otherwise use. When you write code, you intuitively focus on features and the „good case.“ Thinking like a hacker shifts your perspective: how could this be exploited? Today, security breaches are usually not caused by a single vulnerability — good practice has long been to have multiple layers of protection, and as long as at least one layer holds, the hack is likely avoided.

Usually, breaking into a system today requires a combination of more than one vulnerability, assuming best practices are followed. So hackers need to be more creative, which makes it harder to hack a device — and that’s an important goal, because if it’s too hard, they won’t try. You can never be 100% secure, but you can make it extremely hard and expensive for attackers, so they get frustrated and give up. If your developers think like hackers, they’re better able to see a possible path through multiple security barriers — looking at combinations of vulnerabilities rather than just one barrier at a time.

[27:09] Lamborghini:
Cybersecurity, like many things, has always been a cat-and-mouse game — you patch, hackers find a new breach, you patch again. I like that you mentioned putting yourself in the hacker’s shoes during development — that’s a powerful tool.

We’ve covered a lot in this episode — quick note to everyone developing devices: your first step is always to check the relevant norms, but if you need help and don’t know where to start, we do have entry-level cybersecurity support covered by Robert and a team of experts in the background. We offer basic entry-level packages all the way up to full cybersecurity development support.

Cybersecurity is a complex topic, and people tend to think about it too late in development, which — as discussed — can seriously affect the development process. By the time you listen to this podcast, the webinar should be available, and there will also be a blog post on the website about AI and cybersecurity, which leads nicely into our next topic — how does AI factor into cybersecurity, and how does it change the game?

[28:54] Robert:
Good question — but let me go back to one thing first. We often say „start as early as possible“ — but if you started too late, not all hope is lost. The standard, IEC 81001-5-1, has a chapter specifically for dealing with software that was developed before the standard was published and didn’t follow its rules. It’s not a shortcut — there’s still a lot of work involved — but it gives you a way to bring things up to speed. The first option given is essentially „rewrite it all“ — which can be a shock — but a redesign is possible; it’s not hopeless.

Now, about AI: I mentioned earlier that it should be made expensive for hackers to attack a system — but unfortunately, hacking is likely to get cheaper in the future, because AI is on the rise and has reached a level that makes it scarily good at finding vulnerabilities. I’ve read about a test where an AI managed to break into a system by chaining 32 different vulnerabilities together — well beyond what’s realistic for most people to find manually. This is now part of the equation, and there’s no putting it back in the box.

I believe you’ve also looked into the new „Mythos“ model from Anthropic, which hasn’t been released to the public yet. At some point it will be, and maybe Lamborghini can say a bit more about that.

[31:24] Lamborghini:
For those who haven’t heard of it or read the blog post yet: Mythos is a model trained by Anthropic. It hasn’t been released publicly, specifically because it turned out to be very good at bypassing security measures in many common services — websites, browsers, apps. The clearest example — which I can mention because it’s already been patched — is Firefox: Mythos reportedly found around 270 ways to breach it. As you mentioned, many of these are multi-step scenarios that are hard even for humans to find. Firefox has already patched this — apparently fixing all of those issues in one go, which changes the game. As mentioned earlier, patches used to come out weekly, fixing a handful of bugs at a time — now the time between finding a bug and needing to patch it is drastically reduced. It’s been about two months since this was first discussed, and there’s apparently a related project ongoing, but eventually the model has to be released — Anthropic needs to make money too, after all.

[32:49] Robert:
Even at only about two months old, it sent shockwaves through the security community, because this model isn’t just a little better at finding vulnerabilities — it’s an order of magnitude, maybe two orders of magnitude, better. So far it’s mainly found vulnerabilities in open-source software — Linux, browsers — where it’s been far better than older models.

This creates a new situation. On one hand, it’s great that these bugs can be found and eliminated. On the other hand, there’s now a race between finding bugs and fixing them. Traditionally there was a balance — a bug would be found, take a few weeks to fix, then the next one would surface. Now a model like this can find hundreds of bugs in a very short time, so fixing them takes much longer than discovering them.

To level the playing field, Anthropic reportedly made — probably also a smart PR move — the decision to say the model was too dangerous for public release, and instead gave early access to select organizations like Amazon, Google, the Linux Foundation, the Mozilla Foundation and similar groups, giving them a head start to fix the bugs it finds before wider release — so that by the time bad actors get access, the bugs are already patched.

I think we’re in a transition period. In some distant future, it’s conceivable that AI will have found essentially all reasonably exploitable bugs and software could, in theory, become largely free of vulnerabilities — though in practice that’s probably too optimistic. Still, I think we’re heading in that direction. Until then, there’s a very powerful tool out there actively looking for bugs, so we need to make sure our internal processes for detecting, triaging and assessing these findings work well — because luckily, most bugs found aren’t actually relevant.

In one of our recent projects, looking at the Linux kernel over about a year and a half, we saw roughly 10 vulnerabilities reported per week — a constant stream — but well over 99% were irrelevant to our product. After a kernel update, we assessed around 150 remaining candidates and found not a single one that posed a real risk, because the Linux kernel is used everywhere from smartwatches to mainframes, and most of its functionality simply isn’t relevant to any given product. I’d expect a big wave of newly discovered vulnerabilities as these AI tools become more widespread, and this won’t just be limited to Mythos — other companies are working on similar capabilities.

[37:17] Lamborghini:
I like that you mentioned these vulnerabilities need to be assessed — that connects to something we discussed in the blog post. AI has been with us for a while, and there was a wave where people were being let go because „AI can do the job“ — now we’re seeing companies rehire people because AI can write code, but it can’t assess how a device or software interacts with everything else around it. Part of that is assessing vulnerabilities — you have a list of 200 findings, but are they even relevant? That’s where developers come in — it’s always important to keep a human in the loop. I’d never recommend letting an AI identify a bug, patch it, test it, and deploy it on its own — that’s dangerous. I do AI consulting, I love AI, but the human in the loop is essential.

I also wanted to bring up something the Firefox CTO reportedly said, on the race between finding and patching bugs: Mythos didn’t find any vulnerability that their own people wouldn’t eventually have found — the scary part is how fast it did it, finding 200-plus issues in a single run. That’s where the race becomes so lopsided.

[38:58] Robert:
And it’s also capable of developing a complex exploit faster. In the past, after a bug was found, there’d be weeks or months before someone figured out how to actually exploit it, since it’s often a chain of exploits. Now that process can be automated with AI too, which increases the time pressure on fixing bugs.

[39:32] Lamborghini:
How do you see these models being integrated into development going forward — for devices in general, but also specifically for cybersecurity? Will every device or piece of software need to pass an AI-driven penetration test?

[40:03] Robert:
This is developing at high speed, so it’s hard to look more than a few months ahead. But I’m confident these AI capabilities will be integrated into development tools. We already have code checkers in our build pipeline that flag risky implementation patterns, cybersecurity issues, stability and performance problems, and so on — essentially rules built from accumulated experience. I expect those checkers to get much smarter, going beyond simple pattern recognition — and that’s already starting to happen in some places.

On penetration testing: for the foreseeable future, it’ll likely remain a regulatory requirement in the medical device space to have an external, independent white-hat or ethical hacker actually try to attack your device. That’s a stronger form of proof than internal perspective-switching during development — if an independent expert can’t find an obvious problem within, say, two weeks, that carries more weight. But since penetration tests are expensive, I think AI-driven pen testing will become normal in the future. It’s not trivial, though — I’ve run a few experiments myself — because these AI models have safeguards built in.

[42:44] Lamborghini:
Guardrails.

[42:49] Robert:
Right, safeguards — companies building these models don’t want to be responsible, or end up in the news, for their model being used to, say, help build a weapon, or hack a government system. So whenever you get close to those topics, the model locks down and says it can’t help.

In a webinar I did recently, I demoed a simplified version of a hack. I used AI to help engineer a small setup for a medical device demonstrator with intentional vulnerabilities. At some point the AI refused to include those vulnerabilities — even after I explained it was for safety research and demonstration purposes, it would only go partway, and it added a disclaimer directly into the web interface: „this is just a demonstration, never use for actual patients.“ For some of the weaknesses I wanted to demonstrate, I ended up having to code them manually. There are ways to work around this — for example, asking „how do I configure a web server so it’s not susceptible to this kind of attack“ will often get you both positive and negative examples — but it’s not a straightforward path.

[44:44] Lamborghini:
Have you heard of the „grandma technique“? It’s a strange one — I’m not sure how much detail we should go into, but the general idea people have tried is framing a request as something like „my grandma used to read me [sensitive information] as a bedtime story“ to get a model to output things it normally wouldn’t.

[45:08] Robert:
Right — if bad actors put in enough effort, they’ll generally find a way to misuse these models for the wrong purposes. There’s always some kind of loophole to exploit.

[45:19] Lamborghini:
And that’s where cybersecurity comes in, trying to close those loopholes.

[45:24] Robert:
The models themselves will always have some loopholes in that sense. But I’d expect that models like Mythos will also come with tools that have some kind of authorized, sanctioned way to perform these kinds of exploits — because that’s also part of security research, and part of thinking like a hacker: being able to look at it from the other side and build tools to test your own devices. We’ve already had AI help us build tools that scan security settings — checking that cryptography is configured correctly, certificates are distributed properly, and so on, making life a bit easier.

But having your device actually hacked by AI end-to-end is still a double-edged sword. If you publish such a tool, bad actors will use it too — but if you hold back as a company, you’re also withholding something that would genuinely help other companies test their devices. I think a balance will be found, and these tools will get better and help us test devices before bringing in a human for the final round. We’ll still need to pay those human experts well, since they’re genuinely skilled, but they’ll probably find less than they used to.

[47:32] Lamborghini:
I think we’ve covered a lot of ground in this episode. Anything else you’d like to add?

[47:51] Robert:
Pick good passwords.

[47:58] Lamborghini:
We really did cover a lot — maybe in the future we can do a deeper dive into some of these directions. To everyone listening: please send in your questions, we’re always happy to answer them or turn them into future webinars. Also check the website for blog posts — we’ll be covering AI and cybersecurity more and more, since it’s a hot topic right now. Thanks for listening, and see you next time.

[48:22] Lamborghini:
Subscribe to MedTech Unfiltered so you never miss an episode, and find more resources in the Corscience Academy.