This essentially boils down to two scenarios:
- You set up a nice, compliant QMS, which describes your software development process. You then document your software and create lots of documentation while you do it. Everyone is happy and high-fives each other for developing software in a perfectly compliant way.
- Or: You prioritize building your software first, because you know that building the right software on schedule is hard enough already. Once your software is mostly done, you document everything retrospectively. This will be a bit painful. Going forward, after the software is certified as a medical device, you document everything during development.
The benefits of approach #1 are that everything is compliant. That’s about it. The drawbacks are that you will most likely never finish developing your software, or your software development will be so slow that 1) your software will be crappy or 2) you develop the wrong software because you didn’t have time to iterate.
The benefits of approach #2 are that you actually finish developing your software. The drawback is that you’ll have to document everything retrospectively for your first release, which will be painful. But is it that bad? Let’s look at a few aspects of retrospective documentation.
Retrospective Documentation: What Needs to Be Done?
Greatly simplified as always, these are the main documents you’d need to create during or after software development:
- Software Requirements: Basically a software specification. Note that this is not just an export of GitHub issues – more like a list of features.
- Software Tests: Have at least one test which covers each software requirement. Can be a manual test.
- SOUP List: List of your third-party libraries.
- Risk Analysis: A table of things which can go wrong in your software and what would happen to your patient.
- Usability Tests: Test your final software with at least 5 users and document it.
If left out lots of small things which probably add up to another big chunk of work. But you get the idea.
The interesting observation here is that most of these documents are easier to create once the software is done anyway – looking at Software Requirements, the fact that they should be written as “absolute” specification makes it near-impossible to create them while your software is undergoing development. You’d at least need to know what the final software would look like. Same for the risk analysis and usability tests.
Retrospective Documentation: Is It Slower?
Yes, of course you’d be slower when documenting your software retrospectively. You’ll be wading through your software while writing your software requirements. But! You’d also be doing all your regulatory documentation in one huge time chunk – say, the last three weeks after your software is done. That’s a huge efficiency boost because your regulatory work is not fragmented by other activities (like, developing and changing the software). So, taking all factors into account, I don’t think there’d be much of a difference.
Now you could say, “but if we have a fancy QMS and all software documentation and tests are automated, we only have to set them up once”. True. But I’ve never seen that type of QMS. Maybe you can create it and sell it to me. For all other companies on this planet, there’s always lots of manual documentation work involved.
Retrospective Documentation: Is It Accepted By Auditors?
In general, yes – for your initial product certification. The idea is that auditors are also humans, and therefore know that more or less every medical device company on this planet didn’t start out as a medical device company. It usually started out as a bunch of dudes with a crappy code base who suddenly noticed that it could be a medical device.
Accordingly, the focus during an initial product and company audit is whether the company has understood all requirements and whether the documentation is complete. Like, do you have a list of software requirements? Is it complete? Is each requirement covered by a test? Has that test been executed? Etc.
If you’ve created your software requirements document one day before your product hand-in, that may raise some flags, but it probably won’t be a deal-breaker. Again, the focus is on completeness of documentation, not time plausibility.
And again, all of this changes after you first product certification. Now you have to document something called change requests. That means that you have to describe and approve each change before you start implementing it. Quite obviously, this will mean that you can no longer document things retrospectively.
And I suppose that’s okay, because you wouldn’t want to just go ahead and implement random changes to a medical device which is on the market and used by patients.
