In a few words, what was the assignment about?
THOMAS: It was about examining how their organisation processes look today and how they will look when they start including more products in their production. Can we point at a Gap between the two scenarios?
Why conduct a risk assessment/ GAP analysis?
CHRISTIAN: If you have to start from the top, the GMP regulations stipulate that it must be done at this point in time. If you are a GMP company, you must draw up a risk assessment – not only on the new products but often on existing products also. Conducting an assessment was previously optional. However, it does give you a good picture of where the shoe pinches, so to speak – and where to implement changes. In the areas where the risk is high and the impact is high, you should address the problems promptly. If the impacts and risks are small, it is OK to wait. It is a prioritising method which originates from the military. NASA uses it, the aviation industry uses it, as does the oil and gas industry. What is the probability of a power plant exploding? Of a valve starting to leak? In this case, we look at equivalent issues.
Is it primarily when you start up something new that there is a need for a risk assessment?
CHRISTIAN: You will typically prepare a risk assessment at the startup of a new project / new product, but the assessments also have to be updated continuously. A lot of quality measurements are conducted during the first year, and then you look at how often things are happening. Do the measurements deviate from our estimates? In which case you must revise the probability in the risk assessment and it may suddenly turn into “critical”. That is the continuous inspection. Most of the work with this is done at the beginning and follows the maintenance work which often does not require much.
How did you perform the assignment in practice for Bavarian Nordic?
THOMAS: All in all, it took approx. 6 months to collect and process all data and then to report our findings. We agreed on a fixed meeting structure, we were there regularly every Friday and we also notified them as to which people we needed to interview and when. In addition, we held weekly steering group meetings during which we briefed the VPs each within their area of responsibility, and we mapped out possible challenges and problems in relation to the process.
CHRISTIAN: We normally take our starting point in a combination of our experience from previous assignments and guideline Q9 of the GMP regulations. We broke the project down into 5 main areas that needed addressing: Production, quality, laboratory, automation, and IT and logistics. We first looked at what thoughts they had already given to these different issues. For example in relation to the requirement specification for the extension. We read through their documentation at the instruction and procedure level. We conducted interviews with key personnel for the 5 main areas and we looked at the production in person.
We then drew up a risk matrix with probabilities and impacts. We identified in the area of 80-90 issues of concern of a larger or smaller degree and boiled them down to 16 critical points. That is points that might stand in the way of approval and execution of the project. The remaining points we included in an improvement catalogue.
Concurrently, Bavarian themselves conducted 2 workshops attended by a high number of employees, from operators to academics, and they prepared a project catalogue with over a thousand issues. We compared this with our list and found that our issues covered all their issues as well as a number of issues that they had not themselves identified, which showed us that we had been all the way around.
THOMAS: The report itself was prepared based on a combination of an audit and a traditional GAP analysis. We handed it over with a list of issues which are either classified as green, yellow or red. The issues classified as green indicate that everything is as it should be. The yellow issues need a closer look at some point in time, and the red issues need immediate attention.
What challenges / problems did you run into in connection with preparation of the report?
THOMAS: Many of the employees were not sure what it would be used for. They may have suspected that we were starting on some form of rationalisation exercise, so not everybody felt like opening up to us. It was important that we broke down that barrier and it was one of the issues that were discussed at the steering committee meetings and which they had to take up for discussion in the organisation. Ultimately, it is question of “help to help yourself”.
How did Bavarian Nordic receive the report?
THOMAS: They gave it a very good reception. After that, we conducted a review of the report, a workshop with the parties involved in the report, management groups et cetera, where the employees could also ask questions if there were unclarified issues or if they had questions to ask us or about the report.
What do you feel that Bavarian has gained from the risk assessment?
THOMAS: They have ”pressure-tested” some things. They now know in which areas they are keeping up. They know which areas look reasonable and it has been confirmed that they can easily proceed with these areas. For example, their training programmes are at a very high level – quite a bit higher than what we have seen in other places. There were thus a lot of areas where we could tell them that they are doing this well. We also found areas they needed to take a closer look at. For example, there was a question of some ID cards that they brought into the cleanroom area due to requirements specified by the American authorities. In this case there was a risk of contamination so this was changed right away.
How do you set about interviewing the individual employees?
THOMAS: We held meetings with many of them several times. We followed them around in their work and we gained an insight into what tasks they perform and how, and we met with them a few times. Typically 1-2 hours at a time. Some of them also brought a colleague the second time around if there were some areas they did not have immediate knowledge about.
What made an impression on you as you looked at Bavarian Nordic?
CHRISTIAN: Their training plans. As mentioned before, they are very advanced. We think that many others could benefit from the way they conduct their training at Bavarian Nordic. This is also one of the issues called attention to by the authorities. Most of the errors are human errors. The equipment is usually working so it is therefore important that good training programmes are in place for the personnel. In the nature of things, this issue is given a lot of attention by Bavarian Nordic as the authorities are well aware that Bavarian Nordic is a company which traditionally comes from development. It is a company that is good at inventing new things, changing things and adjusting things. All this is against the GMP principles which are about everything being the same again and again and again. So the training plans at Bavarian Nordic are part of their efforts to proceed from being a development organisation exclusively to also include production in a proper way.
What is important when you want to draw up a risk assessment? Are there elements that receive your special attention?
THOMAS: We give special attention to sending in people with vast knowledge. These must be experienced people who can challenge the issues and ask the right questions. The GAP analyses require experience. It is a task that we like to take on and our competencies have definitely grown in step with us having a number of similar tasks behind us. It is important for us to find the right people for the task and our structure allows us to quickly and effectively use our networks and look outside the organisation if the exact match is not already found at our company.
Bavarian Nordic will run a campaign production. How does this impact the risk assessment?
CHRISTIAN: In a good way. If they had decided to run two parallel production lines, it would increase the risk of cross-contamination. Running one product and then cleaning and gassing the area prior to running the next product decreases that risk.
Did you yourselves learn anything from the Bavarian Nordic project?
THOMAS: We have learned something about handling the customer’s employees. As mentioned, at the start we were met with a certain measure of distrust. “If I speak my mind will I lose my job” et cetera. It is about trust. We have to find the truth. We need to get an honest picture of the company. We must therefore convince the employees that this is not a rationalisation exercise. It is an opportunity for them to make themselves heard. Management must communicate this in a good way and we have to help them do this. We have decidedly gotten better at tackling this very situation.
Do you ever see an overlap between risk assess-ment and rationalisation?
CHRISTIAN: Yes, there can be. We place the organisation under a magnifying glass so sometimes the company’s attention is drawn to certain issues. For example, we may observe that a pallet is sitting in the production area for a very long time, and this is a problem as far as quality is concerned; so we ask “why is this pallet sitting there”? Can we move it along faster? This thus creates a parallel rationalisation exercise which may / may not have the effect that the production process can be speeded up or that there is less waste. The way we look through the filter is based on quality-related viewpoints so we do not consciously look for such issues.
In relation to Bavarian Nordic, many of the over 1,000 issues pointed out by themselves were more of a lean manufacturing nature than a quality nature. They were improvement suggestions which would also for them be economic gains, but seen from the risk assessment point of view these were not issues we deemed very important. The issues will surface so it is difficult to separate risk from rationalisation completely, but our focus is always on quality. This is what we are there for.