The UK government evaluates university research every six years. For the 2014 assessment, there is a new feature to the Research Excellence Framework (REF): each university has to provide a number of Impact Case Studies.
An Impact Case Study demonstrates how some piece of university research, done in the last twenty years, has had impact in the last five years. Impact could be on industry, the economy, society, public policy, or the environment. The one thing that does not count as impact is inspiring and influencing our colleagues in other universities.
These Impact Case Studies sounded straightforward when I first heard of them, eighteen months ago. My university department has done good research that has had significant impact: we have many successful spin-out companies, we have experts who advise government, we get regular media coverage influencing public debate. I thought it would be easy to demonstrate impact.
Eighteen months on I find that it is harder than we expected. The problem is providing concrete evidence of the impact. For example, we can prove that a professor has spoken before a select committee at the Houses of Parliament. That is easy; it is a matter of public record. But can we go any further than that? Can we demonstrate an unbroken chain of connections from his evidence to the select committee’s report to the draft legislation to the final law? That is more challenging.
It has even proven difficult to provide concrete evidence of industrial impact. For example, we know that a particular piece of university research led to the founding of a company. We were there. We watched it happen. But how do we provide concrete evidence? We can give you the contact details of the CEO, who can explain the connection if someone telephones her. Would that be sufficient to convince you? We can provide a link to the company’s website, if the company thought it useful marketing to say that their product results from research at our university. Would that convince you? Should it convince you?
Assuming that you are happy to accept that this company is based on university research, how do we then demonstrate economic impact? After all, this company could be some fly-by-night outfit that has no customers and is about to call in the receivers. We can refer you to any information that is in the public domain, but the evidence we need may not be in the public domain.
Let me give you a hypothetical example. We have a (hypothetical) company which is clearly based on our published research. Our company’s principal customers are amongst the largest manufacturers in Europe. Our company’s product has given significant competitive advantage to its customers. But the company will not release the names of its customers to the university, nor release any details of the size of the competitive advantage that it is able to achieve for its customers. These details are the evidence we need to demonstrate economic impact but these details are commercially confidential. If the company were to release them, even to us, there could be legal action from its customers or its shareholders. The company has a vested interest in keeping the confidential information to itself.
Now, the government’s REF mechanism allows a university to flag such information as confidential. This means that the flagged information is accessible only to the REF assessment panel, and is not released to the public. However, even with this provision, the company will not release its confidential information to the university. It is simply not in their best interest.
So, what can I say to the REF panel about this (hypothetical) company? I can tell the panel that your taxpayer’s pounds, funding our university research, have had a significant benefit for UK PLC. I cannot tell you what that benefit is nor how big it is because the company that we spun out has its hands tied by commercial confidentiality. I am reduced to saying “you will just have to trust me on this one”.
We failed to grasp this disconnect when the REF mechanism was set up: Although the REF mechanism allows for confidential information, we cannot tell the REF panel about the confidential information if we do not know it ourselves. The assumption was that the company and the university would cooperate to submit to government the best possible evidence. The reality is that the company and the university have different best interests.
The image of aircraft engine construction is courtesy of the US Library of Congress via Flickr. I chose it because the long-term research partnership between Rolls-Royce and the University of Cambridge is a fine example of the way in which a university has economic impact. Rolls-Royce is not the hypothetical company.