Practice Impact of IS Research

In the editorial of the last issue of BISE in 2014, we discussed key aspects concerning the quality of research and quality standards in our community (Bichler et al. 2014). For 2015, we are planning three more editorials that address topics which we think are relevant to our community with different lead authors. The first one is on the impact of our research results on practice, which includes business, public services, private lives, and society overall. Of course, research also takes effect through teaching and education, not to forget the impact of our research results on other communities. Related to this, Mertens (2009) provides an overview of the various tasks of university professors which all have an effect. In this editorial, however, we will limit ourselves to the rather direct impacts of IS research on practice as outlined above.

The impact of research results is an important element of the discussion about quality and most of us will agree that apart from producing reliable results using rigorous scientific methods, research in IS should contribute to relevant questions and make a difference for economic or societal problems.

The impact of research has become a contemporary issue in times where publication rankings and citation counts seem to dominate the evaluation of research. Some even claim that the increased importance of journal publications has elevated academic research into an ivory tower where academics initiate a discourse with academics but do not connect to the outside world. Nevertheless, many academics consider the effect of their research on practice as a very important goal. Impact has not only become part of the mission statement or branding of distinguished universities (such as the Erasmus University Rotterdam or the University of St. Gallen). It has rather turned into a decisive factor in the assessment of universities in different countries (such as the Research Excellence Framework in the UK) as well as the subject of conference tracks and panel discussions in conferences (at ICIS or at this year’s WI 2015 in Osnabrück). For business schools, the Association to Advance Collegiate Schools of Business (AACSB) recently published a report on the impact of research (AACSB 2012), and it has emphasized its paramount consideration in the accreditation process.

While people tend to agree on the importance of IS research impact, there is less of an agreement about how to define such an impact. Indicators include patents, spin-offs, tools and methods developed for companies and other organizations, research funding from industry collaborations, public media citations, consulting reports, invitations to serve as experts on policy questions and government consultation, publications in practitioner journals, and many more.

As academics, we are trained to measure and compare phenomena. Why should we not try to measure research impact? It turns out that measuring impact on a single scale is difficult, to say the least. Let us illustrate this with a few examples, some from neighboring fields such as informatics or economics. When Stephen Cook published his paper “The complexity of theorem-proving procedures” in the ACM Symposium on Theory of Computing in 1971 (Cook 1971), probably only few people would have thought this theoretical question would have a practical impact. While the fundamental question whether NP-complete problems are solvable in polynomial time (P = NP?) is still open, it is accepted nowadays that complexity theory is very important to many problems in business practice. It has become a very practical means to classify problem types and understand which problems in supply chain management, operations, and logistics are tractable, and which problems are not. Differences in an exact and a heuristic solution can often mean huge cost savings in practice. In other words, papers which seem to be theoretical at first sight might turn out to have a substantial practical impact. As Kant (1793) or Lewin (1952) stated, there is nothing more practical than a good theory.

As another example, models about the economics of information goods in the book “Information Rules” by Carl Shapiro and Hal Varian (Shapiro and Varian 1999) arguably had a substantial impact on management practice and thinking although no immediate product resulted and no jobs were created that could be directly linked to this book. There is no doubt that the book had considerable impact on academic research in our field as well, beyond the direct practice impact discussed in this editorial.

While there might sometimes be a gap between the academic and practitioner, it appears to be a non-reflected prejudice that practitioners do not read academic papers. In spectrum auction design and market design in general, academic papers are quite influential in the related policy discussions, and the same is true for technical fields such as cloud computing. Many reference models and methods developed at universities have been adopted by practitioners, and these are no exceptions.

On the other hand, typical examples for research impact such as university spin-offs might not always be an indicator for research impact. Many spin-offs are successful in the market with a product or service which does not rely on research results at all. There is nothing wrong with such an enterprise, but it should not serve as an indicator for research impact. Whether there is real research transfer into a product might not be obvious at first sight.

Putting it differently, the practical impact of research is paramount for the technology innovation strategy of our discipline. First, it facilitates and catalyzes the transformation of theory into technology and, second, it helps to identify new theoretical phenomena through observations or technological use in practice. As Chmielewicz (1994) states, every technology can be considered the outflow of one or multiple theories. Conversely, theory can and should be a useful foundation for technology – no matter whether we refer to information or other technologies (Heinrich et al. 2011). These are compelling reasons why it would be careless if we reduced or lost our focus on practical impact.

No doubt, there are many lessons to be further learned for academics who also work as entrepreneurs. Everybody with experience in starting up companies or industry projects knows that these activities can lead to many new insights and interesting questions for research. In some fields it is important to have substantial industry experience to fully understand problems in a specific domain and to evaluate research results. Invention and innovation are both important and closely connected, but they also constitute different tasks and they both take time. It depends on the field and certainly also on the personal preferences of academics how much time they spend on research or the transfer of research results.

The above examples aimed at illustrating that the impact of research can take on various forms. Research on information systems outsourcing will have a different impact than research on main-memory databases or on business process modeling. The departments of our journal deal with different problems and phenomena, ranging from decision support to economics of IS and IS strategy. All of them have a value of their own and they all matter to IS professionals nowadays, one way or the other. However, it is also clear that research impact cannot be compared by using a single dimension across these fields.

The assessment of research requires a holistic view of the depth of a contribution, the specifics and state-of-the-art in a field, and the way how the research results have already impacted or could impact practice. This also holds for the review process in our journal, a process which should bring out papers with scientifically solid research results, but also with an impact. While sometimes impact can only be assessed ex post many years after a paper has been published, a paper’s potential impact should be a central criterion for its evaluation, and this cannot be “replaced” by rigor.