High-quality decision-making matters. This is especially true in the healthcare sector where wrong decisions can be a matter of life and death. As a result, the first guidelines on how to take decisions based on the best available evidence emerged in the late 90s. Although developed in the healthcare sector, evidence-based approaches quickly became popular in law enforcement, public administration and across many other areas of management. What is evidence-based management and what benefits are there for management practitioners?
Evidence-based Management for practitioners explained
Evidence-based Management refers to an approach that guides management practitioners about how to make better decisions. Being evidence-based means focusing on practices that have been tested in real-world situations and shown to work in meeting pre-defined quality standards. This makes “evidence” a better foundation for decision making compared to just relying on untested anecdotal experience, beliefs and other less rigorously tested sources such as business books, training and blog posts. From a practitioner’s point of view there are three main building blocks you should consider to become an evidence-based manager:
Four sources of evidence
Structured decision making process
Four sources of evidence
First, there are four different sources of evidence that help you to make decisions on a sound evidence basis:
Stakeholder values and concerns
Insights from scientific research
Management practitioners who consider this 360-degree spectrum of available evidence avoid missing important insights not yet on their radar and relying on an incomplete picture when taking decisions. In addition, the four sources of evidence make sure that both contextual (e.g. organization-specific knowledge) and meta-knowledge (e.g. problem-specific knowledge) is considered. However, not all four sources of evidence are of the same quality. You have to select the best available evidence which leads us to the next building block.
Structured decision-making process
The second evidence-based management building block is a structured decision-making process similar to the plan, do, check and act cycle (PDCA-Cycle). The structured decision-making process starts with asking yourself: what needs to be achieved? Being aware of the current state and the desired future state before taking the decision is the very basis for step two: acquiring evidence. Once you have collected the evidence relevant to your business challenge, you need to critically appraise it. This third step ensure that you only consider high quality evidence for your decision-making process. In terms of the PDCA-Cycle, these three steps can be summarized as “plan”.
After determining the key take-aways from the four sources of evidence it is time to make your decisions and move your organization towards the desired future state. This fourth step reflects the “do” in the PDCA-Cycle. However, as organizations are complex systems embedded in a volatile, uncertain, complex and ambiguous (VUCA) environment, there is no 100% guarantee that your decision is sufficient to reach the desired future state. Steps five and six are about analysing the outcomes of your decision on a regular basis and re-adjusting it with additional interventions, if necessary. In terms of the PDCA-Cycle we are talking about “check” and “act”.
Evidence quality criteria
Management knowledge is not immune to the ongoing fake news and alternative facts discussion. To combat these pitfalls, management practitioners are urged to critically appraise the quality of evidence before using it in their decision-making process. However, how do you judge whether a piece of content should be considered as solid evidence or just another management fad? While there are no fool proof criteria that helps you to make this distinction, there is a set of tools and guidelines you can consider. We will briefly address three of them and point to where you can get more information about evidence quality criteria.
Evidence hierarchies have been around for many years now and give a good appraisal of how solid a piece of evidence really is. On the top of evidence hierarchies , you will usually find meta-analysis and systematic reviews of randomized control trials (RCT) which represent the “gold standard” in quantitative research design. However, in management we deal with real life settings which makes is sometimes hard to apply lab trial results. At the other end of evidence hierarchies, you will usually find theoretical knowledge and case studies without much empirical evidence. Interestingly, surveys which are usually conducted at a certain point in time and (cross-sectional studies) are pretty low in the evidence hierarchy.
Reliability and validity
Reliability and validity are two of the most important quantitative research quality criteria. This makes them a must know for evidence-based management practitioners. In a nutshell, reliability refers to the consistency and reproducibility of results when measuring the same things – making sure that your tools are measuring what they say they are measuring not just once but a multiple times. Whether results accurately represent what they claim to represent is subject to the quality criteria assessing their validity. You’ll find a more detailed introduction into reliability and validity our CQ Dossier “How to Evaluate the Research Quality of Scientific Studies?”.
Solidity rating is a concept we at CQ developed to rate the state of the literature covering a specific business challenge in terms of evidence quality. It is based on five levels (from 0 to 5) and assigned to all our CQ Dossiers. The idea behind the solidity rating is to provide management practitioners an immediate indication of the state of the literature underlying our CQ Dossiers.
Evidence-based Management benefits
Management practitioners that follow an evidence-based management approach can expect numerous benefits. First and foremost, you will improve your decision-making quality which will add up to an increased organizational performance over time. The awareness that knowledge varies in quality and that there is no ‘silver bullet’ to solve business challenges will also make you a more critical consumer of the management literature. You will also learn more about the human and organizational factors behind a good place to work and a high-performance organization. This will also help you to manoeuvre your organization through challenging times when your leadership is needed more than ever.
Evidence-based Management shortcomings
While evidence-based management is associated with numerous benefits, there are also some shortcomings you should bear in mind. We will address four of them below, and suggest strategies on how to mitigate them.
Evidence is never complete
The body of evidence related to a specific business challenge is always an incomplete snapshot at any given point in time. As knowledge production progresses there will be new insights and the available body of evidence changes as well. Therefore, it is critical that you keep an eye on the latest developments in the body of evidence, ensuring relevance to the business challenges that are most important to you.
Evidence-based Management is inclined towards scientific methods
Evidence-based management strongly relies on scientific methods for knowledge production and research design. One could argue that this is a benefit rather than a shortcoming. The problem here can be that many scientific and academic studies are filled with statistics and dense data sets, making it difficult and time-consuming for practitioners to get to the practical application of their findings.
Quantitative research tends to be abstract and hard to apply in practice
Evidence-based management is rooted in the natural science paradigm which assumes that it is possible to derive (up to a certain extent generalized) conclusions about which management interventions work or don’t work. As a consequence, quantitative research findings tend to be abstract which makes them hard to apply in management practice. Depending on the business challenge you face, it can be beneficial to combine quantitative with qualitative research findings which tend to be much more context sensitive but can lack generalizability.
Evidence-based Management is time consuming
For busy management practitioners going through a structured decision-making process and screening the four sources of evidence requires a considerable time commitment. Over time you will find that practice makes perfect and the structured decision-making process will become the norm rather than the exception. Another tool that can help you to speed up evidence-based management decision making are our CQ Dossiers that summarize the state of the literature related to specific business challenges.
Evidence-based Management implemented at CQ
CQ relies on many of the principles, tools and methodologies rooted in the Evidence Movement. We have started with publishing CQ Dossiers that summarize the state of the scientific literature on the business challenges that management practitioners face in their daily business. Our ‘Solidity Rating’ provides you an indication how strong the available body of evidence for each particular CQ Dossier is.
In addition, you will find a critical appraisal paragraph assigned to each CQ Dossier that discusses limitations and weaknesses of the presented evidence. All our CQ Dossiers are Open Educational Resources (OER) you can use, copy and share for free.
Evidence-based Management is much more than aggregating scientific literature. This is why we also offer, trainings seminars and short courses addressing specific business challenges from an Evidence-based Management point of view. However, a team of management practitioners supported by a professional trainer does more than just gain insights from scientific research. Being part of real-life organizations, the ‘learning team’ also addresses the other four sources of evidence relying on their professional expertise and their stakeholder’s expectations.
The unique combination of an evidence-based approach, professional trainers, a team of management practitioners and real-life business challenges makes trainings and consulting services from CQ an ideal Learning & Development intervention.