“The greatest danger in performance measurement is not seeing too little, but believing you’ve finally seen everything.”
Nine KPI Lessons, One Underlying Problem
Throughout this series, we have explored what might seem like nine separate problems with measuring performance. Each article examined a different symptom, manifestation, paradox, or unexpected consequence of using KPIs to understand a complex organization. Viewed on their own, these issues seemed only loosely related. When viewed collectively, however, they tell a surprisingly coherent story.
Our journey began with the KPI Theatre, where we observed how the act of measurement changes behaviour. Once individuals know they are being observed, they instinctively begin to focus on what is visible. This is not necessarily because they are engaged in manipulation or deceit, but simply because human attention is finite and people attend to that which attracts attention. Slowly, performance shifts from being good to looking good.
Then, we examined Goodhart’s Law to see what happens when that observation becomes a target.
What began as a useful indicator of performance becomes, over time, the objective itself. People stop asking whether the organization is serving its purpose and start asking if the numbers look OK. The simple act of optimizing has quietly replaced the process of understanding.
It would seem natural that if one metric blinds you, then two will be better, but that then begets something called KPI Saturation – when everything, every function, every initiative, every strategic priority, and every operational process is represented on a dashboard.
Everything becomes so overwhelming that we are blinded by what is actually present.
Information overload has given way to attention scarcity, and when attention becomes a scarce organizational resource, metrics themselves become valuable. In The Politics of KPIs, we examined how indicators gradually evolve beyond measurement tools into instruments of influence. As dashboards grow more central to an organization’s decision-making, knowledge that is difficult to reduce to a number becomes marginalized and increasingly ignored.
Institutional knowledge, relationships, craftsmanship, intuition, judgment, and the trust required for collaboration – all fall off the strategic radar as information that cannot be easily expressed in numerical terms disappears from the conversation, as was illustrated by KPI Memory Loss.
Organizations do not necessarily fail because they lack information. Rather, they fail because certain forms of knowledge no longer receive attention, or, even worse, are actively dismissed by those who treat them as irrelevant data points. The effects of these trends, however, do not remain confined to the organization itself. The final issues that we uncovered in this series impacted the very people who were employed to deliver on those numbers.
In the KPI Identity Trap, we witnessed how those being measured can become so completely identified with the metrics by which they are assessed that they forget to ask whether they are doing worthwhile work and simply focus on what the dashboard or scorecard shows. It could be seen that the last, and perhaps most fundamental point in this series – KPI Blind Spots – is almost a natural consequence. At best, a dashboard tells us something about the reality of our situation and, at worst, it tells us an unintentional lie.
Yet the lie is never told; it simply emanates from what is left unsaid: the part of reality deliberately omitted to focus on what is believed to be most important.
Realistically, none of these were separate problems at all: behaviour, targets, information overload, politics, organizational memory, personal identity, blind spots. These are not independent phenomena, but simply different facets of the same flawed logic. A logic that proceeds on the basis that reality, as it is currently expressed in KPIs, continues to surprise us only because we have not yet measured enough.
This is what can be called the Completeness Fallacy, perhaps the most insidious belief underpinning modern performance measurement – the belief that performance (and measurement) can eventually be made complete.
Why Every Surprise Seems to Demand Another KPI
These are scenes every seasoned executive has lived out.
An unforeseen event derails a critical business process.
A major customer segment starts leaving faster than anticipated.
A manufacturing defect escapes quality control.
A major project comes in late even though all milestone flags are green.
A cyber attack bypasses the security systems that should have prevented it.
A high-performing, tenured employee quits with little notice.
The post-mortem begins, with executives staring at dashboards and data visualizations, trying to pinpoint where the red flags should have popped up and when someone should have noticed that something was going terribly wrong.
One question invariably surfaces: “Didn’t we have a KPI for this?”
Sometimes the answer is yes, and it was simply ignored. Most of the time, more frequently than many organizations are comfortable admitting, the answer is no. Thus, the logical conclusion seems obvious.
“Let’s add one.”
On its face, this is completely reasonable. Every failure is an opportunity to refine the measurement system. If an important early warning was missing, then the dashboard should be augmented to track it. It often is the right solution, truth be told.
The danger emerges when adding a new KPI becomes the default response to unexpected events, because the next time the unexpected happens (and it always does), another KPI is added. Then another and another.
The dashboards get bigger, the reports get longer, the task manager bloats, and the analytical tools become more sophisticated. Still, unexpected events persist doggedly. Each surprise seems to reinforce the idea that something else must still be missing, driving the organization toward the impossible goal of complete measurement.
This is what can be called the Completeness Fallacy.
The Completeness Fallacy is the mistaken belief that all organizational surprises stem from missing dashboard metrics and that simply adding enough KPIs will eliminate uncertainty entirely.
Complex organizations aren’t like simple machines, made of gears & cogs. They are living, breathing systems made up of people, incentives, cultures, relationships, informal networks, dynamic markets, shifting customer expectations, evolving technologies, and countless interactions that cannot be fully predicted.
Every solution creates its very own new problems. Every intervention changes the system it attempts to measure. Having a complete representation of all possible futures is impossible. Ironically, as organizations pursue completeness, they move further away from true understanding because the question asked subtly changes.
We stop asking:
“What have we misunderstood?”
Instead, we start asking:
“What KPI are we missing?”
These two questions sound similar, but there is a profound difference. The first question is about understanding and assuming that the reality of the situation is more nuanced than the dashboard’s representation. The second question focuses on measurement and suggests that the dashboard just needs an additional piece.
The Endless Expansion of the Dashboard
Think about a dashboard that includes 50 carefully chosen KPIs. A few weeks pass, and then a completely unanticipated problem arises that those 50 KPIs couldn’t have foretold.
“We need to add one more KPI!” – Leadership.
The dashboard grows to 51 KPIs. A few months later, an even greater shock arises. A new KPI is added. The dashboard now has 60. Soon 80. Then 100. Eventually, someone gets tired and asks, “If we have 100+ KPIs on our dashboard, how did this still catch us out?“
It’s a strange psychological paradox at work here.
On the one hand, leaders think “everything that is important must be on the dashboard.”
On the other hand, when something isn’t on the dashboard, it is, at least initially, discounted precisely because it is unmeasured.
Whenever reality disappoints, we try to achieve completeness by expanding the dashboard. As the dashboard grows, our confidence in it grows. That, in turn, makes the next surprise that happens all the more baffling. People begin to wonder in dismay how all of this can be happening, since they are measuring everything.
Except they are not. They never can and will never be able to. No dashboard can perfectly mirror reality; reality is always larger than its reflection.
The danger isn’t what dashboards leave out; the danger is that we forget what they have to leave out.
Babies & Video Games: Why More Doesn’t Always Mean Better
The Puzzling Perplexity of Predicting Progeny
The experience of dealing with babies is a universal (if not always enjoyable) one. Say your baby – a perfectly healthy baby, no less – is inconsolable, though not crying as a result of any obvious issue. They have been fed, cleaned, kept comfortable, and healthy, so why do the tears persist?
You hand the child a colorful toy, and for a brief moment, the wailing subsides before resurfacing with renewed vigor.
“Maybe the problem is simply that we don’t have the right toy!” and so you rush out and acquire one, only to have it achieve the same limited result. You then acquire another that sings, and another that flashes. The child continues to cry, and your resolve is steadfast: there has to be one “right” toy out there to soothe their little agitated spirits!
This process seems almost logical, and the conclusion (that another toy is just around the corner) feels almost automatic. After all, if there were a truly suitable toy, the baby would just stop crying. Right? RIGHT?
This assumption misses a crucial detail: the baby wasn’t looking for another toy at all. Maybe they simply wanted to be held, or was bored lying in one position for too long, or maybe they wanted someone to talk to them, or just to feel the comfort of their parents’ closeness. The parents didn’t need a better toy; they needed to understand the baby.
In the course of our lives, we will all learn an invaluable lesson. Sometimes the quickest, easiest path to resolution doesn’t lie in introducing a new component or finding a missing element, but in paying more attention to the subject of our concern.
Organizations are remarkably similar. Organizations often take a very different, and rather analogous, approach. Every unforeseen problem requires a new key performance indicator (KPI); every anomaly requires a new dashboard; every overlooked blind spot demands a new metric.
It’s quite possible that the problem isn’t that another metric is needed. It might be that the organization needs to better understand what it is trying to achieve in the first place. Instead of reading reports, managers might need to speak with their employees more often. Instead of filling out surveys, customers might want to talk to real humans to air their frustrations.
For example, to understand a problem in production, supervisors should walk the factory floor rather than stare at a production-tracking dashboard. While the dashboard asks, “What else can we measure?” reality is asking, “Have you truly understood me?” That distinction is at the heart of the Completeness Fallacy.
The Curious Compulsion to Continue
It plays out just as clearly in an area where organizational management can’t possibly be expected to surface: video games.
Any person who plays role-playing games or massively multiplayer online games knows that optimization quickly becomes a way of life if left unchecked. Let’s assume you load one up, make a character, and it just isn’t putting out the damage-per-second (DPS) that you were aiming for.
The logical first step, you assume, is to get a DPS meter, take some measurements, and look for the source of the deficit.
The meter tells you that you’re falling short of the damage output of everyone else on your team. You now have an answer to your problem. Or do you?
Upon inspecting your equipment, you notice that some items are suboptimal, so you immediately replace them with better ones.
You head back into the game’s content, only to see a minuscule difference in your damage output. There must be some other missing factor, something else you did not account for yet again.
You realize your equipment isn’t enchanted, and so you spend hours painstakingly applying the most potent enchantments possible.
The result is still marginal. You now invest in better gems, talent points, food buffs, consumable potions, and specialization changes. Eventually, you end up with half your screen clogged by meters tracking DPS, timers, combat logs, raid frames, cooldowns, boss warnings, and who knows what else. Still, your character’s damage output does not noticeably improve.
Add-ons are useful, but the most obvious limiting factors, such as positioning, decision-making ability, encounter awareness, or knowing when not to attack, cannot be directly measured by another number or add-on. These qualities must be learned through trial and error and by recognizing patterns. However, the more information cluttering your screen, the easier it becomes to miss the giant boss looming directly in front of you.
The Misguided Mission to Measure More
Organizations can find themselves facing this type of music in exactly the same way. The dashboards continue to grow, the reports become more elaborate, the metrics increase, and the alerts pile up, giving executives an enormous volume of information to analyze.
In practice, the actual amount of real-world understanding often advances at a far slower pace because it becomes so easy to confuse the process of measurement with actual understanding:
You can measure customer satisfaction and learn that it’s trending downwards. However, you can’t measure the sound of disappointment in a customer’s voice on a support call.
You can check a productivity dashboard, which will tell you that the project velocity has decreased, but it won’t reveal the engineer who’s afraid to challenge a deadline they know is unrealistic.
You can monitor employee morale via an engagement survey, but that can’t capture the quiet moment in the hallway, months ago, when confidence eroded just a little bit more.
You can assess sales conversion rates and identify where prospects drop off, but you can’t measure the trust that was lost in a single rushed conversation.
Wisdom doesn’t grow automatically simply from an increasing quantity of observations. Wisdom is built through an interpretation of those observations within the human context in which they occurred.
This is the true danger of the Completeness Fallacy: it persuades leaders to believe that the missing piece of the puzzle is simply another data point, and that there is never a need to stop looking at data and start listening to their people.
From Hospitals to Software: How Industries Share the Completeness Fallacy
The form in which the Completeness Fallacy shows up may vary significantly between industries, but the core principle doesn’t: whenever the world spits out an answer that surprises us, we start hunting around for some new measure, instead of asking a question: “Are we perhaps measuring the wrong thing to begin with?“
Healthcare: Measuring Patients While Missing Care
Modern hospitals amass vast quantities of data: Patient wait times, Bed capacity, Readmission rates, Length of stay, Medication adherence, ED wait times, Length of procedure, and Infection rates, among many others.
They’re all crucial metrics, but none of them fully explains why some patients with similar clinical profiles take weeks to recover while others take days. Many of the things that drive outcomes – whether the patient actually grasps what the doctor just said, has someone to nudge them about medications, or simply trusts their physician with what’s bugging them – are tough to fit into a standard spreadsheet.
We see a bad readmission outcome and naturally reach for a new quality measure. Sometimes that makes sense, but more often than not, we need to make sure the conversation was handled properly, not simply as a way to create another quality benchmark.
Manufacturing: Perfect Machines, Imperfect Systems
Manufacturing organizations usually have extremely detailed operational dashboards: Machine usage, Cycle time, Defect rates, Overall Equipment Effectiveness (OEE), Scrap rates, Downtime, or Energy usage, to name just a few.
When product quality drops inexplicably, the first reflex is often to monitor it even more closely. Very often, the mentality is “time to add another production KPI and slot in another quality check.”
Yet, investigations often find that technical issues are not at the heart of the matter; rather, the heart itself is gone.
Production targets subtly discourage the workforce from signaling minor deviations until they become significant ones.
None of these had anything to do with a lack of data regarding the machinery – these were all human systems impacting technical ones. What was needed wasn’t more sensors but a better understanding of those humans operating them.
Software Development: Measuring Productivity Without Seeing Complexity
Software teams today arguably measure more things than any team has before: Sprint velocity, Story points completed, Lead time, Cycle time, Deployment frequency, PR approvals, Bug counts, Code coverage, or Incident response time.
When the wheels are slowing unexpectedly, leadership frequently adds a new metric for engineering teams to work with.
“Maybe we need more code!”
“Maybe the reviews are taking too long!”
“Maybe the deployments aren’t happening often enough!”
Maybe the real problem might be that a legacy architecture is now extremely difficult to maintain. The team has quietly accumulated technical debt over the years and is now spending much more time understanding systems than developing features, and on top of that, cross-functional communication has broken down.
The biggest thing stopping the team from shipping may not be an absent KPI, but decades of accrued complexity that only a truly seasoned engineer can even see. Such nuance cannot be encapsulated in a dashboard.
Aviation and High-Reliability Organizations: When Safety Lives Between the Metrics
Few industries are more serious about measurement than aviation. Aircraft systems generate vast amounts of operating data, which are measured and remeasured with exceptional rigor.
Flight schedules are closely monitored.
Safety incidents are rigorously documented.
Procedures are formalized.
Performance is routinely assessed.
Nevertheless, aviation professionals know a secret that few leaders recognize: if no safety event has occurred, it doesn’t mean that safety is present.
The organization could show perfectly good safety performance on its metrics, while easily and effectively hiding its own psychological limitations regarding the issue from itself. Pilots may be unwilling to speak about hazards. Maintenance crews might be reluctant to acknowledge near-miss events because they might feel embarrassed or appear incompetent. A Junior team member might have no idea how the process could be made safe and would have no incentive to raise such concerns.
The overall safety indicators might be beautifully and blissfully green right up to the day of a disaster. When such a disaster occurs, the investigators rarely attribute the failure to a single additional KPI or a particular missing index or tool. Instead, they generally attribute the disaster to communication failures or human errors that may have predated the measurement systems.
The organization didn’t lack yet another metric – it lacked a deeper understanding of the system it thought it had measured.
In all 4 of the industries we’ve showcased, the situation repeated itself:
Disasters occurred
Organizations assumed they must be missing a metric
The metrics were expanded
They were surprised again and again
Such scenarios don’t happen because leaders lack the intelligence to be able to design and implement appropriate measurements, but because no complex system, as a matter of principle, can ever be as large as its own representations of itself. No dashboard can become the entity being measured.
When organizations reach that stage, they focus so intently on measurement devices that they neglect the journey and miss where they are going.
Final Thoughts
Looking Through the Windshield Instead of at the Dashboard
Never before has there been so much readily accessible operational information available. Never before has there been a way to see performance across continents in real-time, spot emerging trends within minutes, or turn vast amounts of raw data into simple, intuitive pictures. Organizations have become faster, more coordinated, better informed, and ultimately, more agile.
Problems arise only when dashboards stop being tools for understanding and start becoming substitutes for it. This is, ultimately, what’s dubbed the Completeness Fallacy.
It has nothing to do with whether KPIs are useful or useless, or whether measurement systems need upkeep or not. Rather, it is the false belief that uncertainty can eventually be engineered out of the organization by judiciously selecting enough indicators.
In modern-day organizations, every KPI answers one question but begs another. Every bit of extra visibility alters behaviour, thereby changing what the measurement actually means. Performance measurement isn’t a race towards a state of completeness. It’s a never-ending conversation between what can be measured and what requires only observation, judgment, curiosity, and experience.
Dashboards are the thing that should inform a conversation, never replace it. Think of them as a car windshield. It has never been designed to capture every detail of the road ahead. It won’t reveal what is hiding behind every building, nor will it show every hidden pitfall lurking around the bend. Its purpose is to provide just enough visibility to help us navigate while, at the same time, reminding us that the road itself deserves our attention.
Modern performance dashboards have precisely the same role. They are not the organization, but the windows through which we may view organizations in a richer, more dynamic manner than any collection of KPIs can adequately convey.
If we start to look at the windshield instead of through it, we are liable to lose sight of where we are going, and that may well be the most profound irony of modern performance management:
We have never measured more, and with better tools. At the same time, we have never depended more on the interpretation, conversation, experience, and context provided by the people powering the soul of our workplaces.
*********
Looking to strengthen your expertise in KPI design and performance measurement? Gain practical knowledge, proven methodologies, and globally recognized certification through The KPI Institute’s Online Certified KPI Professional.
In most enterprises, there is at least one, often multiple, dashboards. These glowing arrays of data, updating at lightning speed, are producing weekly reports that land in everyone’s inbox, from the CEO down to the floor supervisor.
The platforms used to gather such metrics are significant investments in configuration, and continued investment is required to extract this information and make it consumable by diverse stakeholders.
The problem is that, in many of these enterprises, the same metrics haven’t influenced a key decision for months. The underlying paradox of measurement within most of our current organizations is that the more we attempt to quantify something, the less effective we seem to become at deriving meaning and insight.
As the volume of available information proliferates, it seems increasingly challenging to discern which actions are necessary, let alone effective. Measuring for progress now seems to have morphed into measuring for mere participation.
Measurement in excess has transformed the metrics themselves into little more than participation indicators that have little to no genuine impact, when, really, you should measure only what can be measured well.
The Measurement Maximalism Trap: How Organizations Got Addicted to Data
The Dashboard That Glows But Doesn’t Guide
The dashboards originally designed for illumination, not for eye-popping designs, have become the organizational equivalent of corporate wallpaper. Visually engaging complexity, not driven to a specific action. This isn’t the flaw in the underlying technology. It’s the flaw in how you’ve defined what is worth measuring.
Today’s analytic toolset makes it easy to record virtually anything. Coupling the ease with an organization whose desire for control equals its belief in the power of more information to be that, led to the phenomenon of what you can call dashboard inflation: metrics accumulate until they crowd out the signal. When a dashboard reports 40 or 50 different metrics, there isn’t the capacity to read them deeply enough to explore anomalies and derive decisive strategic conclusions.
The organization skips and skims through. It glances over, looking for simple, overt, bright, shining red flags. Then it checks the numbers that matter and moves on. What was initially intended to drive action ends up as a task to be filed away as evidence.
The Math That Should Make Any Leader Nervous
Here’s a calculation that is great at cutting conversations short. Imagine an organization with six to ten departments or business units. Give each department an average of 8-10 KPIs to follow, which seems perfectly logical if you think about the unique needs of a single department alone.
Multiply the two, and you’re looking at 48-100 specific measurements the company is technically keeping an eye on.
Next, ask the questions that should follow:
Of all these metrics, how many are really key?
How many truly have an impact on whether or not the company is winning or losing against its highest strategic objectives?
How many exist only because someone decided back then that, since the system made them cheap and easy to generate, someone someday might even use them?
For most companies, the number of strategic indicators is considerably smaller than the hundreds one typically sees on a dashboard.
All of the others survive simply as products of inertia, and each of those hundred “key performance indicators” consumes time and brain space that should instead be devoted to the small subset of data that is truly indicative of performance.
If we had 100 “key” indicators, “key” would essentially become a meaningless word.
The “If We Can Measure It, We Should” Fallacy
The measurement maximalism trap is, at its core, a confusion between capability and wisdom. Modern data infrastructure has given organizations the ability to track almost anything in near real-time. However, capability and strategic judgment are not the same thing, and treating them as equivalent is exactly where organizations start losing the plot.
Just because something can be measured doesn’t mean it has strategic value.
Just because a platform supports 200 custom metrics doesn’t mean you need 200 metrics.
Just because data is available doesn’t mean adding it to your dashboard brings you closer to understanding your business.
What it typically brings instead is noise – an ever-growing collection of numbers that require time to maintain, energy to interpret, and attention that could otherwise go toward the things that genuinely drive performance. Gradually, the act of measurement begins to crowd out the act of improvement. Teams work harder at tracking their progress than actually making any. The organization becomes, in a very specific and avoidable way, busy without being productive.
KPI vs. PI: The Critical Distinction Most Organizations Have Forgotten
Not Every Metric Earns the Word “Key”
About 90% of what we call “KPIs” within organizations are not actually KPIs. They are PIs – performance indicators – and the difference is far more material than many admit.
A performance indicator measures something that occurs within an organization: ticket resolution times, report generation volumes, packing efficiency rates, or training completion percentages. These are all valuable numbers, certainly. They offer an operational view to the teams that own the associated processes and help them establish benchmarks for quality. Yet, they are not necessarily key.
A true KPI, used to the full potential of its name, links directly to a strategic objective.
The organization aims to increase its market share > the strategic KPI is market share.
The organization is trying to retain customers > the strategic KPI is customer retention.
It tells leadership at the highest level whether the organization is winning the game it believes it’s playing: revenue growth, net margin, profitability per customer, market share – these are the top-tier measures. Everything else is essentially noise or in service of those top-tier measures.
We have blurred the lines as organizations have grown, technology has become ubiquitous, and we’ve democratized metric-taking across teams.
It’s easy for each department or business unit to grab hold of operational metrics and dub them “KPIs” without asking whether they actually contribute to high-level organizational strategic outcomes.
Dashboards have become cluttered with PIs dressed up as strategic goals. Nobody realized they were promoted; they just sort of got there.
The 40,000-Foot View vs. Getting Lost in the Weeds So, what does a CEO actually need to know about the state of his company at any moment?
He really doesn’t care about the rate at which the warehouse packs goods unless that number affects a critical cost and/or the customer experience of the company as a whole. What he cares about are a couple of well-communicated indicators:
Is the company growing?
Is it making money?
Are customers staying?
Are we executing on the strategy we agreed we would execute upon? At that 40,000-foot level, you typically need at most six to ten really important indicators to accurately describe what’s happening. Everything else, all the departmental and operational process-level metrics, all the ratios that management needs to manage the day-to-day functions, is nobody else’s business in any of these discussions.
The weakness of measurement maximalism is, to some extent, the weakness of the hierarchy: not being able to sort and separate strategic vs operational measurements.
When you start bubbling up all the individual and departmental PIs to an organizational review meeting, the view gets blurred; the managers are going over meeting click-through rates and newsletter open rates rather than the items that are important and telling.
When KPIs Stop Driving Behaviour and Start Decorating Reports
There is a simple test that measures whether a metric warrants the label of KPI:
Does it change the way people behave?
It should detect issues early and clarify what success looks like and what’s left to do: in short, it should help teams focus.
Once a metric accomplishes these 3 things, it may deserve the “KPI” tag. When it doesn’t change behaviour or decision-making, it’s decorative.
If an organization suffers from KPI proliferation, it probably has too many decorative KPIs. Most of their metrics have gradually become decorative out of mere habit or routine report filler. They have ceased to prove anything useful. Those metrics may simply fill the gaps where data collection and reporting are requested, but without providing insights or stimulating any form of change in people’s work or behaviour. When a good KPI changes the organization’s functioning, a bad KPI or one of 50 other KPIs simply doesn’t.
Any company unable to differentiate between them faces a measurement challenge that is beyond the reach of mere dashboard adjustments.
The Real Cost of KPI Overload: Cognitive Fatigue, Decision Paralysis, and Teams That Stop Thinking
When More Data Produces Fewer Decisions
The hypothesis on which measurement maximalism operates is that more data equates to better decisions. It seems sound and scientific. It’s also quite wrong.
When executives see the dashboard, they usually don’t feel their decisions are being enhanced; the opposite generally occurs. Productivity may have increased, but consumer satisfaction has fallen. Moreover, 40 separate indicators are currently being updated simultaneously. As a result, decision makers delay while scheduling another round of meetings and trying to determine the scale of the real threat.
This is what experts call decision-making paralysis, a symptom of excessive reliance on indicators. Whenever individuals’ capacity to process additional input is overwhelmed, they automatically delay making decisions until a broader range of statistics is available. As more information is considered and more individuals are involved, the perceived uncertainty also grows, and, meanwhile, whatever issue the metrics were intended to uncover deteriorates.
The tragic reality is that these measurement frameworks, which have been used to accelerate decision-making, only lead to a stagnation of decision-making.
Hitting the Metric While Missing the Mission
Here’s a situation that illustrates the danger of mismatched KPIs more effectively than almost any theoretical statement we could make.
A company sees that its Net Promoter Score, its gauge of customer loyalty and enthusiasm, is falling. They identify a solution. Compensation and enticements are provided when feedback is being collected.
NPS rises. The number looks healthy in the next quarter’s report. The actual causes for the customers’ discontent – the friction or the failure of the product – were not, however, altered in the slightest. Many organizations develop this practice, often unwittingly, as they learn to prioritize the score rather than the outcome for which the score was originally developed.
Organizational staff who primarily gain recognition for meeting KPIs develop mechanisms to meet them.
It’s not pessimism – but rather human conduct in reaction to incentive design. Colleagues can tell which things are tested, observed, noticed, and rewarded. Individuals shift accordingly. They obtain experience in offering the appearance of efficiency, but not necessarily the efficiency itself. This is called “conquering the score while neglecting the objective” – one of the most costly forms of failing a company may encounter, since it is quite hard to spot in retrospect.
The dashboard seems all right, and values develop from the correct orientation. However, real life is slowly being corroded.
The Quiet Epidemic of Reporting Fatigue
There’s another cost of KPI saturation that doesn’t show up on any dashboard but that everyone inside organizations swimming in it can feel: the sheer time it takes to feed the beast that is the measurement system.
Getting a hundred different KPIs to tick and tock requires somebody or somebodies to collect, validate, refresh, format, and disseminate that data, frequently, sometimes weekly or monthly.
To a mid-size company, for example, those hours pile up in a hurry. Its analysts produce reports, managers pour over numbers they sort of get, and department heads struggle to fill out the same old forms with numbers only slightly different from last quarter.
That’s time spent feeding the system rather than fixing what’s broken, creating what’s needed, improving customers’ lives, helping their team develop, or making whatever executive decision they truly need to make. It’s the modern corporate bureaucracy, disguised as diligent management work. Over time, a unique, unspoken kind of demoralization seeps into these companies. The people who signed up to build things or help people come in and feel as though they’re spending an outsized fraction of their time on activities that yield little beyond raw data.
The link between their efforts and actual business outcomes begins to blur. Engagement lags persistently, subtly, maybe not to a critical degree, maybe, and not all at once, but to devastating effect down the line. The chosen metrics were designed to empower them to do more. They’re instead burning the fuel that could help them do so.
Signal-to-Noise Collapse: When Reporting Becomes the Work
Vanity Metrics and the Illusion of Progress
There’s an all-too-common disease lurking in metrics-driven workplaces: the proliferation of what could be described as “vanity metrics,” a collection of figures that make a report or a presentation look good but have little or no connection to real business performance: total hits to our website, our number of Facebook followers, the number of features we shipped this week, the amount of customer support tickets we logged, and so on.
These are all relatively easy to produce and easy to feel positive about, and, in most cases, have nothing at all to do with the really important questions like “Are we growing the right way?” or “Are customers truly getting value from what we produce?“
Vanity metrics are seductive because the directionality is right when you simply add more effort.
Your number of social followers will increase as you post more social updates. Your output quantity will increase as you produce more output. Your customer activity will rise when you do more of it. However, the link to a positive result is nonexistent.
Worse, vanity metrics muddy the waters. Genuine metrics like customer retention and the quality of outcomes you help users achieve are harder to work with than tracking output, and they rely much more on human interpretation than the former does.
It’s therefore all too easy to focus on those and neglect to measure those that might not look as good in a report but do provide far more actionable information.
The Bureaucratization of Measurement
There comes a scale at which KPI culture transforms from managing performance to compliance. It’s where measurement has become bureaucracy, full stop. You know when you’re getting there through certain signs.
Measures with no clear strategic explanation, but which were introduced into the monthly report so recently that it feels too dangerous to try to take them out again.
KPI meetings in which nothing much gets followed up afterward.
Reports that are seen, signed, stamped, and stored away.
People who know the targets they have to meet, but don’t know how they relate to anything else that the company does or wants to achieve.
As soon as measurement has begun to develop a life of its own, it loses any reason for it to have had one in the first place: to promote action that enhances performance.
Its purpose becomes simply to ensure the self-preservation of a framework for producing reports that prove reports are being produced in a seemingly organized way.
As the system appears to be very active, the system is also largely uncontested – the dashboard is refreshing, and the monthly reports are being circulated; therefore, it is clear that something is being controlled.
How Good Metrics Quietly Become Bad Incentives
The most insidious part about KPI saturation, perhaps, is what it does to our behaviour in the long run, even when the metrics were a well-intentioned effort to start with. Every single metric, the second that it’s tied to an evaluation, begins to drive behaviour. That is, after all, its job, but that’s different from improving the system the metric was designed to measure.
People get good at gaming the system to produce the number. We optimize around a specific KPI. We cut corners to hit the number. Risk-aversion increases because a miss, however minor, kills the score, and suddenly we have a workplace perfectly optimized for the appearance of performance while the real work goes unimproved.
The issue isn’t individual greed or lazy employees. It’s the predictable consequence of over-measuring and under-trusting. It seems like, by now, we’d have learned that when we tie evaluation to everything, the only smart move is to play the game, not do the work. Metrics were intended to indicate how we could improve things. In systems of over-measurement, we have replaced honest indicators with carefully managed signs of activity: noise dressed up as useful data.
From Measurement Maximalism to Measurement Intelligence: How to Build a Leaner, Smarter System
Start With the Question, Not the Dashboard
The antidote to measurement maximalism, to this idea of just measuring more and more, doesn’t actually mean “measure less for the sake of measuring less.” It means “measure with intent,” and to do that, we need to start with the most important thing, and most organizations get this part wrong more often than they get it right.
What typically happens, if you look at most organizations’ KPI frameworks and dashboards, is that they approach the creation of those frameworks based on the answer to “What’s out there for me to measure?”
Therefore, we assess what data we have, what the analytics tool can tell us, and what can fit on the dashboard, and we build the dashboard from what’s available. That feels pragmatic, and the dashboard ends up looking great, and, by and large, we’ve approached the problem backward.
The real right thing to do is actually to come back to another question: “What decision am I trying to use this metric to inform?”
If the question of what decision I’m trying to make doesn’t have a really, truly clear answer, then maybe that metric shouldn’t be on the dashboard. Maybe that KPI, that part that I’m measuring, just doesn’t belong on the dashboard unless there’s a decision tied to it, a meaningful way for me as a decision-maker to consume it.
If it’s not driving a decision for someone, it won’t serve as a helpful management tool; it will become a floating, isolated data point, and that is the real distinction between measurement intelligence and measurement maximalism. It’s not “how much do we measure” but “how focused do we measure,” which ultimately comes down to a company having more than just an objective in mind.
It should have a specific decision for every metric or set of metrics, a specific type of decision it’s going to serve, and a specific accountable owner who is expected to act as a consequence of observing the metric. If any of those three requirements are not met, perhaps that metric should be dropped.
The KPI Audit: A Framework for Cutting Without Going Blind
The practical dilemma for most people is not a lack of clarity regarding having too many KPIs; rather, it’s determining how to reduce them while maintaining visibility on the really important items. The best tool for identifying which KPIs to prune and which to keep is a system, and it does not have to be overly complicated.
For every measurement indicator on every dashboard, ask:
Does this measurement relate directly to a company’s strategic goal (the ones that really are on there)?
Does it drive a decision and prompt a quick response as the score fluctuates?
Can an informed employee explain what the measurement is tracking and why it’s important?
Would a person use this measurement to make better decisions than they otherwise would without the measure?
If the answer to any of the questions is a simple no, it means the measurement warrants significant evaluation. If you answer simply no to two or more of these questions, you could also answer simply no, since it means your measurement is unlikely to be as productive as you had initially hoped.
In addition to asking these three questions, there is a logical framework for determining how many KPIs the organization requires at different levels, which may serve as initial talking points among interested parties.
Disclaimer:The following numbers are not set in stone and are not end-all be-all guidelines; they should serve only as a starting point for a theoretical discussion on cutting down KPIs in an environment that sprouted so many of them that you don’t even know what each one tracks. They do not represent a cookie-cutter suggestion or a golden standard – they are merely the beginning of a conversation. Each company and industry is different and requires distinct efforts to maximize the use of KPIs.
For practical guidelines or a detailed plan, tailored to a specific organizational situation, get in touch with us here: https://kpiinstitute.org/contact-us
With that out of the way:
A) At the individual employee level, there is some empirical support for certain limits. Often, that revolves around the idea that one person should own no more than three KPIs at any level. If that exceeds 3, few owners can dedicate time to it, and possession skews towards fiction rather than fact.
B) At the team level, you may set up team dashboards of about ten to fifteen KPIs, as long as every measure in the dashboard is genuinely owned, assigned to a goal, and the KPIs are regularly and critically examined and are not simply noted and filed away.
C) At the organizational level, usually around no more than six to ten genuinely strategic KPIs are to be shown to executives, not out of some arbitrary constraint but out of an awareness of the cognitive limits of human beings when focusing on complex and interrelated decisions. Numbers greater than 6 – 10 make it more of a data repository than a viable system. People start focusing on quantity rather than quality.
Treat Metrics as Signals, Not Verdicts
The cultural shift that distinguishes high-performing organizations from those drowning in the metric-maximalist world is this: high-maturity organizations do not use metrics to replace judgment; they use them to inform judgment. In a KPI filled organization, metrics are always verdicts. If a metric is green, things are okay. If it’s red, someone is failing.
Teams spend their time explaining away numbers rather than understanding the system that generated them. Leaders look at averages and move forward regardless of what the averages mean; anomalies are not explored because there are too many data points to examine. In a measurement-intelligent organization, those same numbers initiate dialogue rather than conclude one. An unusual move in a metric isn’t a verdict; it’s a prompt to go deeper and inquire further, to understand what caused the movement.
Why has a number moved?
What is it signaling about the underlying system?
Is it still the thing being measured?
What action is really indicated?
Qualitative insight is as important as quantitative data, not less. The number may signal a shift, but it usually doesn’t say what to do about it. The judgments of people closest to the work, who understand the context far better than any dashboard could, are considered insights, not distractions.
Moreover, accountability in a measurement intelligence organization remains human. Decisions don’t get handed off to dashboards. They are held by humans, with dashboards as backup.
If It Doesn’t Drive a Decision, It Doesn’t Belong
The simplest reframing any organization can adopt for serious measurement culture improvements is this: a KPI that doesn’t inform a decision is not a KPI – it’s noise.
Much like writers kill their darlings when they remove words, sentences, paragraphs, or entire chapters, businesses should do the same with KPIs. Not every metric residing in your data system today will survive or should survive.
Some will be metrics that only made sense three years ago when an entirely different priority was at play. Others will be internal departmental KPIs quietly slipped into the executive dashboards. Many are vanity metrics that are too vain to keep. Pruning these will enable you to stop operating blind and start to see clearly for the first time in what feels like a long time.
Final Thoughts
Measurement is not evil.
Measuring things up is a response that makes complete sense – that impulse to understand if whatever you are up to is actually happening, to detect what might soon become an acute problem, to gauge what may be a slept-on trend, and to want to hold people to account for results.
Yet, when measurement takes on a life of its own, it seems more crucial to do measurement for its own sake. Dashboards become more of a concern than just tools that support decision-making. Teams devote so much effort to feeding some form of measurement tool to demonstrate progress toward the desired end that the effort shifts away from running the business to the business of measuring the business.
Metrics are powerful tools. Used with intention, they drive the kind of accountability that genuinely changes things. However, they are terrible masters, and the organizations that remember the difference (that keep humans in charge of judgment while using data to sharpen it) are the ones that turn performance measurement into a real competitive advantage.
Measure less to understand more to decide better.
**********
Looking to strengthen your expertise in KPI design and performance measurement? Gain practical knowledge, proven methodologies, and globally recognized certification through The KPI Institute’s Online Certified KPI Professional.
