“The greatest danger in performance measurement is not seeing too little, but believing you’ve finally seen everything.”
Nine KPI Lessons, One Underlying Problem
Throughout this series, we have explored what might seem like nine separate problems with measuring performance. Each article examined a different symptom, manifestation, paradox, or unexpected consequence of using KPIs to understand a complex organization. Viewed on their own, these issues seemed only loosely related. When viewed collectively, however, they tell a surprisingly coherent story.
Our journey began with the KPI Theatre, where we observed how the act of measurement changes behaviour. Once individuals know they are being observed, they instinctively begin to focus on what is visible. This is not necessarily because they are engaged in manipulation or deceit, but simply because human attention is finite and people attend to that which attracts attention. Slowly, performance shifts from being good to looking good.
Then, we examined Goodhart’s Law to see what happens when that observation becomes a target.
What began as a useful indicator of performance becomes, over time, the objective itself. People stop asking whether the organization is serving its purpose and start asking if the numbers look OK. The simple act of optimizing has quietly replaced the process of understanding.
It would seem natural that if one metric blinds you, then two will be better, but that then begets something called KPI Saturation – when everything, every function, every initiative, every strategic priority, and every operational process is represented on a dashboard.
Everything becomes so overwhelming that we are blinded by what is actually present.
Information overload has given way to attention scarcity, and when attention becomes a scarce organizational resource, metrics themselves become valuable. In The Politics of KPIs, we examined how indicators gradually evolve beyond measurement tools into instruments of influence. As dashboards grow more central to an organization’s decision-making, knowledge that is difficult to reduce to a number becomes marginalized and increasingly ignored.
Institutional knowledge, relationships, craftsmanship, intuition, judgment, and the trust required for collaboration – all fall off the strategic radar as information that cannot be easily expressed in numerical terms disappears from the conversation, as was illustrated by KPI Memory Loss.
Organizations do not necessarily fail because they lack information. Rather, they fail because certain forms of knowledge no longer receive attention, or, even worse, are actively dismissed by those who treat them as irrelevant data points. The effects of these trends, however, do not remain confined to the organization itself. The final issues that we uncovered in this series impacted the very people who were employed to deliver on those numbers.
In the KPI Identity Trap, we witnessed how those being measured can become so completely identified with the metrics by which they are assessed that they forget to ask whether they are doing worthwhile work and simply focus on what the dashboard or scorecard shows. It could be seen that the last, and perhaps most fundamental point in this series – KPI Blind Spots – is almost a natural consequence. At best, a dashboard tells us something about the reality of our situation and, at worst, it tells us an unintentional lie.
Yet the lie is never told; it simply emanates from what is left unsaid: the part of reality deliberately omitted to focus on what is believed to be most important.
Realistically, none of these were separate problems at all: behaviour, targets, information overload, politics, organizational memory, personal identity, blind spots. These are not independent phenomena, but simply different facets of the same flawed logic. A logic that proceeds on the basis that reality, as it is currently expressed in KPIs, continues to surprise us only because we have not yet measured enough.
This is what can be called the Completeness Fallacy, perhaps the most insidious belief underpinning modern performance measurement – the belief that performance (and measurement) can eventually be made complete.
Why Every Surprise Seems to Demand Another KPI
These are scenes every seasoned executive has lived out.
An unforeseen event derails a critical business process.
A major customer segment starts leaving faster than anticipated.
A manufacturing defect escapes quality control.
A major project comes in late even though all milestone flags are green.
A cyber attack bypasses the security systems that should have prevented it.
A high-performing, tenured employee quits with little notice.
The post-mortem begins, with executives staring at dashboards and data visualizations, trying to pinpoint where the red flags should have popped up and when someone should have noticed that something was going terribly wrong.
One question invariably surfaces: “Didn’t we have a KPI for this?”
Sometimes the answer is yes, and it was simply ignored. Most of the time, more frequently than many organizations are comfortable admitting, the answer is no. Thus, the logical conclusion seems obvious.
“Let’s add one.”
On its face, this is completely reasonable. Every failure is an opportunity to refine the measurement system. If an important early warning was missing, then the dashboard should be augmented to track it. It often is the right solution, truth be told.
The danger emerges when adding a new KPI becomes the default response to unexpected events, because the next time the unexpected happens (and it always does), another KPI is added. Then another and another.
The dashboards get bigger, the reports get longer, the task manager bloats, and the analytical tools become more sophisticated. Still, unexpected events persist doggedly. Each surprise seems to reinforce the idea that something else must still be missing, driving the organization toward the impossible goal of complete measurement.
This is what can be called the Completeness Fallacy.
The Completeness Fallacy is the mistaken belief that all organizational surprises stem from missing dashboard metrics and that simply adding enough KPIs will eliminate uncertainty entirely.
Complex organizations aren’t like simple machines, made of gears & cogs. They are living, breathing systems made up of people, incentives, cultures, relationships, informal networks, dynamic markets, shifting customer expectations, evolving technologies, and countless interactions that cannot be fully predicted.
Every solution creates its very own new problems. Every intervention changes the system it attempts to measure. Having a complete representation of all possible futures is impossible. Ironically, as organizations pursue completeness, they move further away from true understanding because the question asked subtly changes.
We stop asking:
“What have we misunderstood?”
Instead, we start asking:
“What KPI are we missing?”
These two questions sound similar, but there is a profound difference. The first question is about understanding and assuming that the reality of the situation is more nuanced than the dashboard’s representation. The second question focuses on measurement and suggests that the dashboard just needs an additional piece.
The Endless Expansion of the Dashboard
Think about a dashboard that includes 50 carefully chosen KPIs. A few weeks pass, and then a completely unanticipated problem arises that those 50 KPIs couldn’t have foretold.
“We need to add one more KPI!” – Leadership.
The dashboard grows to 51 KPIs. A few months later, an even greater shock arises. A new KPI is added. The dashboard now has 60. Soon 80. Then 100. Eventually, someone gets tired and asks, “If we have 100+ KPIs on our dashboard, how did this still catch us out?“
It’s a strange psychological paradox at work here.
On the one hand, leaders think “everything that is important must be on the dashboard.”
On the other hand, when something isn’t on the dashboard, it is, at least initially, discounted precisely because it is unmeasured.
Whenever reality disappoints, we try to achieve completeness by expanding the dashboard. As the dashboard grows, our confidence in it grows. That, in turn, makes the next surprise that happens all the more baffling. People begin to wonder in dismay how all of this can be happening, since they are measuring everything.
Except they are not. They never can and will never be able to. No dashboard can perfectly mirror reality; reality is always larger than its reflection.
The danger isn’t what dashboards leave out; the danger is that we forget what they have to leave out.
Babies & Video Games: Why More Doesn’t Always Mean Better
The Puzzling Perplexity of Predicting Progeny
The experience of dealing with babies is a universal (if not always enjoyable) one. Say your baby – a perfectly healthy baby, no less – is inconsolable, though not crying as a result of any obvious issue. They have been fed, cleaned, kept comfortable, and healthy, so why do the tears persist?
You hand the child a colorful toy, and for a brief moment, the wailing subsides before resurfacing with renewed vigor.
“Maybe the problem is simply that we don’t have the right toy!” and so you rush out and acquire one, only to have it achieve the same limited result. You then acquire another that sings, and another that flashes. The child continues to cry, and your resolve is steadfast: there has to be one “right” toy out there to soothe their little agitated spirits!
This process seems almost logical, and the conclusion (that another toy is just around the corner) feels almost automatic. After all, if there were a truly suitable toy, the baby would just stop crying. Right? RIGHT?
This assumption misses a crucial detail: the baby wasn’t looking for another toy at all. Maybe they simply wanted to be held, or was bored lying in one position for too long, or maybe they wanted someone to talk to them, or just to feel the comfort of their parents’ closeness. The parents didn’t need a better toy; they needed to understand the baby.
In the course of our lives, we will all learn an invaluable lesson. Sometimes the quickest, easiest path to resolution doesn’t lie in introducing a new component or finding a missing element, but in paying more attention to the subject of our concern.
Organizations are remarkably similar. Organizations often take a very different, and rather analogous, approach. Every unforeseen problem requires a new key performance indicator (KPI); every anomaly requires a new dashboard; every overlooked blind spot demands a new metric.
It’s quite possible that the problem isn’t that another metric is needed. It might be that the organization needs to better understand what it is trying to achieve in the first place. Instead of reading reports, managers might need to speak with their employees more often. Instead of filling out surveys, customers might want to talk to real humans to air their frustrations.
For example, to understand a problem in production, supervisors should walk the factory floor rather than stare at a production-tracking dashboard. While the dashboard asks, “What else can we measure?” reality is asking, “Have you truly understood me?” That distinction is at the heart of the Completeness Fallacy.
The Curious Compulsion to Continue
It plays out just as clearly in an area where organizational management can’t possibly be expected to surface: video games.
Any person who plays role-playing games or massively multiplayer online games knows that optimization quickly becomes a way of life if left unchecked. Let’s assume you load one up, make a character, and it just isn’t putting out the damage-per-second (DPS) that you were aiming for.
The logical first step, you assume, is to get a DPS meter, take some measurements, and look for the source of the deficit.
The meter tells you that you’re falling short of the damage output of everyone else on your team. You now have an answer to your problem. Or do you?
Upon inspecting your equipment, you notice that some items are suboptimal, so you immediately replace them with better ones.
You head back into the game’s content, only to see a minuscule difference in your damage output. There must be some other missing factor, something else you did not account for yet again.
You realize your equipment isn’t enchanted, and so you spend hours painstakingly applying the most potent enchantments possible.
The result is still marginal. You now invest in better gems, talent points, food buffs, consumable potions, and specialization changes. Eventually, you end up with half your screen clogged by meters tracking DPS, timers, combat logs, raid frames, cooldowns, boss warnings, and who knows what else. Still, your character’s damage output does not noticeably improve.
Add-ons are useful, but the most obvious limiting factors, such as positioning, decision-making ability, encounter awareness, or knowing when not to attack, cannot be directly measured by another number or add-on. These qualities must be learned through trial and error and by recognizing patterns. However, the more information cluttering your screen, the easier it becomes to miss the giant boss looming directly in front of you.
The Misguided Mission to Measure More
Organizations can find themselves facing this type of music in exactly the same way. The dashboards continue to grow, the reports become more elaborate, the metrics increase, and the alerts pile up, giving executives an enormous volume of information to analyze.
In practice, the actual amount of real-world understanding often advances at a far slower pace because it becomes so easy to confuse the process of measurement with actual understanding:
You can measure customer satisfaction and learn that it’s trending downwards. However, you can’t measure the sound of disappointment in a customer’s voice on a support call.
You can check a productivity dashboard, which will tell you that the project velocity has decreased, but it won’t reveal the engineer who’s afraid to challenge a deadline they know is unrealistic.
You can monitor employee morale via an engagement survey, but that can’t capture the quiet moment in the hallway, months ago, when confidence eroded just a little bit more.
You can assess sales conversion rates and identify where prospects drop off, but you can’t measure the trust that was lost in a single rushed conversation.
Wisdom doesn’t grow automatically simply from an increasing quantity of observations. Wisdom is built through an interpretation of those observations within the human context in which they occurred.
This is the true danger of the Completeness Fallacy: it persuades leaders to believe that the missing piece of the puzzle is simply another data point, and that there is never a need to stop looking at data and start listening to their people.
From Hospitals to Software: How Industries Share the Completeness Fallacy
The form in which the Completeness Fallacy shows up may vary significantly between industries, but the core principle doesn’t: whenever the world spits out an answer that surprises us, we start hunting around for some new measure, instead of asking a question: “Are we perhaps measuring the wrong thing to begin with?“
Healthcare: Measuring Patients While Missing Care
Modern hospitals amass vast quantities of data: Patient wait times, Bed capacity, Readmission rates, Length of stay, Medication adherence, ED wait times, Length of procedure, and Infection rates, among many others.
They’re all crucial metrics, but none of them fully explains why some patients with similar clinical profiles take weeks to recover while others take days. Many of the things that drive outcomes – whether the patient actually grasps what the doctor just said, has someone to nudge them about medications, or simply trusts their physician with what’s bugging them – are tough to fit into a standard spreadsheet.
We see a bad readmission outcome and naturally reach for a new quality measure. Sometimes that makes sense, but more often than not, we need to make sure the conversation was handled properly, not simply as a way to create another quality benchmark.
Manufacturing: Perfect Machines, Imperfect Systems
Manufacturing organizations usually have extremely detailed operational dashboards: Machine usage, Cycle time, Defect rates, Overall Equipment Effectiveness (OEE), Scrap rates, Downtime, or Energy usage, to name just a few.
When product quality drops inexplicably, the first reflex is often to monitor it even more closely. Very often, the mentality is “time to add another production KPI and slot in another quality check.”
Yet, investigations often find that technical issues are not at the heart of the matter; rather, the heart itself is gone.
Production targets subtly discourage the workforce from signaling minor deviations until they become significant ones.
None of these had anything to do with a lack of data regarding the machinery – these were all human systems impacting technical ones. What was needed wasn’t more sensors but a better understanding of those humans operating them.
Software Development: Measuring Productivity Without Seeing Complexity
Software teams today arguably measure more things than any team has before: Sprint velocity, Story points completed, Lead time, Cycle time, Deployment frequency, PR approvals, Bug counts, Code coverage, or Incident response time.
When the wheels are slowing unexpectedly, leadership frequently adds a new metric for engineering teams to work with.
“Maybe we need more code!”
“Maybe the reviews are taking too long!”
“Maybe the deployments aren’t happening often enough!”
Maybe the real problem might be that a legacy architecture is now extremely difficult to maintain. The team has quietly accumulated technical debt over the years and is now spending much more time understanding systems than developing features, and on top of that, cross-functional communication has broken down.
The biggest thing stopping the team from shipping may not be an absent KPI, but decades of accrued complexity that only a truly seasoned engineer can even see. Such nuance cannot be encapsulated in a dashboard.
Aviation and High-Reliability Organizations: When Safety Lives Between the Metrics
Few industries are more serious about measurement than aviation. Aircraft systems generate vast amounts of operating data, which are measured and remeasured with exceptional rigor.
Flight schedules are closely monitored.
Safety incidents are rigorously documented.
Procedures are formalized.
Performance is routinely assessed.
Nevertheless, aviation professionals know a secret that few leaders recognize: if no safety event has occurred, it doesn’t mean that safety is present.
The organization could show perfectly good safety performance on its metrics, while easily and effectively hiding its own psychological limitations regarding the issue from itself. Pilots may be unwilling to speak about hazards. Maintenance crews might be reluctant to acknowledge near-miss events because they might feel embarrassed or appear incompetent. A Junior team member might have no idea how the process could be made safe and would have no incentive to raise such concerns.
The overall safety indicators might be beautifully and blissfully green right up to the day of a disaster. When such a disaster occurs, the investigators rarely attribute the failure to a single additional KPI or a particular missing index or tool. Instead, they generally attribute the disaster to communication failures or human errors that may have predated the measurement systems.
The organization didn’t lack yet another metric – it lacked a deeper understanding of the system it thought it had measured.
In all 4 of the industries we’ve showcased, the situation repeated itself:
Disasters occurred
Organizations assumed they must be missing a metric
The metrics were expanded
They were surprised again and again
Such scenarios don’t happen because leaders lack the intelligence to be able to design and implement appropriate measurements, but because no complex system, as a matter of principle, can ever be as large as its own representations of itself. No dashboard can become the entity being measured.
When organizations reach that stage, they focus so intently on measurement devices that they neglect the journey and miss where they are going.
Final Thoughts
Looking Through the Windshield Instead of at the Dashboard
Never before has there been so much readily accessible operational information available. Never before has there been a way to see performance across continents in real-time, spot emerging trends within minutes, or turn vast amounts of raw data into simple, intuitive pictures. Organizations have become faster, more coordinated, better informed, and ultimately, more agile.
Problems arise only when dashboards stop being tools for understanding and start becoming substitutes for it. This is, ultimately, what’s dubbed the Completeness Fallacy.
It has nothing to do with whether KPIs are useful or useless, or whether measurement systems need upkeep or not. Rather, it is the false belief that uncertainty can eventually be engineered out of the organization by judiciously selecting enough indicators.
In modern-day organizations, every KPI answers one question but begs another. Every bit of extra visibility alters behaviour, thereby changing what the measurement actually means. Performance measurement isn’t a race towards a state of completeness. It’s a never-ending conversation between what can be measured and what requires only observation, judgment, curiosity, and experience.
Dashboards are the thing that should inform a conversation, never replace it. Think of them as a car windshield. It has never been designed to capture every detail of the road ahead. It won’t reveal what is hiding behind every building, nor will it show every hidden pitfall lurking around the bend. Its purpose is to provide just enough visibility to help us navigate while, at the same time, reminding us that the road itself deserves our attention.
Modern performance dashboards have precisely the same role. They are not the organization, but the windows through which we may view organizations in a richer, more dynamic manner than any collection of KPIs can adequately convey.
If we start to look at the windshield instead of through it, we are liable to lose sight of where we are going, and that may well be the most profound irony of modern performance management:
We have never measured more, and with better tools. At the same time, we have never depended more on the interpretation, conversation, experience, and context provided by the people powering the soul of our workplaces.
*********
Looking to strengthen your expertise in KPI design and performance measurement? Gain practical knowledge, proven methodologies, and globally recognized certification through The KPI Institute’s Online Certified KPI Professional.
Picture the following: a customer service team boasting average response times under two minutes. Customers are always getting responses right away. Every target is being met. Yet, customers keep complaining, and complaints are only growing.
How is that possible?
After investigation, you find that while customers are receiving rapid responses, those responses may be doing little to resolve their issues. The team members have been incentivized to close tickets quickly because that is what’s being measured. The original purpose (making customers happy) is now secondary.
This can be a very common issue within companies of any size. KPIs that once represented success are slowly becoming success itself. They stop asking “are we accomplishing what we intended to accomplish?” and start asking “are we meeting the target?” The difference between these questions might seem negligible, but the implications can be significant.
KPIs have a place and are indeed beneficial. Companies need a way to evaluate performance, track progress, and understand where improvement is needed. Without measurements, we operate based on assumptions and intuition alone.
The difficulty is that there are many things that organizations care about that are not easily measured. A single number can’t capture employee loyalty. Employee engagement isn’t the same as a survey score. Collaboration, trust, innovation, and long-term business value just don’t fit on a dashboard. As a result, companies use leading indicators, which we assume represent a desired outcome.
Response time is often seen as a sign of good customer service. Attendance is assumed to show employee commitment. Productivity numbers are assumed to prove effectiveness. This is okay to an extent; in fact, these are often necessary indicators to track. Problems arise when the leading indicator outweighs the outcome it was originally designed to represent.
Economist Charles Goodhart explained this concept best in a statement now known as Goodhart’s Law: “When a measure becomes a target, it ceases to be a good measure.” This may sound academic, but the underlying concept is easy to grasp. The moment people are measured, rewarded, or punished by a metric, they naturally seek to optimize for that metric. This optimization might increase performance, but sometimes it only improves the metric.
Consider training for employees. We often measure learning by tracking whether training has been completed. Seems fair on the surface – if an employee completes the training, they are surely learning, right?
Well, not necessarily. When the number of completed trainings becomes a target, the focus shifts.
Employees quickly click through > managers ensure there’s 100% completion before deadlines > dashboards turn green > knowledge retention, skills development, and behavioural change stagnate. The company succeeded in increasing the number but made little to no progress on the desired outcome.
This trend plays out across various industries and sectors. Salespeople push for revenue through deep discounts, thereby impacting long-term profitability. Marketing campaigns aim for engagement numbers even though engagement might be disconnected from real customer value. Project teams celebrate on-time delivery even though the project might not provide tangible benefits. The issue here is not that the metric is necessarily incorrect. The problem is that it only represents a piece of the whole.
A useful analogy for KPIs is to think of them as road signs instead of destinations. Signs tell you if you’re going the right way, but we don’t mistake the sign for the destination itself. Organizations often make this mistake:
Customer satisfaction is not a survey score. ❌
Productivity is not a speed metric. ❌
Attendance does not show employee contribution. ❌
These are signals used to help us understand reality, not reality itself. This difference becomes even more critical when organizations prioritize results while ignoring the actions that lead to those results. A revenue number from last month tells you what has occurred; it doesn’t tell you why. A customer satisfaction number indicates the outcome of a given interaction; it does not show the behaviour displayed during that interaction. By the time a revenue number changes, the behaviours that affected it may have been in place for weeks or months.
That’s why increasingly successful organizations are beginning to differentiate between outcomes and the actions that produce them. Outcomes serve as scorecards, letting you know where you stand. Actions and drivers help you understand how you got there and what you should do next. If leaders focus solely on the scoreboard, they are more likely to react to events after they have occurred. If they understand what causes the score to change, they will be able to influence future outcomes before they become problems.
Through this shift in thinking, we can reach an important conclusion: not all KPIs should be created equal. Some measures help us assess progress towards desired outcomes; others serve as proxies for those outcomes. For leaders, the biggest challenge is recognizing which is which. If the measure becomes the mission, organizations risk optimizing for the numbers rather than for the results they represent.
How Proxy Metrics Quietly Take Over
If most organizations know that KPIs are just indicators, how do so many organizations end up managing the indicator rather than the outcome?
The simplest reason is that proxy measures are convenient.
It’s often hard to measure the actual outcomes we want to influence. For example, real outcomes can take years to show any real results, often can’t be easily isolated from other variables that also affect the outcome, and usually don’t fit well on a dashboard. Proxy measures, on the other hand, are easily and readily available to be captured, reported, analyzed, and benchmarked.
Consequently, organizations tend to get caught in a cycle. Instead of asking “what would indicate we are truly successful?“, they ask, “what data do we already have?“. The available metric slowly evolves into the performance measure.
While this sounds relatively harmless, it quietly creates a shift. Individuals stop focusing on how well they are achieving the actual outcomes and begin talking about achieving the numbers on a dashboard.
Discussions focus on “have we hit the target?” rather than “have we made real progress toward achieving our goal?” The indicator becomes the lens through which we interpret performance, even when it tells only part of the story.
This isn’t to say proxy measures are useless; many of them can provide helpful insights. It’s simply assuming that the proxy and the outcome are one and the same, which is the problem.
For example, completing a training course may indicate that learning has taken place, but it doesn’t confirm any real change in capability. A high customer engagement rate can indicate interest, but does it lead to customer value? An increase in sales calls doesn’t always mean more quality customer conversations were held. These proxy measures may be useful in isolation, but they don’t tell the full story.
Unfortunately, once a metric is valued, people tend to drive it. Usually, this is not due to manipulation or intentional bad practices; it is simply how human beings behave. If a KPI target is linked to rewards, positive feedback, promotions, or performance reviews, people will make sure to meet this metric regardless of whether it aligns with desired outcomes.
The problem then becomes that an increase in a KPI may not necessarily lead to the desired increase in the outcome. There are countless examples throughout history of this behaviour, such as using the enemy’s body count as a measure of success in wars. Such a heinous & vile metric was easier to achieve than actual strategic objectives, and, eventually, simply measuring the metric became the objective itself. The measure dictated the outcome, rather than the outcome shaping the measure.
Now, whether we look at armies or organizations, both can fall victim to the same thinking pitfalls, for they are comprised of people who often err on what is “easier”. Leaders can start managing what’s easy, rather than what’s important.
In a much less combative example, take the instance of a decrease in cost-per-lead: at face value, it doesn’t make much difference if lead quality falls dramatically; an improvement in customer service response times does little if customers still have the same unresolved issues, and a team celebrating meeting all its targets still doesn’t achieve its business goals. Each example shows that the KPI rose or fell as intended, but the desired outcome didn’t.
Perhaps the most intriguing part is that organizations and their people usually know the source of the disconnect:
The sales team knows when target numbers promote busywork
The customer service department knows that quick responses are not the same as solving customer problems
Managers know that an increase in attendees does not necessarily correspond to greater commitment or contribution
However, when people feel a sense of control and certainty that a KPI is moving in the right direction, it becomes difficult to abandon the number, even if we know the real outcomes aren’t shifting as desired.
Numbers, nonetheless, seem more objective and reliable. They are concrete and clear, and they appear to remove uncertainty and complexity from a situation. Clarity, though, is not always accuracy.
A dashboard displaying green lights may suggest great progress, while unseen problems begin to fester beneath the surface of these simple indicators. As an organization becomes adept at performing the actions that achieve the highest success scores on a given metric, it simultaneously develops considerable inertia in achieving its real objectives.
This is why mature performance management systems do not focus on individual metrics, but rather on the overall view. A mature system must incorporate a mix of qualitative data alongside quantitative metrics, so that no individual KPI carries too much weight in determining perceived success.
The real question is not whether there should be proxy metrics at all; it’s whether they are remembered for what they represent. If leaders forget what a proxy metric is supposed to indicate, an organization will spend its energy improving the number rather than the actual desired outcome.
The Five Most Common KPI Traps in Modern Organizations
This quest for proxies seems to manifest itself in infinite ways, yet it follows the same several templates that recur over time, across industries and across hierarchical levels. Although the metrics might vary widely, the error appears eerily similar: the metric eventually succeeds in displacing the thing it was intended to measure.
Response Time Replaces Customer Care
Many customer service teams monitor response time for good reason. Customers typically appreciate quick communication.
The issue is when that speed becomes the primary goal. A team might respond to every single inquiry within minutes, but the response could be generic and fail to resolve the issue. Customers are acknowledged quickly, but still require multiple touchpoints to reach a solution.
This looks good on paper, but in practice, it increases customer frustration. Response time is an important measure, but it isn’t customer service. Customer service is all about understanding problems, solving them, and generating positive experiences. Speed may well be an important factor in achieving these goals, but it alone cannot do so.
Engagement Replaces Value
Engagement has emerged as perhaps the most ubiquitous performance measure in the digital age. Businesses track page views, click-throughs, comments, shares, downloads, logins, and a million other interactive behaviours. Such figures are often collected automatically and can be updated in real-time.
The problem is that this engagement does not necessarily mean any value is being created.
Some content receives millions of page views, while its consumers gain minimal new information. A few software platforms log millions of user logins – their consumers remain stuck performing rudimentary tasks. Several meetings involve many staff members, yet only a handful contribute to improving outcomes.
Engagement does not necessarily mean useful things are happening. It signals that people are attentive. If organizations focus solely on engagement, they create organizations that focus on visibility.
Productivity Replaces Effectiveness
One of the oldest and most frequently measured indicators of performance is productivity.
The number of tasks performed, phone calls made, e-mails sent, reports generated, and tickets closed can tell you something about how busy things are and about operational efficiency. However, you should never confuse activity with effectiveness.
One salesperson can be two or three times as active (in terms of calls made) as another, while identifying far fewer useful sales opportunities. One project team may tick off all the task items on their schedule without having solved the problem the project was designed to fix.
Productivity asks, “How much work got done?“
Effectiveness asks, “Does it matter?“
Organizations that focus on productivity often become incredibly busy without ever becoming more effective.
Attendance Replaces Contribution
One of the easiest measures to monitor is attendance.
People either turn up or they do not. The measurement of contribution, however, is far more involved: someone can attend every meeting and add nothing, whereas another may contribute only two or three times, yet those points may be instrumental in forming key decisions.
It may also be the case that an organization equates attendance with contribution when, in reality, contribution levels depend on involvement, knowledge, collaboration, and the ability to solve problems. Attendance is a good operational measure. That said, it is NOT an indicator of success.
Output Replaces Outcomes
The most frequent KPI pitfall is the confusion between outputs and outcomes.
Outputs are the products an organization puts out.
Outcomes are the effects of these outputs.
Although obvious when articulated, it is often lost when trying to measure things.
Think of a facility team whose job it is to clean an office building. What the facility team measures might include the number of floors cleaned, the time spent cleaning, or the amount of cleaning supplies used. These are all outputs because they show activity. The number of floors is an output; the number of floors scrubbed (to the point they were clean and didn’t feel sticky) would be an outcome.
What if the employees continue to complain that the floors are sticky? The output numbers suggest the team is successful, but the outcome proves otherwise.
The same logic applies to training programs, change management initiatives, marketing campaigns, and transformation projects that are measured by training completion, logins, impressions, and milestones. The output metrics tell us that we did things, but the outcomes measure whether we actually made anything happen. Both are needed.
When we are so focused on outputs, however, we run the risk that they become the sole measure of success, so the team can meet every goal, complete every task, and satisfy every reporting requirement but do absolutely nothing. That’s why there is such risk associated with proxies – they allow us to progress on paper while standing still.
What High-Performing Organizations Measure Differently
At this point, it may sound like the answer is just to get rid of KPIs entirely. Far from it. The matter of fact could not be farther from the truth.
While organizations need measurement, leaders need visibility into performance, and teams need feedback to understand whether their actions are moving the organization in the direction the leadership intends.
The problem is not measurement itself; the problem is making sure the measurement is connected to the thing it’s supposed to be measuring. High-performing organizations understand that KPIs are learning and decision-support tools, not outcomes in themselves. They use metrics to understand performance, and they avoid the urge to turn a metric into an outcome.
I) One of the most critical adjustments they make is to separate outcomes from the behaviours that lead to them.
Many organizations focus almost entirely on outcomes: revenue, customer satisfaction, retention, profitability, market share, and similar figures that often top executive dashboards. These numbers are important, but they are also trailing indicators – they tell you what already happened. When customer satisfaction scores start to slip, the underlying reasons may have existed for months. When revenue declines, the factors that led to the drop may have been building for quite a while.
Whilst high-performing organizations do keep a close eye on outcomes, they also identify the behaviours and performance drivers that contribute to these outcomes:
A sales team might be concerned with revenue as an ultimate outcome, but it also looks at the quality of prospects it’s working on, the level of activity its team has-how many calls and meetings-and its closing rate. All of these will affect revenue and allow leaders to spot problems before they significantly impact sales figures.
A customer service team will continue to track customer satisfaction scores, but it will also look at how many times a customer contacts it for a single issue, how quickly agents respond, the quality of communication, and customer effort.
The objective is not necessarily to replace outcome measures with behaviour measures, but to tie them together.
Outcomes tell you where you are, behaviours give you an idea of how you got there, and where you are likely to go in the future. This changes how you use KPIs from simple reporting tools into proactive management tools.
II) Another difference in mature performance systems: these organizations rarely use a single metric for an important organizational objective.
Let’s use customer experience again: organizations often turn to NPS or customer satisfaction scores. These have value, but no single metric adequately describes the concept. It may make more sense to use customer satisfaction metrics alongside retention rates, complaint counts, resolution speed, customer effort, and actual customer feedback.
Each one captures a different piece of the puzzle, which is why they should be looked at together. The same logic applies to nearly every other aspect of the business.
Revenue should be examined along with profitability.
Productivity along with quality.
Employee engagement along with retention and performance.
Efficiency along with effectiveness.
When measures are viewed as interconnected pieces of information, the temptation to optimize one measure at the expense of another diminishes significantly.
III) Lastly, and probably most important of all, high-performing organizations retain an element of wonder about what they might be missing with their KPIs.
They understand that metrics are a form of simplification and allow us a glimpse into the world of perceptions. No dashboard can fully capture customer trust, employee loyalty, innovation, culture, teamwork, or the ability to adapt; yet all of these can be profoundly important drivers of organizational success.
Instead of assuming that every important thing can and must be expressed as a number, leaders at mature organizations accept the inherent limitations of measurement and complement their data with conversations, observations, customer inputs, employee knowledge, and professional judgment.
In other words, they use data, but not as a replacement for decision-making, since the purpose of performance management is not perfect reports but reports that provide a deeper understanding of performance. Such work takes more than merely watching numbers on a screen.
A Simple Test for Every KPI You Use
The risk of proxy metrics is that it is uncommon for a bad metric to be bad to begin with.
They usually begin as rational indicators of important goals and slowly take on a life of their own as companies get increasingly obsessed with bettering the indicator itself. This necessitates periodic reevaluation.
Each of your KPIs should, on occasion, be examined with a basic but critical question: Is this metric still telling us something about our performance, or has it become the performance?
The answer may not be crystal clear, but a few practical questions can reveal a KPI that might be losing sight of the original goals.
What outcome is this KPI supposed to represent?
Each metric should relate clearly to an organizational goal.
If the goal is unclear or hard to articulate, the KPI might be measuring activity rather than progress. One helpful test is the question “Why should we even care about this number?” The answer often highlights whether the metric is still relevant to the desired outcome.
If the KPI improves, does the outcome necessarily improve?
If you can improve the metric without improving the outcome, there is a risk that the KPI serves as a surrogate for something weaker.
Training completion can increase without any skills being gained.
Website traffic can go up without any value being added.
Response times can increase without the customer’s problems being solved.
You should be very wary whenever it’s possible to optimize a KPI independently of an outcome.
What behaviours does this metric encourage?
Performance metrics influence all actions. Some actions will be productive, some less so.
A sales performance metric can prompt positive customer outreach. It may also prompt undue discounting.
An activity performance metric can prompt work, but it may also prompt busywork.
So, the question is not simply whether a KPI triggers activity, but whether it triggers beneficial activity.
Can people hit the target while missing the point?
This issue seems to be at the very core of Goodhart’s Law: if it is possible to obtain the metric without producing the desired result, then the KPI may become the goal.
A lot of the examples mentioned within the article fall into this category – where the team “hit the number” and still made little real progress toward the overall aim. In these cases, other indicators may be necessary.
What important outcome are we not measuring?
Each KPI measures just one dimension of the business. As attention to any specific KPI increases, another aspect of performance will likely fall into a “blind spot.”
Customer acquisition may be analyzed, while customer retention is neglected.
Productivity may be measured, while quality is left out of the discussion
Operational efficiency may be increased at the expense of innovation
The ongoing question of what is not on the dashboard will ensure that important business outcomes do not fall completely out of the organization’s mindshare.
Final Thoughts
KPIs remain one of the most powerful tools for leaders to align efforts, monitor performance, and allocate resources.
With that said, they are but a tool. They break down when an organization forgets the difference between the metric and the outcome the metric is supposed to capture.
A fast response isn’t great service.
High engagement isn’t value creation.
Productivity isn’t effectiveness.
Attendance isn’t a contribution.
Output isn’t impact.
The best organizations remember and manage accordingly; they use numbers to inform judgment rather than replace it. They focus on outcomes while being acutely aware of the behaviours that produce them. They remain attuned to the fact that a helpful metric today can become a damaging target tomorrow.
At the end of the day, a KPI’s value isn’t in proving that we can win at numbers. Its value lies in helping us improve our numbers. That’s when KPIs truly fulfill their potential as indicators of success rather than proof of it.
