KPI Memory Loss: What Organizations Forget When Everything Must Be Quantified
Every organization remembers its numbers: revenue, profit margins, cost of customer acquisition, employee utilization, defect rates, NPS scores, or average resolution times.
Pull open any dashboard, and you’ll see hundreds of highly selective data points meticulously tracking almost everything happening within the business. Organizations today are astoundingly adept at capturing data. However, they often can’t answer simpler questions.
- Why is this team performing so well when they are tracking only average productivity metrics?
-
Why are customers loyal to this account manager?
- Why did our innovation efforts grind to a halt when our key engineer left, even though the KPIs remained unchanged?
-
What, beyond hitting deadlines, contributed to that project’s success?
Some of the most crucial assets any organization holds cannot be conveniently pinned on a dashboard. As conversations naturally become centered around measurable outputs, organizations gradually risk developing a kind of “KPI memory loss” – an inability to recall the details that fail to fit within a given metric.
This is not a criticism of KPIs. Not at all, quite the contrary! Businesses must have these metrics to measure performance, diagnose issues, understand thresholds, and make decisions. The issue starts when metrics become less tools for observing the world and increasingly the world itself.
When Metrics Become Memory
Picture a brand-new manager being hired to run a thriving customer service department. They’ve taken over a fantastic dashboard, their average response time has dropped, customer tickets are being resolved faster than ever, and overall productivity is growing each month. From their perspective, they’ve inherited a picture-perfect operation.
Six months down the line, customer churn is on the rise.
But why? What gives?
After interviewing veteran staff members, the manager learns that agents have stopped investing a few extra moments to build rapport with their customers. All the targets were being met; everything looked fantastic on the dashboard, but they had slowly let the human side of it all slide: those little interactions that helped customers feel like they mattered. There was nothing in the dashboard to indicate this.
Nothing in the dashboard was accounting for this. This is one of the greatest strengths (and biggest weaknesses) of performance measurement: KPIs can highlight things we would otherwise never know, yet they also narrow our attention to an unhealthy degree, turning focus into horse blinders.
As much as an organization obsesses over what it can measure, it begins to overlook everything that it cannot. When people start aiming for a metric specifically, that metric eventually fails to reflect what it was intended to reflect.
You’ve almost certainly seen this play out in a large organization:
- A sales team prioritizes quick-close deals over long-term customer value because its quarterly target emphasizes sales volume.
- A call center has reduced the Average Handle Time (AHT) by ending calls abruptly, leading to more inbound repeat calls from irate customers.
- A software team has achieved a high number of resolved tickets while allowing technical debt to fester in the codebase silently.
- An HR team fills open positions faster by prioritizing speed-to-hire, but the quality of new hires drops, leading to higher turnover within the first year.
- A manufacturing plant reduces production costs by using cheaper materials, only to see warranty claims and customer complaints increase months later.
The metrics look good, but the underlying reality does not. This often has nothing to do with bad motives or intentions, but more with incentives.
Incentives have been at the basis of human behaviour since the dawn of time. Therefore, if success is defined by what appears on a dashboard, people will focus their attention there. Over time, companies develop excellent memories for metrics but an almost complete memory loss for everything else.
What Gets Left Behind?
Try to think of the best colleague you’ve ever had. What was it about them that made them excellent? Were they the emergency adult everyone called to soothe volatile clients before a situation erupted? Maybe they instinctively knew when a project was careening off course. Perhaps they just knew which other departments would be required long before an issue was apparent, or they just knew how to mentor a junior person in the office from scratch.
Knowing all these aspects, we are posed with a series of questions:
- How could you quantify these skills?
- How could you put a number on them?
- How would they feature on a spreadsheet?
It might not be possible. Is it possible? Is it feasible? Now we are left with more questions than we had before we knew about the aforementioned series!
Let’s take a look at a different example.
What about a company trying to build itself on measurable, trackable KPIs and not much else? We have a massive body of work in knowledge management that draws a line between the explicit and the tacit: the former can be written down and shared, the latter can only be understood and absorbed through experience and judgment, in context, through interaction.
There are numerous studies that indicate that organizations that have solely relied on measurable performance-only systems fail to capture value and knowledge, even in areas that are absolutely critical to long-term organizational success.
Interestingly, it’s often the people who do not appear on many charts in any system, or who have nothing visible to put on a spreadsheet, who make the organization successful.
- The experienced cardiac nurse may have noticed subtle changes in the patients’ physical condition much earlier than the monitors do.
- The savvy machinist may hear an anomaly in the noise from an old tool and just know the machine requires maintenance.
- The proficient project manager might have noticed the relationship between two key stakeholder groups deteriorating well before the tangible signs of breakdown were evident.
- The well-versed account manager may recognize that a client is quietly disengaging long before declining renewal rates or negative feedback makes it obvious.
These can be moments where organizational failures are averted long before anyone even sees an indicator on a dashboard. These are moments that create and deliver value to an organization every day, yet remain invisible to most of its people and many of its systems.
The Things Dashboards Cannot Remember
The majority of businesses believe their decisions are based on facts. In reality, they generally base their choices on whatever facts happen to be quantifiable. Culture is one of the clearest examples of such behaviour.
Companies commonly try to measure culture through surveys, retention data, absence rates, and employee satisfaction scores. While such information is useful, culture itself is not a figure. It is actually the unwritten principles and practices that establish whether workers report errors early or cover them up. It’s that thing that makes junior employees feel empowered to question those higher up. It’s that je ne sais quoi that leads groups to readily volunteer their expertise rather than guard it or choose to assist their colleagues, even when no one is watching.
Boiling these activities down to a handful of quarterly data points has the threat of mistaking the map for the land. The same is true of reliance on craftsmanship, mentorship, interest, durability, and expert judgment. Organizations seldom lose these features overnight. Rather, they simply fail to mention them because they stop measuring them and ultimately stop noticing them.
As soon as something is missing from the discussion, it tends to be absent from decisions on the whole. That is possibly the major peril of KPI memory loss: organizations do not intentionally cease caring about what is most important; they become so adept at remembering their numbers that they fail to remember everything those numbers can not tell them.
The Hidden Costs of Measuring Everything
Most companies do not wake up one morning deciding to disregard culture, relationships, or craft. It happens more subtly, often barely perceptible to the senses.
- A new dashboard gets added.
- An additional KPI arrives.
- Quarterly reviews become more number-focused.
- Charts, scorecards, graphs, and trendlines support decisions.
- Conversations turn to the question of what we can measure versus what we ought to be asking.
It appears to be a reasonable transition. At the end of the day, numbers are objective, are they not? They establish commonalities and help control a complicated organization. However, numbers are also a source of our most profound blind spots.
Think of onboarding. Think really well. While it seems prudent for a company to track the number of days before a new employee reaches full productivity, there are typically no measures around building trust with other staff, the organization’s unspoken rules, or the logic behind past decisions. This results, six months and two seasons later, in a productive individual who, by all accounts, repeatedly makes the exact same mistakes the company had already overcome a decade earlier.
The knowledge had existed, scribbled on meeting minutes or stored in the heads of long-serving staff or within an unheard conversation, but it had never reached the recipient in need. This tendency pervades almost every field of work.
- An oil and gas operation may monitor equipment uptime and production volumes with remarkable precision, while overlooking the field operator whose practical experience prevents a minor anomaly from escalating into a costly shutdown.
- A government agency can report on service delivery targets and policy milestones with detailed dashboards, yet fail to recognize the informal relationships between departments that quietly determine whether complex initiatives succeed or stall.
- A real estate firm may measure listings closed and average time on market with ease, while overlooking the seasoned agent whose local knowledge and trusted network resolve problems before they jeopardize a sale.
- A hospital may monitor how long patients wait with a stop clock, yet it would struggle to assess the level of trust a pair of experienced nurses builds.
- A legal firm could chart the time partners log on individual cases with great precision, while ignoring the unstructured mentoring that cultivates new associates from rookies to confidants.
- A manufacturing operation can track its output by the hour, but may miss the insight of the retired engineer who stops a press before it breaks down, preventing a sensor from triggering.
With all of these cases, tangible output may increase; however, the intangible abilities that support that output go largely unnoticed until they can no longer be ignored.
When Efficiency Begins Replacing Craftsmanship
Nowhere may the dichotomy be stronger than in craft. Craft isn’t limited to woodworkers and machinists – there’s an equivalent for every role. A software engineer’s craftsmanship might not be about delivering features as quickly as possible but rather about writing testable and maintainable code. A customer success manager’s craftsmanship might be recalling some tiny, human detail from a conversation with a customer and using it to make them feel deeply seen. These are habits you practice into being, not lessons you teach into being.
Picture two identical table factories.
One rewards everyone for output alone (units per shift). The other one measures output AND craft (the ability of seasoned employees to mentor and teach the younger ones). Thus, the most experienced artisans have time to think of better ways to practice their craft, and they reject pieces they deem inadequate, even if it slows output, while prepping a new generation that comes after. One year in, the output factory is ahead.
Five years later, the craft factory might have developed an entire workforce capable of creating not just more output, but better & smarter output without sacrificing quality or values. Their competitive advantage wasn’t about today’s output; it was about tomorrow’s capabilities, and quarterly KPIs don’t easily capture them.
It grows over years so subtly you usually only realize it’s gone after you notice its absence.
The Things Employees Stop Doing
Not only do metrics influence what employees do, but they also influence what employees quietly stop doing. Take a veteran project manager who routinely spends their Friday afternoons working through colleagues’ complex, messy projects. There is no metric for mentoring, no dashboard tracking generosity, and no quarterly goal to help other departments meet their targets.
Nevertheless, when the company adopts a utilization rate that values nearly all hours spent on billable activity, the manager is never explicitly asked to halt his mentoring, only that “we’d love for you to be 100% utilization and work your shift’s duration on billable projects”. Over time, the manager has trouble justifying mentoring anyone.
Then, in an instant, poof, it’s gone!
The company gets 3% points of utilization and a loss of something far harder to repair. Moreover, those who, at this point, would be tempted to say “it’s just an individual matter” should remember that a company is made up of hundreds to thousands of living, breathing individuals. It’s not so much that one person stops functioning; entire departments stop sharing knowledge, because collaboration time could be allocated to departmental goals. Managers stop coaching team members because getting stuff out the door right now takes precedence over people’s development and future growth. Employees hesitate to try innovative projects because failed attempts have consequences for their personal evaluations. These things are not deliberate managerial decisions; these are inevitable responses to organizational cues and the incentives we keep mentioning.
Peter Drucker observed well: “What gets measured gets managed.” Yet what is not measured will be ignored, seldom discussed, forgotten, and will surface as unforeseen consequences later on.
When Good KPIs Produce Bad Decisions
The KPIs themselves may not be wrong; they’re just limited. A good metric can become a bad one when it shifts from a guidepost to the destination itself. Organizations of all shapes and sizes have had the same experience.
- Software Development
For many years, developers were measured by the lines of code they wrote. On the surface, the logic seemed fine – the more code written, the more productive the developer. Unfortunately, developers were incentivized to write more code, not better code – ye’ ol’ quantity-over-quality shenanigan. Conversely, modern software engineering holds that good solutions often involve writing less code.
- Healthcare
Patient throughput in the emergency room is routinely monitored for a range of reasons, not least to reduce wait times and improve access to care.
This metric is clearly important, but clinicians are aware that meaningful conversations, nuanced observations, and shared decision-making cannot always be neatly slotted into pre-set time boxes. Hospitals that focus solely on speed do so at the risk of missing key aspects of care.
- Aviation
Even in this highly quantitative field, there is an understanding that not every important thing can be represented by a number.
Commercial airlines meticulously monitor thousands of variables, from fuel efficiency to maintenance schedules. Nevertheless, they spend a considerable amount of time and resources on developing Crew Resource Management (CRM), an approach focused on building communication skills, mutual trust, leadership, and a safe psychological environment within the cockpit. These aspects are not ignored because they are hard to measure. They are carefully nurtured because, as history shows, they save lives.
- Automotive
Perhaps one of the most widely known examples in the business world comes from Toyota, the Japanese automaker. The Toyota Production System (TPS) is well known for its metrics and continuous improvement methodology. Concurrently, it also strongly emphasizes people development, encourages employees to halt the line if they detect quality issues, and views improvement as a collective learning process rather than a numbers game. In essence, the numbers do matter, but so do the conversations that occur around them, and that can be easy to miss.
Companies struggling with KPI memory loss tend to assume that if a metric is not displayed on the dashboard, it cannot be strategically important. The healthiest companies take the opposite approach. They understand that the dashboard offers only a partial picture of the organization’s health.
Some of its most vital components – trustworthiness, judgment, craftsmanship, curiosity, mentorship, and shared experience – remain alive, regardless of whether they are measured. The real problem is not whether to rely on numbers or intuition, but rather the failure to remember that one can never replace the other.
What High-Performing Organizations Choose Not to Measure
That raises an interesting question: if some of the organization’s greatest capabilities are elusive to measure, what do the best organizations in the world do?
They can’t just abandon performance measures, right? RIGHT?
Right, they don’t. In many cases, high performers recognize that measurement has its limits.
Take a look at Pixar. For years, the animation studio has turned out films that win hearts and minds and create core childhood memories for parents and children alike. Of course, Pixar monitors budgets, schedules, and production milestones. Yet some of the real magic happens because the company is willing to make room for what can’t be quantified by a KPI: candid dialogue.
One of the most widely discussed Pixar traditions is the Braintrust, a circle of seasoned directors and writers who regularly gather to roast works in progress.
No scores, no charts, no dashboards, no key performance indicators. What matters is genuine feedback, a psychological safety net, and a willingness to push ideas (not people) to their breaking point. The organization creates room for judgment.
Now let’s go back to Toyota for a second.
Not everything gets translated into a number. The famous Toyota Production System may be well known for its metrics and focus on continuous improvement, but one of the company’s enduring guiding principles is respect for people.
Its workers feel empowered to halt a production line when they spot a flaw not because a performance measure mandates it, but because their judgment is trusted and valued.
This doesn’t mean that Toyota avoids measuring. It has more to do with the fact that it appreciates that its greatest assets reside alongside its measurements, not within them. That theme will appear time and time again across top-tier companies.
Experienced executives don’t just ask, “What should we measure?” ❌
They ask, “What do we need to keep talking about even if we can’t measure it perfectly?” ✅
Beyond Dashboards: Remembering the “Why“
One theme that echoes throughout the literature on organizational memory is that organizations are pretty good at recording what happened. They’re a whole lot worse at remembering why it happened.
Minutes of meetings show what was decided, project plans show when the decision was made, dashboards show what the result was; however, even with all that, the reasoning behind the decision (the trade-offs it required, the alternatives it rejected, the hunches it relied on) often remains elusive.
Think about walking into a company where the same customer policy has been in effect for a decade. Everyone adheres to it, but nobody knows why. Its memory has been lost among dusty desks and cramped file cabinets. A manager suggests tweaking it, as it seems stale and no longer aligns with the organization’s current state. Their peer protests that “it’s always been done this way,” yet none of them can tap the original logic behind it all. It’s not just that information is missing. The entire context for the origin of the information is missing.
This is the plight of most KPIs as well.
- We recall that our customer satisfaction score dropped four points, and not that our recent reorganization had frayed our client relationships months prior.
- We recall that productivity grew by 12%, and not that our employees started shunning one another to get there.
- We recall that costs declined, but not which abilities those reductions simultaneously hobbled.
- We recall that revenue exceeded its target, and not that a handful of unsustainably large discounts drove it.
- We recall that safety incidents declined, and not that workers had become increasingly reluctant to report near misses.
Numbers capture results or the end product. Stories capture context or the journey to said end product. The best companies value both.
Building Organizations That Remember More Than Numbers
None of that is to say that companies shouldn’t measure less. Often, they should probably measure better. A balanced performance system understands that metrics are evidence, not adjudication.
- When your engagement metric drops, it should start a conversation, not conclude it.
- When your productivity metric improves, you should question your leaders: “What did you change? What may have suffered as a consequence?”
In the same way, when there’s an unexpectedly great result, don’t just look at it on a celebratory dashboard and gloat to everyone near & dear. Dig into it: What did we do differently to get here? Was it more collaboration? Did a senior, intuitive employee make a gut call at just the right moment? Did the team have enough faith in each other to say, “Hey, this isn’t working?”
Some companies consciously strive to keep institutional memory alive through mentoring, after-action reviews, storytelling, communities of practice, intergroup collaboration, and discussions focused on reflecting on the past. These are more than just tools for transferring knowledge. They are tools for transferring judgment because, as the adage goes, judgment doesn’t live in the data alone. It lives from person to person, conversation by conversation.
Final Thoughts
Performance management has revolutionized modern management. Organizations would have a hard time understanding performance, gauging results and failures, allocating resources, or identifying potential risks without KPIs. The use of metrics remains the strongest lever available to leaders. However, every tool has its limitations.
A map shows us the path around a city; it’s not the city itself. Likewise, a dashboard illustrates organizational performance; it’s not organizational performance itself. Organizational performance is much more than just mere engagement numbers; leadership is much more than productivity metrics; organizational innovation is much more than just the number of ideas spewed forth by lateral thinkers; organizational customer loyalty is much more than Net Promoter Scores, and our organization’s memory is much richer than any data we collect in reports and dashboards.
The single largest risk may be that we measure too much, rather than recognizing that there are more ways than measurement alone can provide. Organizations do not become exceptional by quantifying everything; they become exceptional by discerning what must be quantified and what must be conversational, observant, coached, and trusted.
Numbers tell us what happened; people explain to us why the numbers happened.
*********
Looking to strengthen your expertise in KPI design and performance measurement? Gain practical knowledge, proven methodologies, and globally recognized certification through The KPI Institute’s Online Certified KPI Professional.
