Who Are You?
Your title sounds impressive, corporate acumen, a given
Decisions made are rigorous, considered, “data-driven.”
The numbers on the dashboards trump emotion, so you say,
Your quants have stellar pedigrees and lofty GPAs
If all are wise and diligent, credentialed and proficient,
But still the numbers flag, pray tell, how is the team deficient?
Dear C{x}O,
Let me guess. You aspire to make “data-driven” decisions across the organization. You believe in empiricism over emotional, rigor over rationalization, and numbers over narrative. (Hopefully, you prefer alliteration over academic style). Corporations exist as engines that allocate capital to projects, hoping for positive ROI. Naturally, data should inform these allocations. And it is the aspiration of executing those allocations better that justifies your salary and those of the youthful drones (ahem, “analysts”) beneath you on the org-chart. Yet somehow, amidst the dashboards and the discourse and the discussions is this niggling sense that something is amiss. The steering wheel feels detached from the rest of the vehicle, and even when turning the wheel produces the intended result, there is a lack of confidence in the GPS system. It will come as little surprise that the deficiency is not much ado about quantitative talent. You’ve probably hired data scientists and engineers with suitably prestigious pedigrees. The folks writing code all produced exemplary marks in their former academic lives and suitable pseudocode when interviewed. They ain’t dumb. Neither are you. So what’s going wrong?
For one, the science of decision theory adores well-posed, well-organized problems. The proverbial “real world” is messier. For another, we have become exceedingly adept at fooling ourselves with data. So let’s consider an all-too-common example from the 2025 corporate milieu, then dissect the pitfalls into which we step unwittingly, unknowingly, and frequently.
Blindspots
The sober implication, from this humble, corporate jester:
“You think your job’s to manage, when in fact, you’re an investor.”
Corporate leaders invest. They invest in people, facilities, equipment, and other assets. They allocate those resources. No more, no less.
If corporations are engines with which to allocate capital, the latest and greatest “AI” project is consuming plenty of unburned fuel. Someone with a “C” in their title or the board to which they reported demanded that a department or division become “AI-enabled” or some such buzzword. And suddenly, the cash is flowing, expectations are rising, and if the last couple quarters are any indication, the results are likely to disappoint. How did this happen? Let’s consider four common intellectual fallacies, and don’t worry, I’ll steer clear of anything that evokes traumatic memories of statistics professors or even particularly esoteric chapters of “Thinking Fast and Slow.”
Numerator/Denominator
If all decisions made reduce to acts of allocation,
And typically, we fixate on the upside speculation,
We’ll skip, negate, or understate, within that transformation
The subtle loss, and ling’ring cost within the calculation
Corporations allocate capital to projects. Success is measured in ROI. ROI is little more than a fraction, with the denominator defined by the investment’s size and the numerator defined by the capital it produces. A value greater than one means more capital out than capital in. So far, so good?
Here’s where we err. We spend a great deal of time envisioning the upside, the numerator, the potential gains. And this is, to some extent, desirable. Who wouldn’t want to work with optimistic colleagues who see, in their minds eyes, brighter days ahead? And the denominator seems tiny by comparison, especially with those “AI” projects. The code is going to do the work, and code is cheap! Of course, that code is going to be written by human beings (who get paid). The folks writing the code are managed by other human beings (who get paid even more) who discuss their progress early and often with senior folks (who get paid more still). Once the imagined software exists and nirvana is reached in the form of automated processes that once demanded labor, the costs cease? Of course not. The software is maintained by human beings (who are paid) and connected to existing infrastructure to supply it with the requisite inputs by individuals with the relevant expertise (who are paid). The performance of this new wonder-tool will be measured and monitored by another set of individuals (who are paid). These AI interfaces will increase the surface area cybersecurity engineers will need to defend, especially if these “agents” will have access to critical systems (oh, and cybersecurity professionals are paid).
Rarely, if ever, are the fractional allocations of highly-paid professionals’ time considered, especially if that temporal investment is ongoing. But clearly, those costs live in the denominator and meaningfully alter an ROI calculation. Everything is viewed through rose-colored glasses when these “hidden” costs are obscured.
Zero/Infinity
If efforts focus solely on the on gains in gold that’s glittered,
And never tend to tradeoff spend still lurking, unconsidered,
The optimal solutions from each analytics team must
Land on all-or-nothing answers at numerical extrema
Economists are fond of “tradeoffs.” Corporate chats are often…not. Absent some well-articulated notion of tradeoffs, the optimal solution is essentially always zero or as many as humanly possible. Once upon a time, I worked for a company that sent what were essentially cold outreach emails to prospective users. Initially, we would model click-through-rates, estimate the proportions of those clicks that would yield conversions, and project the revenue that would arrive as a result. In other words, we were the diligent little dashboard makers that C{x}Os love.
One small problem emerged.
If the goal is to maximize revenue (and when isn’t it?), then clearly, sending more outreach emails is better! And sending emails is “free!” Just write a piece of software that iterates over a list of prospects, tailors the emails appropriately, and wait for the money to start pouring in! Better still, perform some A/B tests on the content of those emails the optimal follow-up cadence and you’re really in business!
Ah, but each outreach isn’t “free”.” Sure, it’s cheap. But cheap and free are, as Mark Twain might have said in some fluorescently-lit boardroom before utilizing that plastic waste bin as a spitoon, as different as “lightning and a lightning bug.”
Each individual element of outreach contained its probability of conversion (greater than zero). Each individual element of outreach was truly independent (John Doe has no idea if Jane Doe received the same email, etc). However, each email increases the probability that the recipient unsubscribes, sends future messages to spam, or otherwise exclaims, “no more!” Each entreaty that does not yield a conversion erodes the value of that potential user/customer. This has a cost. Calculating that cost is complicated. Intuiting that it ain’t zero is not.
Absent some understanding that cold outreach is not “free,” the discussion continues to center around maximizing conversions, which inevitably leads to the conclusion that more outreach is better. The campaigns proliferate, with their derivatives and split-test duplications multiplying like digital tribbles. The optimal number of emails is now…infinity.
The inverse of this particular example is that of risk mitigation. Whether that risk is fraud or some cybersecurity breach, it is easy (if dangerous and incorrect) to continue descriptions of the potential downsides in monetary terms without enumeration of the costs associated with preventing negative outcomes. I recall a corporate environment in which, as the global head of advanced analytics, literally seven layers of approval were required for me to gain access to a particular set of data. Notably, the number of corporate layers between myself at the C-suite was nowhere near seven. This byzantine process was authored in the name of security, with zero thought given to the hidden cost of asking well-paid professionals to eschew productivity in the name of completing whatever forms, explanations, and CYA-adjacent documentation were required.
Here’s a foolproof way to ensure there will be precisely zero cybersecurity risks. First, shudder the building in which the business operates. Secondly, terminate all internet access in that building and delete/disable all company software. Schedule an implosion of said structure as soon as possible, and to be especially thorough, execute all current and former employees. Obviously, the example is intentionally farcical, but one can easily recognize that elimination of all fraud or cybersecurity risk is simple - cease operating the business! We know there are costs associated with guardrails. A sober discussion demands an accounting of the tradeoffs. Otherwise, the optimal quantity is zero.
As a rule of thumb. If the prevailing discourse would be unable to articulate why the optimal value of something is neither zero, nor infinity, the thought process is fundamentally flawed, regardless of how pristine those bar charts look.
Enthusiasm Gaps
Beneath the objectivity to which we all aspire,
Are knowledge gaps and passion traps and difference in desire,
Enthusiastic humans with an optimistic prior,
Will often find their wanting mind an upside multiplier!
What do the first two sets of examples have in common? Here’s a clue, it isn’t the general cynicism of their narrator. (Ok, they do have that in common, but to [mis]quote Bob Dylan, “I’m just one too many [dashboards], and 1,000 [tasks] behind”). The true point of commonality is the disparity between the level of enthusiasm and engagement for either the numerator or the denominator.
When that long-discussed generative-AI project is considered, the enthusiasm is off-the-charts for the numerator (all the wonderful automations and cost-savings) and no one is remotely enthusiastic about the denominator (the ongoing maintenance, risk-mitigation, monitoring, and infrastructure costs). When that outreach campaign is discussed, the enthusiasm burns brightly for conversions and revenue, and is non-existent for the irritation and subsequent erosion of value within that database of emails. When fraud and cybersecurity are discussed, the enthusiasm (or at least attention) is focused upon the losses avoided, the risks prevented, and the careers saved, not the underlying, hard-to-calculate costs of bureaucracy and inefficiency.
Care to guess that that enthusiasm gap yields when all is said and done (and more is said than done)?
Flawed Experimentation
If problems of humanity that lurk in each transaction,
Are as Pascal would argue, just a loathing of inaction
We struggle, as the adage notes, to sit, in peace, alone,
And thus, devise activity and grind with mill-ish stones.
But every fact of every act which seems profoundly wise,
Must first assess, beyond a guess, a proper, ample prize!
If a corporation’s “job” is to invest in projects that yield positive ROI, and those project costs are measured in both capital and the human beings earning salaries to execute them, then choosing those allocations wisely is crucial. In fact, choosing those allocations is, in some sense, the entire operation - there is little else for an executive to do beyond simply deciding where human and monetary resources are allocated.
Do all these discussions and decisions occur with all due deliberation and mathematical rigor? Doubtful. But we attempt to solve this problem beneath a glossy patina of “experimentation.” Typically, we run experiments to evaluate the impact of a project (in the hopes of justifying the time and money already spent). Ideally, we should be running experiments long before any sizable allocation occurs.
We’re all eager to “do work.” Those of us with hard-wired, left-brained work ethics want nothing more than to start chopping the wood. Get those emails sent. Get those meetings on the schedule. Get that spreadsheet filled with juicy data and start analyzing. Build the next app. Do something.
Of course doing the thing means not doing another thing, which means eschewing some other allocation of time and money. Which means the real value of experimental design begins with understanding what might or might not represent a viable opportunity in the first place.
Years ago, I found myself in a meeting to discuss how we were going to measure the impact of a project, the parameters of which were apparently discussed, like Mario the plumber’s princess, in another castle. After a few back-of-the-napkin calculations, it was clear that if this initiative achieved its objectives beyond any reasonable expectation, the net result would be something like $50K per month of incremental revenue. The project would require several well-paid professionals to implement, a couple more to monitor and measure, and almost assuredly would not generate that full $50K. Simply, there weren’t enough customers of that segment spending enough on those products for any cross-selling exercise to yield significant fruit.
But workers wanna work.
Ultimately, experiments are to be designed long before that first beta test, that first outreach email, that first model calibration. Is the opportunity size sufficient? Is the result measurable? What experimental control is required before one pollutes all prospects with some sawed-off-shotgun campaign? And you, C{x}O, are the person maintaining the robustness of that portcullis against the battering ram that is “build first and ask analysts later.”
Expensive Workers are the Worst Workers
Blue-Collar
Theoretically, every employee and contractor is downstream of some decision-making apparatus that determined such an expenditure was worthwhile. In this case, “worthwhile” would entail generating more revenue or mitigating more costs than the compensation required. Even for folks who neither build the product nor sell it (like, say, the accountant or the in-house counsel) are theoretically in their seats because, in their absence, the costs of errors in invoices or filing taxes improperly or exposing the entity to liability risk exceeds the cost of employing trained professionals to play adroit defense.
The problem is that the “best” workers, with their impressive pedigrees and glossy credentials, are often the worst workers we employ. Why? Consider a counterexample. Somewhere, as you are reading this, a worker is toiling in a factory assembling widgets (yes, in this example, people still manufacture tangible products from physical materials in exchange for pay like in those economics textbooks no one has opened since Y2K was a thing). Let’s imagine that he or she can produce 10 widgets in a day for a modest salary of $50K/yr. Maybe the widgets they produce will sell for $100K. (I promised easy math, and I am a man of my word). Presumably, they’re distractible, somewhat inefficient, and beset with all manner of human foibles and frailties. So maybe as a consequence of their imperfections, they produce 8 widgets. Thus, they’re 20% inefficient, and so we wind up with $80K worth of widgets from our $50K employee.
(This would be, approximately, the moment when consultants are procured at great cost to discern how to optimize the efficiency of these widget-makers, the production processes associated with widget making, and whether or not we could automate a substantial proportion of the widget-making by “leveraging-AI.” I’m sure the powerpoint presentation will be stellar.)
We fixate on the widget-maker because all the numbers fit nicely into spreadsheets. Cost of input materials. Cost of employee. Sale price. And so. We choose the problems we can solve easily rather than the ones that matter more like, say…
White-Collar
…a $200-$250K senior director of marketing or VP of analytics or whatever title looks suitable on LinkedIn. That individual, with a Zoom-laden schedule and a vocabulary befitting what is often many years of academic training followed by dues-paying-vocational experience was also hired because (we hope), positive ROI will be the result. And to state the obvious, that hurdle is far higher.
Worse, their sub-optimal results aren’t necessarily the byproduct of distraction or malaise (though to be fair, my fantasy football teams fare quite well, and the mid-day consumption of ESPN articles was assuredly counterproductive to someone’s bottom-line…clearly, I’m the only professional to ever operate inefficiently in this manner). These individuals are part of the apparatus of allocation. These individuals help decide whether that next million lands in the coffers of a project that will return two million or none at all.
Again, for those scoring at home, the difference between two million and nothing is much larger than the difference between $100K and $80K (100x larger to be precise…almost as though I crafted this example for that particular, illustrative purpose).
This cadre of VPs, PMs, and other prestigious acronyms are probably much “worse” in terms of the amount of capital they leave on the table than the widget-makers. And while consultants will gladly take a hacksaw to middle management, what is rarely considered is the potential for these individuals to become truly savvy decision-makers, rather than victims of the fallacies above (and many others we’ve yet to discover).
Your Task
This, dear C{x}O, is your task. It is not to solve every quantitative dilemma known to mankind, nor even your particular corporation. It is your task to ensure that the lieutenants carrying out your mission are as optimized as the frontline whose inputs and outputs are well-modeled.
This is the hard work that justifies your compensation. Surely, you’re worth every penny?