“Without data, you’re just another person with an opinion.” — W. Edwards Deming
Building a Data Program from Scratch: Lessons from the Ground Floor
When companies talk about becoming “data-driven,” they often imagine dashboards lighting up like a control room in NASA’s Apollo era. But behind those screens lies something less glamorous and far more difficult: building a data program from scratch.
This work isn’t about plugging in a tool or buying the latest AI widget. It’s about cultivating trust, designing for change, and building the social and technical infrastructure to turn raw data into durable value.
A Short History of Data Programs
The idea of treating data as a core business asset isn’t new. In the 1960s, Peter Drucker wrote that “what gets measured gets managed,” pushing executives to see information as a lever for competitive advantage. By the 1990s, Thomas Davenport’s work on “Competing on Analytics” cemented the idea that data wasn’t just back-office recordkeeping—it was a moat.
Yet history is littered with failed data initiatives. In the 2000s, many enterprises pursued massive data warehouse projects that ran over budget, failed to adapt, or lost the trust of business users. The lesson? Technology without adoption is shelfware.
Step 1: Start with Listening, Not Building
The first stage of a data program isn’t architecture diagrams. It’s interviews.
Who do you talk to?
- Executives: to understand strategic priorities.
- Operators: to uncover pain points and inefficiencies.
- Analysts: to map out current “shadow pipelines” and Excel workarounds.
- Customers (internal or external): to hear what they wish they knew but can’t.
The goal isn’t to create a requirements list. It’s to surface themes. Which metrics drive decisions? Which reports gather dust? Where is the friction between data supply and demand?
Step 2: Identify the Champions
Every successful data program has what DJ Patil, former U.S. Chief Data Scientist, called “data champions.” These aren’t always the most senior leaders. They are the believers—the product manager who spends weekends building Tableau dashboards, the operations lead who is tired of bad forecasts, the finance director who wants one version of the truth.
These champions will advocate for you when resistance comes—and it will. They’ll help socialize wins, defend budgets, and keep momentum when enthusiasm wanes.
Step 3: Establish a Communication Cadence
Too many programs die in silence. Establish rituals early:
- Weekly standups with the core data team.
- Monthly stakeholder updates with champions.
- Quarterly business reviews (QBRs) with executives, explicitly checking if the value drivers have changed.
These cadences serve a dual purpose: they keep you honest, and they keep stakeholders engaged. A stagnant data program is a forgotten one.
Step 4: Create the Right Artifacts
Artifacts aren’t bureaucracy—they’re memory. They keep a program resilient when people rotate or priorities shift. At minimum, a ground-up data program should maintain:
- A data strategy doc: Why this program exists and what value it will drive.
- A stakeholder map: Who uses what data, why, and how.
- A data dictionary / glossary: To fight “metric drift.”
- A roadmap with milestones and clear outcomes.
These aren’t static. They evolve with each QBR and milestone achieved.
Step 5: Get—and Keep—Buy-In
Initial buy-in often comes from a compelling business case. But sustaining it requires proving value in increments. Borrow from agile software development:
- Start small, solve a high-visibility pain point.
- Show a demo, not just a deck.
- Socialize early wins through champions.
The worst programs chase boiling-the-ocean perfection before delivering value. By the time the “big reveal” comes, executives have moved on.
Step 6: Define Milestones That Matter
Milestones shouldn’t just be about technology (“data lake stood up”). They should tie to business outcomes (“reduced forecasting error by 20%”). Good milestones have three qualities:
- Observable: You can measure completion.
- Relevant: They tie to a stakeholder priority.
- Incremental: They prove progress without waiting years.
Step 7: Build the Team
Once you’ve proven early value, the business case for expanding your team grows stronger. A well-sequenced data team usually follows this arc:
- Foundational engineer: to build pipelines and ensure reliability.
- Analytics engineer: to model data and make it usable.
- Data analyst or product analyst: to deliver insights directly tied to business needs.
- Data product manager: to orchestrate priorities and stakeholder alignment.
- Data scientist / ML engineer (later): when predictive or AI workloads emerge.
Hiring too many too early leads to “idle talent syndrome.” Hiring too late means your foundation collapses under demand.
Done Well vs. Done Poorly
Done Well: Shopify, early in its growth, made data core to its culture by embedding analysts within product teams, treating them as partners rather than report factories. Their data team was a trusted advisor, not a service desk.
Done Poorly: Countless enterprises stood up seven-figure data lakes with no clear business questions to answer. The result? “Data swamps”—expensive, underused, and mistrusted.
Wrapping up…
Building a data program from scratch isn’t about pipelines or platforms. It’s about people, trust, and clarity of purpose. Technology amplifies what already exists—it doesn’t fix cultural gaps.
The leaders who succeed treat data as a program, not a project. They cultivate champions, deliver incremental value, and constantly revisit whether the business still cares about the problems they’re solving.
As Drucker might remind us: what gets measured gets managed. But only if the measurements matter, the people believe in them, and the program adapts as the business evolves.
