Measure the Boring Baseline Before You Build With an LLM
Aaron Wise
LLM baseline first: measure the rule, lookup or small model on the same test, and fix in advance the margin the LLM must beat before you fund it.
Editor's Pick
View All
Leadership & Management
More in Leadership & ManagementCandidate Experience Is Your Employer Brand
Aaron Wise
Candidate experience is your employer brand: how you treat candidates, especially rejected ones, tells the market what working for you is like.
What a Week of AI Coding Agents Actually Cost Me
Aaron Wise
Real numbers from one week of Claude Code and Codex — cost, cache savings, CI minutes, what ran, and what broke.
Leading Teams Where Agents Write Half the Code
Aaron Wise
Leading engineering teams with AI coding agents: make review the main job, keep a named human owner per system, and retrain juniors for judgment.
Product Management
More in Product Management
AI Features Without a Feature Factory
Aaron Wise
AI feature prioritization: fund a model only when there is a measurable user problem, data you can use and a way to evaluate it. Otherwise it is output.
Pricing Is a Product Decision Engineering Needs to Be In On
Aaron Wise
SaaS pricing decisions set architecture: seats, plan limits and usage billing each have a build cost. Bring engineering in to cost options before launch.
Engineering
More in EngineeringWhat a Week of AI Coding Agents Actually Cost Me
Aaron Wise
Real numbers from one week of Claude Code and Codex — cost, cache savings, CI minutes, what ran, and what broke.
Your Coding Agents Are Making Supply-Chain Decisions
Aaron Wise
Coding agents and supply chain security: agents add packages and run install scripts nobody approved. Gate new dependencies, sandbox agents, pin builds.
Must Read
View All
Engineering, Insights, Leadership & Management, Must Read
The Wrong Kind of Early: When Startups Aren’t Ready for Senior Engineering Leadership
Senior engineering leaders need a clear problem, founder-market fit, and rough business plan before joining a startup.
Roadmap Roulette: Choosing the Right Prioritization Model for Every Stage of Growth
The right roadmap prioritization model depends on your company’s growth stage and strategic goals.
Data Science
More in Data ScienceMeasure the Boring Baseline Before You Build With an LLM
Aaron Wise
LLM baseline first: measure the rule, lookup or small model on the same test, and fix in advance the margin the LLM must beat before you fund it.
Know the Cost per Answer Before You Ship an LLM Feature
Aaron Wise
LLM cost per query, fully loaded: what one answer costs, what it is worth, what heavy users do to both, and the levers that fix the math before launch.
Evaluations Are the Spec for an LLM Feature
Aaron Wise
LLM evaluation sets are the spec: write twenty good-answer test cases before building, with product owning the bar and engineering keeping it green.
Recent Posts
Search all postsKnow the Cost per Answer Before You Ship an LLM Feature
LLM cost per query, fully loaded: what one answer costs, what it is worth, what heavy users do to both, and the levers that fix the math before launch.
What a Week of AI Coding Agents Actually Cost Me
Real numbers from one week of Claude Code and Codex — cost, cache savings, CI minutes, what ran, and what broke.
Your Coding Agents Are Making Supply-Chain Decisions
Coding agents and supply chain security: agents add packages and run install scripts nobody approved. Gate new dependencies, sandbox agents, pin builds.
Pricing Is a Product Decision Engineering Needs to Be In On
SaaS pricing decisions set architecture: seats, plan limits and usage billing each have a build cost. Bring engineering in to cost options before launch.
Crossing Organizational Boundaries
More in Crossing Organizational Boundaries
Cross Organizational, Leadership & Management
Candidate Experience Is Your Employer Brand
Candidate experience is your employer brand: how you treat candidates, especially rejected ones, tells the market what working for you is like.
Cross Organizational, security
The Security Questionnaire Is Part of the Sale
Enterprise security questionnaires stall deals nobody owns. Run review as a sales stage: answer library, trust page, one owner, SOC 2 when deals need it.