Leading in the Age of Ubiquitous Models

Written by Adrian Maharaj

(Views mine, not Google’s.)

Thesis
Foundation models have crossed from novelty to infrastructure. The advantage no longer comes from merely having models; it comes from how you instrument, govern, and continually improve the socio‑technical system around them.

What changed

The operating system CEOs need

  1. Interoperability layer: standardize how teams discover, invoke, and observe models (internal & vendor). Treat prompts, tools, policies as versioned assets.

  2. Data flywheels: usage → feedback → fine‑tuning. Instrument outcomes (task success, cost per successful task, latency distributions).

  3. Risk governance: risk register mapping use cases to impact & controls (evals, red teaming, oversight). Make it auditable.

  4. Judgment loops: pair model outputs with expert review where cost of error is high; log reviewed vs auto ratios by domain and lower them with evidence.

  5. Portfolio & P&L discipline: owners, budgets, target unit economics (e.g., cost per assisted case closed); kill or rework experiments quickly. (DORA’s flow principles still apply.) https://dora.dev/research/2023/

What to stop doing

  • One‑model bets (lock‑in masquerading as strategy).

  • Feature theater (AI icon ≠ value).

  • Compliance last (evaluations and red‑teaming should be in the pipeline).

Start this quarter

  • Map sprawl: inventory AI touchpoints, vendors, datasets; tag owners.

  • Define 5 enterprise KPIs: cost per successful AI task, assist‑rate, deflection rate, win‑rate delta, time‑to‑first‑draft.

  • Stand up an AI Evaluation Guild (product, security, legal, domain experts) with block authority.

Decision review for high‑risk use: human sign‑off with structured logging to train both people and models.

Next
Next

Flash‑Partnerships Are Coming