Project Success Rate

Goal of the analysis:

Quantify the proportion of R&D projects that meet defined success criteria and diagnose why projects succeed or fail. For executives, Project Success Rate is a holistic productivity and predictability measure: it reflects planning accuracy, execution discipline, product–market fit, and commercialization readiness. A robust approach defines success along multiple dimensions (time, budget, scope/quality, and business outcome), normalizes for complexity, distinguishes incremental from transformational work, and links results to portfolio governance (stage-gates, WIP limits), resourcing, and delivery practices.

Data required:

  • Project master and taxonomy:
    • Project ID, name, BU/product line, program/family, product category (software/hardware/services), strategic theme/horizon (H1/H2/H3), project type (sustaining, new feature, platform, new product, research).
    • Stage-gate history (entry/exit dates), start and closure/launch dates, on-hold flags, scope change records.
  • Plan, budget, and scope baseline:
    • Original and latest approved plan dates (MVP/GA), baseline budget/EAC, scope baseline (features/requirements, TRL targets), quality/regulatory requirements.
    • Change requests with dates and reasons (customer request, compliance, dependency, defects).
  • Execution and cost:
    • Actuals: spend by month (labor/non-labor), capitalized development, FTEs by role/location, vendor spend, burn rate.
    • Schedule performance: milestone hits/misses, cycle time, lead time to gates.
  • Quality and readiness:
    • Defect density/escape rate, reliability metrics, performance benchmarks, compliance/validation results, security/privacy test outcomes.
  • Business outcomes (post-launch):
    • First-year revenue/ARR vs plan, adoption/attach/upgrade rates, margin, NPS/CSAT, renewal/churn (for SaaS), launch readiness scorecards.
    • For non-launched (killed) projects: kill stage and reason codes.
  • Context and normalization:
    • Complexity (story points, components, regulatory class, platform novelty), dependency and risk scores, team maturity indices.
    • Accounting and FX policies; program-to-project hierarchies to avoid double counting.
  • Systems:
    • PPM/ALM/PLM (Jira/ADO/Planview/Teamcenter), ERP/GL, FP&A, HRIS/time-tracking, QA/test systems, CRM/CPQ for bookings.

Detailed step-by-step instruction on how to conduct the analysis:

  1. Define “success” and scoring policy.
    • Adopt a composite definition with thresholds:
      • On-time: GA/MVP delivered within ≤10% of baseline schedule (or within X weeks).
      • On-budget: Actuals ≤105–110% of approved EAC.
      • Scope/quality met: ≥90–100% of must-have scope delivered; defect/validation targets met.
      • Business outcome: ≥80–100% of first-year revenue/ARR or adoption targets, or validated learning achieved (for H3).
    • Create a Success Index (0–100) with weights (e.g., Time 25%, Cost 25%, Scope/Quality 25%, Outcome 25%). Define a pass threshold (e.g., ≥70 overall and no red on critical criteria).
    • Set alternate definitions by project class (e.g., research options “succeed” if learning milestones are met, regardless of launch).
  2. Build the cohort and handle censoring.
    • Choose cohorts by start quarter or Define gate date. Include projects closed/launched in the lookback (e.g., last 12–24 months). Treat in-flight projects separately; do not classify them as success/failure yet.
    • For business outcomes, measure at a consistent post-launch window (e.g., 6 or 12 months) and label more recent launches as “pending outcome.”
  3. Assemble and normalize the dataset.
    • Extract plan/budget baselines, actuals, scope changes, quality, and outcome metrics; standardize to reporting currency/units.
    • Attach complexity and project-type tags to enable apples-to-apples comparisons.
  4. Compute KPIs.
    • Project Success Rate = # projects meeting success definition ÷ # closed/launched projects in cohort.
    • Dimension rates: On-time %, On-budget %, Scope/Quality pass %, Outcome pass %.
    • Success Index distribution (median, 25th/75th percentiles).
    • Kill quality: % killed by stage and % kills pre-Develop (proxy for governance rigor).
  5. Segment and compare.
    • By BU/product, project type (sustaining/new feature/platform/new product/research), horizon (H1/H2/H3), technology (software/hardware/medtech), complexity band (S/M/L/XL), sourcing model (in-house/vendor/hybrid), and geography/site.
    • By team maturity (agile adoption, automation level), dependency risk, and regulatory class.
  6. Trend and cohort views.
    • Quarterly cohort success rates and Success Index medians over 8–12 quarters.
    • Time-to-success distributions; survival curve from start to launch vs kill.
  7. Driver and root-cause analysis.
    • Model success odds vs drivers (planning accuracy, WIP, team stability, automation, vendor share, complexity).
    • Pareto of failure reasons (requirements churn, technical risk, vendor delays, regulatory issues, under-resourcing).
    • Link to upstream metrics (Stage-Gate Conversion, WIP, Cost per Project) to identify systemic causes.
  8. Quality checks.
    • Validate baseline freeze dates and change control; exclude projects with missing baselines from rate or report separately.
    • Ensure bundles/programs aren’t double-counted; reconcile financials to GL.
    • Sample outcome calculations for consistency (ARR attribution, bundle allocation).
  9. Targets and thresholds.
    • Set targets by segment (e.g., sustaining ≥85–95%, new features ≥70–85%, new products ≥50–70%, H3 options ≥60–80% learning goal attainment).
    • Define red/amber thresholds for On-time and On-budget (e.g., red if >15% miss), and for Success Index (e.g., red <60).

Format of the output of analysis:

  • Executive dashboard: overall Project Success Rate, Success Index median, on-time/on-budget/scope/outcome pass rates, by latest cohorts.
  • Segment heatmaps by BU/product, project type, horizon, and complexity with traffic-light thresholds vs targets.
  • Cohort trend charts (8–12 quarters) for success rate and Success Index; survival curves and kill distribution by stage.
  • Pareto of failure reasons and driver diagnostics; scatter plots of Success Index vs complexity and vs automation/maturity.
  • Bridge charts: variance in success vs prior year explained by planning accuracy, WIP, vendor share, complexity mix.
  • Appendix: success definition, scoring/weights, cohort and censoring rules, outcome measurement windows, data quality notes.

How to interpret results:

  • High success rates with balanced ambition (not just easy wins) indicate healthy portfolio governance and execution. Verify complexity mix and horizon balance to avoid “sandbagging.”
  • Low success in new products but high sustaining success suggests technical debt crowding or weak discovery/commercialization; strengthen early gates and launch readiness.
  • High on-time but low outcome pass points to building the wrong things; improve customer validation, pricing/packaging, and GTM.
  • Low on-time/on-budget with acceptable outcomes may reflect poor planning or scope churn; tighten baselines, change control, and WIP limits.
  • Segment disparities (site/vendor/technology) reveal capability gaps; pair low performers with top-quartile practices and talent.
  • Improving trends should be confirmed against constant definitions and cohort composition (complexity and horizon) to rule out optics.

Steps a company can take to improve on this measure:

  • Strategy and portfolio governance:
    • Clarify success definitions by project class; enforce stage-gate evidence and kill early to avoid late, costly failures.
    • Set WIP limits; align starts with team capacity; ring-fence capacity for platform/H2/H3 to reduce crowd-out.
  • Planning and scope control:
    • Adopt rolling-wave plans with stable, frozen baselines at each gate; institute rigorous change control and impact assessment.
    • Use MVP scoping, outcome-based roadmaps, and feature thinning to protect predictability.
  • Execution excellence:
    • Scale agile/DevOps, test automation, CI/CD (software) and model-based systems engineering, DFM/DFA (hardware) to reduce rework and cycle time.
    • Stabilize teams; reduce context switching; deploy playbooks for integration and verification.
  • Discovery, validation, and GTM readiness:
    • Strengthen customer discovery, pricing experiments, pilots; require launch readiness scorecards (enablement, supply, certifications).
    • Tie funding to validated learning milestones, especially for H3.
  • Vendor and dependency management:
    • Use outcome-based contracts, milestone payments, and tight SOWs; track vendor SLA adherence and defect escape rates.
    • De-risk critical dependencies early (architecture reviews, supply commitments, regulatory pathways).
  • Measurement and incentives:
    • Publish Success Index by BU/team; reward early, evidence-based kills and successful launches; avoid perverse incentives tied only to starts or feature counts.
    • Automate dashboards; audit baselines and outcomes for integrity.
  • Scenario guidance:
    • If outcome pass is weak while delivery is predictably on-time: increase pre-Define market tests, adjust pricing/packaging, and enhance GTM partnership.
    • If schedule/budget misses dominate: enforce WIP limits, improve estimation (reference class forecasting), and reduce scope churn.
    • If H3 “failures” are high: reframe success as learning milestones; tranche funding and improve option discipline.

Benchmark comparisons:

General benchmarks (directional; definitions vary):

  • Sustaining/maintenance projects: 85–95% success (time/cost/scope).
  • New features/modules (software): 70–85% success in mature agile organizations; lower (50–70%) with low automation or high dependency churn.
  • New products/platforms (hardware/regulated): 40–70% success depending on complexity and regulatory class.
  • Research/H3 options: 60–80% learning milestone attainment (success-by-learning); launch success intentionally lower due to option portfolio design.
  • Kill timing: ≥70% of kills before heavy development (pre-Develop) in healthy portfolios.

Segment- or industry-specific benchmarks:

  • Software/SaaS: success rates 70–85% for features; outcome attainment hinges on GTM/PLG; Validate→Launch conversion ≥85–95% for well-governed teams.
  • Industrial/hardware: 50–70% for new products; DFM/DFX and supplier readiness materially increase odds.
  • Medtech/devices: 40–60% to launch (clinical/regulatory risk); higher milestone attainment pre-clinical.
  • Consumer electronics/CPG: 60–80% depending on seasonal locks and supply readiness; late kills should be rare.

Because success definitions differ across firms, construct internal benchmarks: (1) fix a composite Success Index and pass thresholds by project class; (2) compute 8–12 quarter cohort baselines by BU/type/complexity; (3) target top-quartile internal performance; and (4) track mix-adjusted trends to separate real improvement from portfolio composition shifts. Refresh targets annually and after major methodology or governance changes.

How to get started

1

arrow-down-blue

Tell us about your project

2

arrow-down-blue

Interview candidates

(We’ll provide bios within 48 hours on average)

3

Select your consultant and start work

Find a Consultant

or email us at: [email protected]