Most dev productivity metrics are vanity metrics — AI makes them even more misleading
With AI adoption, superficial metrics explode: lines of code increase by 40%, the number of PRs doubles, commits become more frequent. Management sees this as a sign of increased productivity. In reality, more code does not mean more value. Some teams that are "productive" on paper deliver more bugs and technical debt. The metrics that count are those that measure the value delivered to the customer and the health of the delivery process — not the volume of activity.
The problem
Measuring developer productivity is a minefield. For decades, the software industry has been looking for reliable metrics and regularly getting it wrong. AI has put the topic back on the table — and amplified the usual mistakes.
Vanity metrics have exploded. GitHub reports that Copilot users accept 30% of suggestions and write code "55% faster". These figures are true but misleading: writing code faster is not the same as delivering value faster. The code also has to be reviewed, tested, deployed and maintained. If AI speeds up writing but degrades quality, the net gain can be negative.
A notable share of AI-generated lines of code contain at least one quality issue
Several analyses of large volumes of AI-generated code converge on this finding. The most frequent problems: duplicated code, unnecessary complexity, anti-performance patterns, and missed error handling. These problems are not visible in volume metrics but explode in maintenance costs.
Management wants simple numbers. "How much time are we saving with AI?" is the question most often asked by CTOs and technical directors. The honest answer is: it depends on what you measure. If you measure lines of code, the gain is 30 to 50%. If you measure the value delivered to the customer per sprint, the gain is 15 to 25% in the best teams — and sometimes negative in teams that do not frame the use of AI.
The AI solution
A framework for measuring augmented dev productivity rests on three categories of metrics, from the most reliable to the most contextual.
DORA Metrics: the reference
The four DORA metrics measure software delivery performance objectively: deployment frequency (how many times you deploy to production per week), lead time for changes (from commit to deployment), change failure rate (% of deployments that cause an incident), and time to restore (recovery time after an incident). AI must improve the first two without degrading the last two.
Cycle Time: from ticket to deployment
Cycle time measures the total time between picking up a ticket and deploying it to production. It includes development, review, testing and deployment time. It is the metric most directly impacted by AI — if AI speeds up development but PRs wait longer in review (because volume increases), cycle time can stagnate or get worse.
Quality Ratio: value vs debt
The ratio between the time spent on features (value) and the time spent on bug fixing, refactoring and technical debt. A healthy ratio is 70/30. If AI adoption pushes this ratio to 60/40 (more bugs to fix), the apparent productivity gain is in reality future debt. Measure this ratio before and after AI adoption over 3 months.
Implementation
Here is how to set up a dev productivity dashboard in three steps, from baseline to continuous optimization.
Establish the baseline (weeks 1-4)
Before adopting or rolling out AI, measure your current state over 4 weeks. Automatically collect the DORA metrics from your CI/CD (GitHub Actions, GitLab CI), the cycle time from your management tool (Jira, Linear), and the quality ratio from your backlog (features/bugs ratio). Use tools like LinearB, Sleuth or the DORA Dashboard (free in GitHub) to automate the collection. Also note the subjective metrics: team satisfaction, sense of productivity.
Measure the AI impact (weeks 5-12)
Deploy AI (Copilot, Cursor, Claude Code) and measure the same metrics over 8 weeks. Compare week by week with the baseline. Wait at least 4 weeks to let the team ramp up on the AI tools (productivity often drops in the first 2 weeks). Focus on trends, not absolute values. A weekly dashboard is enough. Expected result on our clients: see our Augmented Code Sprint.
Dashboard and continuous optimization
Create a dashboard visible to the whole team with 5 indicators:
## Augmented dev productivity dashboard
1. Median cycle time → Target: -20% vs baseline
2. Deployment frequency → Target: +30%
3. Change failure rate → Target: stable or -10%
4. Quality ratio (feature/bug) → Target: ≥ 70/30
5. Team satisfaction (NPS) → Target: ≥ +40Hold a monthly review in retrospective. Adjust the AI practices (prompts, workflows, tools) based on the metrics. If the change failure rate increases, strengthen the generated tests. If cycle time stagnates despite faster code, the bottleneck is probably in review or deployment.
Results
Frequently asked questions
Are lines of code a good measure of productivity with AI?
Absolutely not — it is even worse than before AI. A developer with Copilot can generate 500 lines of code in an hour, but if 200 are useless boilerplate and 50 contain bugs, the real productivity is negative. Lines of code measure volume, not value. With AI, this bias is amplified because code generation is almost free.
What are the DORA metrics and why are they reliable?
The DORA metrics (DevOps Research and Assessment) measure four dimensions: deployment frequency, lead time for changes, change failure rate, and time to restore. They are reliable because they measure the business outcome (delivering value quickly and reliably) rather than activity (number of commits, lines of code).
How do you isolate the impact of AI from other productivity factors?
Measure for 4 weeks without AI (baseline), then 4 weeks with AI, on sprints comparable in complexity. Compare the DORA metrics and the cycle time. For more rigor, alternate sprints with and without AI on two parallel teams (A/B testing). In practice, a before/after measurement over 2 to 3 months is enough to reveal a reliable trend.
For technical profiles
Comparison of dev productivity measurement tools (January 2026):
| Criterion | LinearB | Sleuth | GitHub DORA Dashboard | Jellyfish |
|---|---|---|---|---|
| DORA Metrics | 4/4 | 4/4 | 4/4 | 4/4 |
| Detailed cycle time | By stage (code, review, deploy) | By stage | Global only | By stage |
| Quality ratio | Feature/bug/debt auto | Manual | No | Automatic |
| Jira/Linear integration | Jira + Linear | Jira only | No | Jira + Linear |
| Management view | Executive dashboard | Technical dashboard | Basic | C-Level dashboard |
| Price | Freemium / ~$20/user | From $350/month | Free | On quote (> $500/month) |
The 4 vanity metrics to never use to evaluate dev AI:
❌ Lines of code / day → AI artificially inflates volume
❌ Number of commits / week → More commits ≠ more value
❌ Copilot acceptance rate → Accepting code ≠ quality code
❌ PRs merged / sprint → Splitting into micro-PRs inflates the counter
✅ Use instead:
Cycle time (ticket → production)
Change failure rate (% of failed deployments)
Quality ratio (time on features vs time on bugs/debt)
Median review time (stagnation = bottleneck)
Developer satisfaction (quarterly internal NPS)GitHub Actions configuration for automatic DORA metrics:
# .github/workflows/dora-metrics.yml
name: DORA Metrics
on:
deployment_status:
issues:
types: [opened, closed]
workflow_dispatch:
jobs:
collect:
runs-on: ubuntu-latest
steps:
- uses: dorametrics/github-action@v2
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
dashboard: true # Publishes to GitHub Pages