How does GitHub Copilot Business create value?

GitHub Copilot speeds up developer tasks:

  • Code generation
  • Code documentation
  • Code refactoring
  • Code understanding
  • Complex coding tasks

How does saving developer time create value?

  1. Direct Time Savings: Reduced task duration correlates to less developer time paid for the same work.
  2. Increased Productive Capacity: Developers create value in other areas with time saved. Ex. improving team processes, mentoring a junior developer, refactoring, etc.
  3. Downstream Benefits: Organizational improvements made as a result of sped up developer tasks. Ex. faster PRs, faster releases, reduced tech debt (tests, defects, vulns), etc.
  4. Mitigated Risk: Unit tests written or vulnerabilities fixed that otherwise would not have taken place without GitHub Copilot.

Sources

What can my company expect to gain by adopting Copilot?

In addition to the metrics provided in the studies above on productivity and developer satisfaction, here are some gains experienced by our customers from GitHub Copilot.

  • Duolingo: 25% increase in developer speed with GitHub Copilot
  • Coyote Logistics: 50% less time to write Terraform config files with GitHub Copilot
  • TomTom: 85% of developers feel more productive with GitHub Copilot
  • CloudZero: 4x faster at coding when using Copilot, 300% faster at bug fixes when using Copilot
  • Hover: 85% of boilerplate code written entirely with GitHub Copilot
  • Mercado Libre: ~50% reduction in time spent writing code with GitHub Copilot with 9000+ developers using Copilot

How should Copilot impact be measured?

Core Resources

1. Developer Surveys

Self reported time saved, productivity, and satisfaction gains.

2. GitHub Copilot Metrics API

Sample Dashboards

3. Developer Productivity/Experience Measurements

Although it is difficult to isolate the impact of one tool in the complex world of Developer Productivity. The best way to measure the impact of GitHub Copilot on Developer Productivity is to measure Developer Productivity. This is easier said than done but some helpful resources include:

4. Copilot Value Calculator

The GitHub Value Engineering team has created a Self-Service Copilot Productivity Impact Calculator. The calculator is a spreadsheet that contains input values for variables like # of developers using Copilot, the percent of time spent coding, the percent of time saved per task, etc. It takes these inputs to generate several business value calculations. Please reach out to your GitHub Account Representative to receive a copy.

What are some GitHub Copilot value assessment pitfalls to avoid?

  • Lack of Rollout Strategy: Select a pilot team, plan a rollout, provide training, office hours, and additional support
  • Knowledge Gaps: GitHub Copilot is easy to pick up but best results are achieved with a fundamental understanding of prompt engineering
  • Survey Fatigue: Rotate surveys across teams, keep them concise, do surveys temporarily, or as a spot check
  • Biased Self-Reporting: Validate reasoning
  • Change Resistance: Involve teams early and incorporate feedback
  • New Bottlenecks: Increased development speed could exacerbate other functions like security review
  • Unrealistic Expectations: GitHub Copilot is a tool, the developer is still in charge
  • Varied Impact: GitHub Copilot results may vary across team function, technology used, average time coding, etc.