Tim Anderson reviews research on how generative AI affects software development, revealing that longstanding best practices play a bigger role in success than the specific AI tools themselves.

Research: AI’s Impact on Developer Productivity Hinges on Best Practices

Author: Tim Anderson

Generative AI tools have the potential to greatly accelerate code writing, but their overall effect on developer productivity and code quality is more nuanced than it may seem at first. Research from Atlassian’s DX team, Google DORA, and LaunchDarkly underscores the importance of maintaining robust pre-existing development processes and practices, as these have a greater impact on outcomes than the simple adoption of AI.

Key Research Findings

  • Variation in Productivity Gains: Atlassian’s DX aggregated anonymized data from 135,000 developers at 400 organizations, showing that some companies see major productivity and confidence gains from AI, while others experience setbacks.
  • Limited Average Gains: Google DORA reports modest average improvements (2.6% in code quality, 0.11% in change failure rate) from generative AI—but these averages mask wide variations.
  • Central Role of Best Practices: According to DX’s Justin Reock, teams that perform comprehensive work on software development lifecycle (SDLC) and code hygiene experience the most gains. The same factors that fostered productivity before AI tools remain vital.

Factors Affecting AI Impact

  • Time Savings: Simple code completion with AI saves developers about 3.8 hours per week on average. However, the biggest obstacles to productivity often lie elsewhere: meetings, context switches, build delays, and review queues.
  • Use Cases: The top use case for AI in development is stack trace analysis, according to DX’s research. Other high-impact areas include pull request generation and code search.
  • Variation by Role and Language: Junior developers engage more actively with AI tools, but senior engineers derive greater time savings due to more efficient workflows. Modern programming languages (e.g., Go) see higher AI productivity gains than older ones (e.g., COBOL).
  • Psychological Safety: The most important factor for success, according to Reock, remains team psychological safety—a pre-AI principle even more critical with AI adoption.

Cautionary Notes

  • Quality vs. Quantity: AI users create substantially more pull requests, but researchers warn this may include a lot of non-productive code if teams don’t maintain high standards.
  • AI Trust and Learning: While 94% of surveyed developers see coding speedups, 91% have low trust in shipping AI-generated code to production, and 81% ship code with unresolved risks due to delivery pressures.
  • Developer Learning Concerns: Some worry that AI can impede developer learning if used uncritically—but personal motivation to study generated code remains a differentiator, as with past cut-and-paste tendencies.
  • Companies adopting AI for coding should focus on reinforcing existing best practices and fostering psychological safety to maximize benefits.
  • The full impact of AI varies widely by organization, workflow, language, and developer culture—AI is a powerful tool, but not a silver bullet.
  • Ongoing assessment and adaptation of processes—not just AI tool adoption—are vital for sustained improvements.

References:

  • Atlassian DX guide to AI-assisted engineering: https://getdx.com/guide/ai-assisted-engineering/
  • LaunchDarkly AI control gap report: https://launchdarkly.com/ai-control-gap/
  • Google DORA Reports

For further information or to discuss this article, contact devclass@sitpub.com.

This post appeared first on “DevClass”. Read the entire article here