An AI coding assistant suggests code as you type, answers questions about a codebase, and in its newer forms takes a task and works through it on its own. Adoption has been extraordinarily fast and the evidence about what it does to quality and speed is more mixed than the marketing suggests. This guide sets out what the published research actually found, and what to do about it.
Updated September 2026.
What the tools do now
GitHub’s own documentation splits the behaviour of its assistant into two modes. The assistive mode is the familiar one: the tool reacts while you type, offering completions, explanations and answers about the surrounding code, and nothing lands until you accept it. The agentic mode inverts the relationship. You hand over a goal, and the tool reads around the repository, drafts a plan, changes files, runs whatever tooling it needs and returns a finished attempt. GitHub states that reviewing and approving the result stays your responsibility in both cases.
That second mode is the important shift. It turns the tool into an AI agent operating inside your repository, which changes both what it can accomplish and what it can break.
How many developers use one
- The 2025 Stack Overflow Developer Survey, with 33,662 respondents, found 84 percent using or planning to use AI tools, and 51 percent of professional developers using them daily.
- The 2025 DORA report, based on responses from nearly 5,000 technology professionals and published on 24 September 2025, found 90 percent using AI at work.
- GitHub Octoverse 2025 reported that nearly 80 percent of new developers on GitHub used Copilot in their first week, and that the Copilot coding agent authored more than one million pull requests between May and September 2025.
What the research says about speed
The most rigorous study so far points the other way from the sales pitch. In July 2025 METR published a randomised controlled trial in which 16 experienced open-source developers completed 246 real tasks in their own mature repositories, with AI randomly allowed or disallowed per task. Developers took 19 percent longer on the tasks where AI was permitted. Before starting they had forecast a 24 percent speedup, and afterwards they still believed they had been 20 percent faster.
METR is careful about what this does and does not show. It does not claim that AI fails to speed up most developers, and it notes the setting was specific: very experienced maintainers, large codebases they knew well, and tools available between February and June 2025. The useful lesson is narrower and still holds. Your sense of whether an AI coding assistant is helping is not reliable evidence that it is.
The 2025 DORA report found something compatible. AI adoption showed a positive relationship with software delivery throughput and product performance, and a negative relationship with software delivery stability. DORA’s framing is that AI acts as an amplifier of whatever practices an organisation already has.
The trust and security gap
Stack Overflow’s 2025 survey found only 3.1 percent of respondents highly trust the accuracy of AI output, while 45.7 percent actively distrust it. Favourability fell to 60 percent. The top frustration, cited by 66 percent, was solutions that are almost right but not quite, and 45.2 percent said debugging AI-generated code takes more time than they expect.
On security the numbers are harder. Veracode’s 2025 GenAI Code Security Report evaluated more than 100 large language models across Java, Python, C# and JavaScript and found that 45 percent of code samples failed security tests by introducing OWASP Top 10 vulnerabilities. Java failed 72 percent of the time, C# 45 percent, JavaScript 43 percent and Python 38 percent. Cross-site scripting defences failed in 86 percent of relevant samples. Veracode also reported that security performance stayed flat regardless of model size or sophistication, so waiting for a better model is not a plan.
There is a supply chain angle too. At USENIX Security 2025, Spracklen and colleagues analysed 576,000 generated code samples from 16 models and found package hallucination rates of at least 5.2 percent for commercial models and 21.7 percent for open-source ones, producing 205,474 unique fabricated package names. Attackers can register those names and wait, which makes a careless install into a supply chain attack.
6 rules for using an AI coding assistant safely
- Treat every suggestion as untrusted input. Read it before you accept it. GitHub states plainly that the reviewing and approving stays with you.
- Keep the diffs small. A change you can review in one sitting is a change you will actually review. Large generated patches get skimmed.
- Check that every dependency exists and is the package you meant, before installing it. Fabricated package names are common enough to be a standing hazard.
- Run security scanning on generated code as routine, not as an exception. Given a 45 percent failure rate in controlled tests, sampling is not enough.
- Watch stability, not just throughput. DORA found delivery stability moving the wrong way as adoption rose. Track change failure rate and time to restore alongside speed.
- Know when to switch it off. In METR’s trial the losses came in familiar, mature codebases. Reach for the tool on unfamiliar ground and drop it where you already know the answer.
Where the tools genuinely earn their place
Boilerplate, unfamiliar languages, test scaffolding, one-off scripts, explaining code someone else wrote, and first drafts you intend to rewrite. What the evidence does not support is unreviewed code going into production, which is the pattern people mean by vibe coding. When an answer matters, check it against a primary source, the same way you would any other claim. If you are considering this work as a career rather than a convenience, our guide to the AI engineer role covers what employers ask for.
Common questions
What is an AI coding assistant? It is a tool that uses a language model to suggest code, answer questions about a codebase and, in agent mode, carry out multi-step tasks such as editing files and opening pull requests for your review.
Does an AI coding assistant make developers faster? Not always. METR’s 2025 randomised trial found 16 experienced open-source developers were 19 percent slower with AI allowed, although they believed they were faster. DORA found gains in throughput but losses in delivery stability.
Is AI-generated code secure? Often not. Veracode’s 2025 report found 45 percent of AI-generated samples introduced OWASP Top 10 vulnerabilities, with no improvement from newer or larger models. Generated code needs the same scanning and review as any other.
Do developers trust these tools? Mostly not fully. Stack Overflow’s 2025 survey found only 3.1 percent highly trust AI output accuracy, 45.7 percent actively distrust it, and 66 percent are frustrated by answers that are almost right.
Will an AI coding assistant replace programmers? The hiring data does not show that yet. Indeed Hiring Lab found US software development postings rising into 2026, though heavily weighted towards senior roles and jobs with AI in the title.
Sources and further reading
Where the figures and rules above come from, so you can check them:
- What is GitHub Copilot, assistive and agentic modes: GitHub Docs
- 2025 Developer Survey, AI adoption and trust: Stack Overflow
- Measuring the impact of early-2025 AI on experienced developer productivity (July 2025): METR
- Announcing the 2025 DORA report (September 2025): Google Cloud
- 2025 GenAI Code Security Report: Veracode
- We Have a Package for You! Package hallucinations by code generating LLMs: USENIX Security 25
- Octoverse 2025, Copilot adoption and coding agent pull requests: GitHub
Photo credits: Coding Shots Annual Plan high res-5 by Matthew (WMF), CC BY-SA 3.0, via Wikimedia Commons. Screen-python-code-matplotlib-physics-simulation by MikeRun, CC BY-SA 4.0, via Wikimedia Commons.
Join the discussion