
AI and LLMs Inside FileMaker: What Actually Works in 2026
Production use cases: semantic search, extraction, classification and RAG, with provider choice and UK GDPR notes.
11 min read · Updated April 2026
Customers ask us two questions, usually in this order. Do you use AI to write the code? Yes. Then how do you protect, control and check AI generated code before a business runs on it? This paper is the long answer to the second question. AI made writing code faster. It did not make the rest of the job smaller. There is more code, more change and more to review, and the part of a build that keeps a business safe is the part AI does not do for you.
Updated November 2026 · 18 min read · By Rob Hopkins
The short version is that AI generated code was never the whole job. Every change we deliver is planned before it is written, reviewed by a senior person who did not write it, scanned, previewed, tested by the people who will use it, released in steps and watched once it is live. That was true before AI and it is more true now, because AI produces code faster than people can look at it.
We did not have to take our own word for that. In November 2026 Atlassian published the AI SDLC Index, built on DX data from more than 340,000 engineers at over 500 engineering organisations, from under 50 staff to over 10,000, across Europe, Asia and North America. It is the largest public picture of what AI is doing to software delivery. The first three parts of this paper walk through what it shows. The last two are about us: how we run a build, and what to ask anyone who builds software for you with AI.
The first thing the Index settles is what AI is actually used for. The answer is not "writing new features". Of the work developers hand to AI, the biggest single share is fixing bugs. After that comes planning and architecture, code review and documentation. New features are one of the smallest categories.
Share of AI sessions by type of work, August to October 2026, 180 companies. One session can cover more than one type, so the shares add to more than 100%. Atlassian AI SDLC Index (DX data).
| Share of AI sessions | |
|---|---|
| Bug fixes | 20.2% |
| Architecture and planning | 11.9% |
| Code reviews | 10.6% |
| Documentation | 10.5% |
| Refactors and rewrites | 10.3% |
| Feature enhancements | 9.9% |
| Test updates | 8.3% |
| Questions | 7.2% |
| Dependency or config updates | 6.4% |
| New features | 4.2% |
| Build or test failure fixes | 3.9% |
| Performance improvements | 2.2% |
| Vulnerability fixes and hardening | 2.0% |
| Data or migration fixes | 1.7% |
| Ideation | 1.7% |
| Data generation | 1.1% |
That matches how we use it. Claude Code is in every stage of a build with us, and most of the time it is doing the work that used to be the slow, careful part: tracing a bug, writing the documentation nobody had time for, tidying code that had grown untidy, reading a change and pointing at what looks wrong. It is a colleague in each stage. It is not a replacement for the stage.
That distinction matters for the second question. If AI only typed, the question of control would be about the typing. Because it works across planning, review and documentation too, the control has to sit across all of them.
The Index also asks developers how they rate each part of their work, and compares 2025 with 2026. Most things improved, and the gains are in the places you would hope. Documentation went up by nearly 13 points. Debugging in production, the development environment and the ability to do deep work all improved.
Survey responses, August to October 2025 (382 companies) and August to October 2026 (567 companies). Atlassian AI SDLC Index (DX data).
| 2025 | 2026 | |
|---|---|---|
| Documentation | 47.7% | 60.4% |
| Production debugging | 52.9% | 61.6% |
| Dev environment | 61.0% | 68.1% |
| Deep work | 59.0% | 63.4% |
| Build and test | 57.3% | 60.8% |
| Code maintainability | 77.2% | 79.6% |
| Engagement | 63.1% | 64.7% |
| Planning process | 64.5% | 65.3% |
| Decision-making | 65.8% | 66.6% |
| Cross-team collaboration | 56.5% | 57.2% |
| Change confidence | 79.7% | 78.7% |
| Clear direction | 74.1% | 72.1% |
| Local iteration speed | 77.3% | 74.6% |
| Incident handling | 87.3% | 84.4% |
Four things got worse. Confidence in a change, the speed of local iteration, incident handling and clear direction all fell. None of the drops is large, but they are all in the same place. With more code arriving, people are a little less sure each change is right, and a little less sure what they should be building. Neither of those is a typing problem. Both are process problems, and they are the two that AI on its own makes worse rather than better.
Here is the number we show customers first. The Index measured how long a change waits for its first human review, and split the changes by how much of the code AI wrote. A change where AI wrote between a fifth and two fifths of the code waited about half an hour. A change where AI wrote four fifths or more waited over an hour and a half.
Median minutes to first review, August to October 2026, 441 companies. The AI share is self-reported; the wait is measured from product data and counts only human reviews. Atlassian AI SDLC Index (DX data).
| Minutes to first review | |
|---|---|
| 20 to 40% AI-written | 31.2 min |
| 40 to 60% | 66.6 min |
| 60 to 80% | 72.6 min |
| 80% or more | 97.8 min |
There are two reasons a mostly AI-written change takes longer to review, and we see both. The first is volume. AI writes more code per change, and a reviewer reads all of it. The second is trust. A reviewer reading a colleague's change knows how that colleague thinks. A reviewer reading an AI's change has to check the thinking as well as the code, because the AI will produce something plausible whether or not it understood the problem.
Neither reason goes away. The review is the control, and the review is the part that got slower.
The review queue is where the extra code lands. In January 2026 the average human reviewer in the Index handled about 13 pull requests a month. By October it was over 22, and the last month alone added three.
Distinct pull requests reviewed per active human reviewer, January to October 2026, 627 companies. Atlassian AI SDLC Index (DX data).
| Month | Reviews per person |
|---|---|
| Jan 2026 | 12.8 |
| Feb 2026 | 13.8 |
| Mar 2026 | 16.3 |
| Apr 2026 | 15.8 |
| May 2026 | 15.0 |
| Jun 2026 | 17.2 |
| Jul 2026 | 18.0 |
| Aug 2026 | 17.5 |
| Sept 2026 | 19.3 |
| Oct 2026 | 22.3 |
Put the two charts together and the shape of the problem is clear. Each review takes longer, and there are more of them. A team that adds AI without adding review capacity does not get a faster project. It gets a longer queue, and the customer waits at the end of it.
This is the reason our process has a review rule rather than a review habit, which we come to below.
The Index tracks the share of merged code that AI wrote, and separately the share of that code still in the codebase afterwards. AI's share rose from under half in April to about seven tenths in October. Over the same months, the share that survived fell from four fifths to under three fifths, before recovering a little in October.
April to October 2026, 227 companies. The AI-written share is self-reported; survival is measured in product data as the share of AI-written code still present in commits. Atlassian AI SDLC Index (DX data).
| Month | Written by AI | Still there a month on |
|---|---|---|
| Apr 2026 | 49.0% | 79.9% |
| May 2026 | 54.7% | 79.7% |
| Jun 2026 | 56.4% | 75.1% |
| Jul 2026 | 59.4% | 66.0% |
| Aug 2026 | 65.5% | 61.8% |
| Sept 2026 | 63.5% | 58.9% |
| Oct 2026 | 69.7% | 68.2% |
Read one way, that is alarming. A third of AI-written code is being thrown away. Read the other way, it is the review and testing doing their job. Code that should not ship is being caught and removed. Either way it is work, and somebody has to do it. The survival line is the clearest evidence in the Index that "the AI wrote it" is the start of a change, not the end.
If AI writes most of the code and writes it quickly, projects should be arriving faster. The Index says they are not, and explains why.
Developers in the Index spend about 58% of their time building new things. That number has barely moved since January 2025, through the whole period in which AI's share of the code went from a minority to a majority. The time saved writing went somewhere, and it went into the rest of the lifecycle.
January 2025 to October 2026, 214 companies, self-reported. Atlassian AI SDLC Index (DX data).
| Month | Time on new work |
|---|---|
| Jan 2025 | 55.3% |
| Feb 2025 | 54.9% |
| Mar 2025 | 54.9% |
| Apr 2025 | 56.8% |
| May 2025 | 55.3% |
| Jun 2025 | 56.5% |
| Jul 2025 | 56.1% |
| Aug 2025 | 57.0% |
| Sept 2025 | 56.3% |
| Oct 2025 | 55.4% |
| Nov 2025 | 55.8% |
| Dec 2025 | 57.6% |
| Jan 2026 | 56.8% |
| Feb 2026 | 56.5% |
| Mar 2026 | 56.9% |
| Apr 2026 | 56.8% |
| May 2026 | 56.8% |
| Jun 2026 | 56.9% |
| Jul 2026 | 57.7% |
| Aug 2026 | 58.5% |
| Sept 2026 | 58.4% |
| Oct 2026 | 57.8% |
Where, exactly, is the next chart. Developers lose about one working day in five to friction, which the Index puts at 409 hours, or 51 days, a year. The two largest sources are the same size and they are both waiting: waiting for automated checks to run, and waiting for a review.
August to October 2026, 475 companies, self-reported. The Index puts the total at 20% of work hours, 409 hours or 51 days a year. Atlassian AI SDLC Index (DX data).
| Hours a year | |
|---|---|
| Waiting for CI | 86.5 h |
| Waiting for a review | 86.5 h |
| Understanding the code | 71.5 h |
| Waiting for tests | 58.0 h |
| Looking for information | 55.0 h |
| Dev environment toil | 47.5 h |
Two more figures round this out. The amount organisations spent on AI for each unit of work delivered rose more than fourfold between April and September before it came down, so for six months spend grew faster than output. And the share of pull requests that get an automated review by a bot has grown, but slowly, from about 15% to about 21% in a year.
April to October 2026, 488 companies. Output is merged pull requests weighted by size and complexity. Total spend is not fully captured, so these are a floor. Atlassian AI SDLC Index (DX data).
| US dollars per unit of output | |
|---|---|
| Apr | $8.85 |
| May | $17.84 |
| Jun | $23.83 |
| Jul | $32.33 |
| Aug | $38.77 |
| Sep | $39.98 |
| Oct | $29.11 |
November 2025 to October 2026, 630 companies, among companies using bot reviewers on more than 5% of pull requests. Atlassian AI SDLC Index (DX data).
| Month | Pull requests with a bot review |
|---|---|
| Nov 2025 | 15.3% |
| Dec 2025 | 15.9% |
| Jan 2026 | 16.4% |
| Feb 2026 | 17.1% |
| Mar 2026 | 18.0% |
| Apr 2026 | 19.3% |
| May 2026 | 19.7% |
| Jun 2026 | 20.5% |
| Jul 2026 | 21.0% |
| Aug 2026 | 21.6% |
| Sept 2026 | 21.6% |
| Oct 2026 | 21.2% |
This is the speed gap. The code got faster. Deciding what to build, reviewing it, testing it and releasing it did not, and that is where projects spend their time now. For a customer, the practical consequence is simple. A quote for a system built with AI can be lower than it would have been. The date is not automatically sooner, and a builder who promises both without saying what happens to the review and the testing is promising the part the data says they cannot have.
Before turning to our own process, one more thing from the Index. It asks which parts of a developer's work matter most for four outcomes: speed, quality, how easy the work is to do and how easy it is to release. Then it plots each against how well developers rate it today. The parts that matter most and are rated worst are the ones to fix first.
| Outcome | Matters most, rated worst | Matters most, already working |
|---|---|---|
| Speed | Production debugging, documentation, cross-team collaboration | Decision-making, incident handling, dev environment, change confidence |
| Quality | Planning process, documentation | Incident handling, change confidence, local iteration speed, clear direction, code maintainability |
| Ease of delivery | Production debugging, documentation, cross-team collaboration | Incident handling, change confidence, local iteration speed, dev environment |
| Ease of release | Build and test, documentation, production debugging | Incident handling, change confidence, dev environment, local iteration speed |
The things that come up again and again on the "fix first" side are documentation, planning, build and test, and debugging in production. Only about six in ten organisations have most of their developers rating those well. They are the unglamorous parts of a lifecycle. They are also the parts AI does not do for you, and the parts a good delivery partner spends its time on.
So here is the second answer, step by step. This is how a web or Lovable build runs with us. It applies to every change, from a first release to a one-line fix two years later.
Every change, from a first release to a one-line fix years later, goes round this loop. AI makes the second step faster. The other five are where the control is.
What the change has to do and what it touches, written down and agreed. A senior specialist owns the design.
agreedClaude Code writes with the engineer, in pieces small enough for someone else to read in one sitting.
Claude CodeGitHuba pull requestSomeone who did not write the change reads the code and its impact analysis. The AI never approves its own work.
approvedThe code and its dependencies scanned for weaknesses, the change built and seen working on a preview.
AikidoVercelcleanTested by the people who will use it, with no cap, then released in a small step that can be undone.
liveErrors and usage reported as they happen, to the team that built it.
SentryPostHogNothing is built until we have examined what the system has to do and asked the awkward questions. On a new build that is a workshop and a clickable prototype your team uses before any production code exists. On a change to a live system it is a written description of the change and its impact, agreed before work starts. A senior specialist owns the design from end to end, so nothing is lost in a handover.
AI helps here. Claude Code reads the existing system, drafts the description of a change and asks questions we might not have thought of. A person decides. The Index's finding that planning and documentation are the strongest levers on quality is one we would have guessed, because they are the levers we pull hardest.
Claude Code writes code with the engineer, and the engineer works in small pieces, each one small enough for someone else to read in one sitting. That is a deliberate response to the review-wait chart. A large change where AI wrote most of the code is the slowest thing in the Index to review, so we do not make them.
There are three things the AI never does on our builds. It never pushes anything to a live system. It never touches production data. It never approves its own change. Those are not preferences. They are rules, and the tooling enforces them.
Every change travels through GitHub as a pull request. Every pull request is approved by a Senior Solution Engineer, and it cannot be the engineer who worked with the AI to make the change. Two people see every change, and one of them did not write it.
The reviewer reads the code change itself and the impact analysis that goes with it. Both are done by hand. We automate a great deal, and we come to that next, but the judgement about whether a change does what the business needs and nothing else is a person's.
Around the human review, a set of checks runs on every change without anyone asking. The code and its dependencies are scanned for weaknesses with Aikido. The change is built and deployed to a preview, so it can be seen working before it goes anywhere near the live system. Only after the scan, the preview and the approval can a change be released.
Then the people who will use it test it. We put no cap on testing. It ships when it works for your people, not when it works for us. Appleby's team tried its new system with real work for weeks before the old one was switched off.
A system is released area by area, and then changed in small, frequent releases. Bascule's PolicyHub has had more than a thousand changes since February 2026, each previewed before release. Appleby's system has gone out at about two releases a week since it went live. Small releases are easier to review, easier to test and easier to undo, which is the whole point.
Errors are reported the moment they happen, through Sentry, and usage is watched through PostHog, so we know when something breaks before the customer does. The team that built the system is the team on Teams every day afterwards, and support starts with the first release.
That is the loop. The Index's numbers describe what happens when AI is added to a lifecycle and the lifecycle is left as it was. Ours was built so that adding AI makes the writing faster and leaves every control where it was.
The two questions customers ask us are the right ones. Here are the six we would put to anyone, with our answers, so you can compare.
If a builder cannot answer the third question with a rule, the Index's review charts are what happens next.
It is as safe as the process around it. The Index shows that about a third of AI-written code is removed before it survives a month, which is the review and testing working. Code that goes straight from an AI to a live system with no second person, no scan and no preview is not safe, whoever the AI is. Code that has been reviewed by someone who did not write it, scanned, previewed and tested by its users is as safe as any other code.
It makes writing code faster. The Index finds that the time developers spend on new work has not changed since January 2025, because the time saved writing moves into review, testing and release. A project gets faster when the whole lifecycle is set up for the volume, not when AI is added to one part of it.
Often, yes, and ours usually do, because the writing is a smaller share of the work. What should not be cheaper is the review, the testing and the release, because they are the part that protects you. A quote that is much lower than it was and promises a much earlier date is usually a quote that has cut those.
This paper is about our web and Lovable builds, where every change travels through GitHub as a pull request and the process above applies in full. FileMaker changes are managed differently, and we have written about [AI inside FileMaker systems](/resources/ai-llms-inside-filemaker-2026) separately.
If you are weighing up a system built with AI, the question to ask any builder is not whether they use it. It is who checks the work, and whether that is a rule. Ask us. We will walk you through a real change from the written description to the release, with the review and the checks in between.