There is something slightly weird about using Git history as an experiment.
A commit from 2025 contains information that did not exist in 2023. If I construct an executable rule from an edit in 2023, freeze it, and prevent anything that happens later from changing that rule, then a later commit can act as a label that was genuinely unavailable when the rule was created.
That makes it possible to ask a question I have become increasingly interested in: can an old accepted edit tell me anything about a change a developer will make in the future?
Not in the vague sense that two diffs look similar. I mean something stricter. Construct an executable interpretation of an accepted edit, stop changing it, eventually give it source from a later point in repository history, and only afterward reveal what a developer actually changed.
Does the old program ever produce the same transformation?
One AttuneFinite experiment found a particularly clean example.
In November 2023 (d686dcfa, “chore: include total and new user charts”), a Documenso developer was working on a chart of new-user growth. The function queried monthly signup counts and converted those rows into chart data:
export const getUserMonthlyGrowth = async () => {
const result =
await prisma.$queryRaw<GetUserMonthlyGrowthQueryResult>`
SELECT
DATE_TRUNC('month', "createdAt") AS "month",
COUNT("id") AS "count"
FROM "User"
GROUP BY "month"
ORDER BY "month" DESC
LIMIT 12
`;
return result.map((row) => ({
month: DateTime.fromJSDate(row.month).toFormat('yyyy-MM'),
count: Number(row.count),
}));
};
The chart was being extended with a running total. The commit changed the SQL and result types too, but one small part of the accepted edit was this addition to the object returned for each month:
return result.map((row) => ({
month: DateTime.fromJSDate(row.month).toFormat('yyyy-MM'),
count: Number(row.count),
+ cume_count: Number(row.cume_count),
}));
AttuneFinite took that historical edit and constructed an executable interpretation of it:
language js(typescript)
js"{
month: DateTime.fromJSDate(row.month).toFormat('yyyy-MM'),
count: Number(row.count),
}" => js"{
month: DateTime.fromJSDate(row.month).toFormat('yyyy-MM'),
count: Number(row.count),
cume_count: Number(row.cume_count),
}"
Grit is a structural search-and-rewrite language. The expression on the left describes TypeScript syntax to find; the expression on the right describes the syntax that should replace it. This particular program is deliberately literal: if it finds exactly that two-property object, it turns it into the three-property version.
Then the experiment froze the program. “Frozen” matters here: later repository history was not allowed to modify the rule, select a different rule, or tell the construction process whether the rule would work in the future.
Roughly 2,266 corpus commits and 450 days later in the experiment's chronological history (6993c52b, “feat: mau”), Documenso added a different analytics feature: an admin report for monthly active users. Immediately before that later edit, the relevant function looked like this:
export const getMonthlyActiveUsers = async () => {
const result =
await prisma.$queryRaw<GetMonthlyActiveUsersQueryResult>`
SELECT
DATE_TRUNC('month', "lastSignedIn") AS "month",
COUNT(DISTINCT "id") as "count"
FROM "User"
WHERE "lastSignedIn" >= NOW() - INTERVAL '1 year'
GROUP BY "month"
ORDER BY "month" DESC
LIMIT 12
`;
return result.map((row) => ({
month: DateTime.fromJSDate(row.month).toFormat('yyyy-MM'),
count: Number(row.count),
}));
};
At this point, the later accepted edit was still hidden from the part of the experiment evaluating the old rule.
AttuneFinite ran the frozen 2023 program against the complete historical version of this 2025 file. It changed one thing:
return result.map((row) => ({
month: DateTime.fromJSDate(row.month).toFormat('yyyy-MM'),
count: Number(row.count),
cume_count: Number(row.cume_count),
}));
Then the later human edit was revealed.
It contained exactly the same transformation:
return result.map((row) => ({
month: DateTime.fromJSDate(row.month).toFormat('yyyy-MM'),
count: Number(row.count),
+ cume_count: Number(row.cume_count),
}));
The frozen program changed that object and nothing else. Its output matched the expected historical target byte for byte. I later reran the same frozen program independently against both historical source states and reproduced the result with one operation.
The sealed experiment records the concrete details:
seed commit d686dcfa90109829f3a3ee6dd7d4465e3d689746
future commit 6993c52b2ae7560dc8a42169c5ea75df911b2a3e
distance 2,266 corpus commits between them
elapsed time 449 days (~450)
program cume_count object addition; selection digest 6498536eb51277fe66a50bbb7a2e33583e16c0c71f515898f06db60ca2cc2d5a
operations 1
future exact yes
“Future exact” is the experiment’s term for the recorded outcome: the later accepted result was withheld until the old program had already run, and the program’s output — produced in one operation on the complete later before-file — equaled it byte for byte.
That is what I mean by “predicting the future” here.
The old program did not predict the entire 2025 commit. The developer also changed the SQL, result types, and chart code. The experiment isolates one accepted transformation, applies a frozen program to the complete historical before-file, and asks whether the resulting file is exactly what that transformation would have produced, with no additional edits.
That narrower result is the one that interested me. A developer made a programming decision in one feature, AttuneFinite represented one interpretation of it as a small executable program, and more than a year later the same program still described part of another accepted change in a different feature.
It made me wonder how much supervision is already hiding inside repository history.
Coding agents have made generating plausible code much cheaper. The harder problem is increasingly deciding which plausible code belongs in a particular repository.
Tests provide strong signals, but mature codebases contain a great deal of knowledge that is not fully expressed in tests. There are architectural boundaries, preferred abstractions, migrations in progress, conventions around APIs, and hundreds of decisions that become obvious after working in a repository for months.
Humans communicate this information constantly during review. A patch can be perfectly reasonable in isolation, but somebody familiar with the codebase notices that it violates a local assumption and corrects it.
Once the patch is merged, most of that supervision disappears.
That seems wasteful to me. If coding agents are going to become useful long-term collaborators, I want them to get better at the particular repositories in which they work.
The mechanism for that specialization could eventually involve fine-tuning, learned verifiers, retrieval, persistent policies, or some combination of them. Before choosing a mechanism, though, there is a more basic question: where does the signal come from?
Git history is already full of previous human decisions. AttuneFinite asks whether some of those decisions can be converted into executable evidence rather than left as inert diffs.
The running-total example was not unique. In one experiment, frozen programs exactly reproduced 1,864 later accepted transformations. Of those, 542 happened in a different file from the edit used to construct the program. Another 315 happened in the same file after its complete source had changed.
Those numbers need to be interpreted carefully. The population was constructed to study recurring historical transformations; it is not an estimate of how often arbitrary edits are predictable. Exact bytes are also an intentionally harsh criterion that rejects many alternative implementations that might be equally good.
The result is narrower: some accepted edits contain information that remains executable later.
The immediate problem is figuring out what that information actually is.
An accepted edit does not arrive with its intended abstraction attached.
Consider another historical change:
-import { PrismaClient } from "@documenso/prisma";
+import PrismaClient from "@documenso/prisma";
This program explains it:
js"{ PrismaClient }"
=> js"PrismaClient"
So does this:
js"import { PrismaClient } from \"@documenso/prisma\";"
=> js"import PrismaClient from \"@documenso/prisma\";"
And so does this:
js"import { PrismaClient } from $module;"
=> js"import PrismaClient from $module;"
These were three real executable interpretations retained for the same historical edit.
They agree on the observation that produced them, but they make different claims about what matters.
The first program operates on a small syntax fragment. The second encodes the whole import literally. The third contains a capture, $module.
A capture is a variable inside the structural pattern. In:
js"import { PrismaClient } from $module;"
=> js"import PrismaClient from $module;"
$module means roughly: whatever syntax appears in the module position when this pattern matches, remember it and place that same syntax into the replacement.
Given:
import { PrismaClient } from "@documenso/prisma";
the syntax representing "@documenso/prisma" gets bound to $module.
The important difference is that the rule itself no longer says that @documenso/prisma is essential. It could, in principle, match the same import shape from another module and preserve that other module in the output.
The literal program says the module identity matters. The captured program says it does not.
Both perfectly explain the observation they started with.
One accepted edit does not uniquely determine the rule behind it.
This is where the problem started reminding me of teaching.
For a few years I taught kids to program. A lot of them were extremely smart; one six-year-old could beat my ass at chess.
The useful difference between us was not necessarily raw intelligence. I had mostly spent more time being confused by software.
I knew that not understanding something for twenty minutes was survivable. I knew the documentation might suck and the first explanation might be wrong. More importantly, I knew how to make a problem smaller, change one thing, run it again, and stay with it long enough for something to click.
That became a large part of how I taught.
I also did not want to teach programming as a list of sanctioned answers. Programming languages give you an enormous amount of room to express yourself, and some of the most interesting things happened when a student did something I had never shown them.
That freedom also creates ambiguity.
Suppose I show someone:
record.oldField;
and tell them the accepted result is:
record.newField;
Maybe they learned:
js"record.oldField"
=> js"record.newField"
Maybe they learned:
js"$receiver.oldField"
=> js"$receiver.newField"
The first says record matters. The second says any receiver is acceptable.
On the one example I provided, both interpretations are correct.
The useful response is not to guess which lesson sounds smarter. It is to find another example.
Program synthesis sounds more magical than some of the machinery I actually want to use.
One useful strategy is basically to try every reasonable way of forgetting part of the example.
Suppose an accepted edit contains:
return [
- record.oldField,
+ record.newField,
record.newField,
distractor.oldField,
];
Start with a literal interpretation:
js"[record.oldField, record.newField, distractor.oldField]"
=>
js"[record.newField, record.newField, distractor.oldField]"
Now look at the syntax that remained unchanged. Maybe the exact identity of record is incidental:
js"[$x0.oldField, record.newField, distractor.oldField]"
=>
js"[$x0.newField, record.newField, distractor.oldField]"
Maybe more of the stable syntax is incidental:
js"[$x0.oldField, $x1.newField, $x2.oldField]"
=>
js"[$x0.newField, $x1.newField, $x2.oldField]"
Each eligible stable syntax piece gives a binary choice: keep it literal or turn it into a hole.
A “hole” here is just a position the program is allowed to capture rather than insisting on one literal piece of syntax. If there are eight eligible pieces, each can independently stay literal or become a capture.
That gives:
2⁸ = 256
possible subsets.
That is computationally tiny. There is little reason to guess when the space can simply be enumerated.
The generator does this with a bit mask over stable syntax. Each bit says whether one eligible syntax fragment should remain literal or become a capture such as $x0 or $x1. Every resulting candidate is rendered as an ordinary Grit program.
Each program then has to survive the real runtime.
Conceptually:
candidate
│
▼
compiles? ─────── no ──> reject
│ yes
▼
makes a change? ─ no ──> reject
│ yes
▼
reproduces the
accepted source? ─ no ─> reject
│ yes
▼
retain
The production path really does compile the rendered Grit, execute it against historical source, and retain it only when the resulting source equals the accepted target exactly.
At this point synthesis has produced a set of executable hypotheses that all explain the same example.
But that is only half of what synthesis does in AttuneFinite.
Once several hypotheses survive, the system can turn the problem around. Instead of synthesizing another candidate rule, it can synthesize source code designed to make the surviving rules disagree.
Those generated source states are what I call worlds.
So the synthesis loop has two complementary outputs:
accepted edit
↓
synthesize candidate programs
↓
several programs explain the same edit
↓
synthesize source states where they disagree
↓
observe which distinctions are actually exposed
The first synthesis problem asks:
What executable programs could explain this edit?
The second asks:
What source could distinguish those programs?
That relationship is important. Worlds are not an unrelated testing layer added after synthesis. They are themselves synthesized counterexamples: concrete source states generated because the current hypotheses are still ambiguous.
I think this is more important than the particular enumeration trick. A model does not necessarily need to supervise every other model, and ambiguity does not have to be resolved by choosing whichever explanation sounds smartest.
Source code gives an unusual amount of deterministic structure. The source parses or it does not. The transformation compiles or it does not. It matches or it does not. The output equals the accepted evidence or it does not.
Tests add another source of evidence. Synthesized worlds expose behavioral differences. Static analyses can ask more specific semantic questions about those differences.
Programming has a huge generative space, but it also exposes a lot of mechanically checkable structure.
Code is surprisingly supervisable.
Once an executable interpretation explains the edit that produced it, continuing to modify the program until it explains later examples would destroy the experiment.
The information boundary therefore has to be explicit.
At an earlier accepted edit, the construction side gets the source before and after the change. It constructs executable interpretations, runs them against the earlier source, and retains candidates that reproduce the accepted result.
Then the program freezes.
At a later historical edit, only the source as it existed before that later edit is supplied to the old program. The later accepted result remains hidden until the program has already produced an output.
The chronology is:
observe A
↓
construct candidate programs
↓
execute them on A.before
↓
retain programs that reproduce A.after
↓
freeze
↓
reveal B.before
↓
run old frozen program
↓
produce an output
↓
reveal B.after
↓
compare
Here, A.before and A.after are the historical source before and after the earlier accepted transformation. B.before is a later source state, and B.after is the corresponding accepted target that the evaluator withholds until the old program has run.
A surprising amount of AttuneFinite exists simply to make it difficult to cheat on this boundary accidentally.
In the smaller recurrence apparatus, the evaluator is close to the diagram:
# Execute the old program on the source that existed
# before the later human edit.
var result = program.value().run(
future.path,
future.before,
)
var change = _change_from_result(
result.rewritten,
result.operation_count,
future.path,
future.before,
)
# Only now ask whether the old program independently
# reproduced the later accepted target.
var exact = (
Bool(change)
and change.value().after == future.after
)
The answer-bearing recurrence fixtures even live behind a test-only boundary:
"""Frozen answer-bearing recurrence fixtures.
Only tests may import this module.
src/ must remain blind to B.
"""
I like this because the scientific rule is not only prose in a README. The code has to respect it.
Experiment 25 applies the same basic idea at larger scale. Its construction phase can inspect the first historical example and prove that a candidate reproduces it, but later examples, their accepted future results, and the measurements of whether the candidate succeeds on them remain unavailable until after candidate selection is complete.
This is one reason Git history is useful as an experimental substrate. It supplies examples that existed independently of the hypothesis currently being tested.
History, however, is slow.
Synthesized worlds solve the complementary problem. Git gives naturally occurring future examples that I did not design; world synthesis gives deliberately constructed examples that force ambiguous hypotheses apart now.
Both are useful for different reasons.
Take these two programs:
js"record.oldField"
=> js"record.newField"
and:
js"$receiver.oldField"
=> js"$receiver.newField"
On:
record.oldField;
they agree.
That observation provides no evidence about whether the receiver matters.
The synthesis machinery can therefore ask what source change would place those two programs in different behavioral situations.
One useful answer is:
const record = {
oldField: 1,
newField: 2,
};
const distractor = {
oldField: 3,
newField: 4,
};
record.oldField;
distractor.oldField;
This is a synthesized world: a controlled source state generated because the current candidate programs need somewhere to disagree.
The literal program contains record directly. The captured program is allowed to bind another receiver in the same syntactic position. Adding distractor.oldField creates a source state in which that difference can become observable.
AttuneFinite constructs worlds like this mechanically rather than editing raw source strings blindly. One experiment first finds the syntax node corresponding to the original access:
var selected = -1
for index in range(len(syntax.entries())):
if (
syntax.entries()[index].kind
== syntax.grammar.kind("member_expression")
and syntax.text(index) == "record.oldField"
):
selected = index
A member_expression is the syntax-tree node representing an access like record.oldField. The experiment identifies the intended node by both its syntax kind and its source text.
It can then synthesize a new world by changing only that controlled piece:
var edits: List[SourceEdit] = [
SourceEdit(
syntax.identity(),
start,
end,
"distractor.oldField",
)
]
var changed = apply_edits(syntax, edits)
if not changed.valid:
raise Error("control produced invalid native syntax")
The validity check matters. A world is not just an arbitrary mutation. It should remain valid source in the language being studied, because the goal is to distinguish executable interpretations under plausible source states rather than under broken syntax.
So when I use the word world, I mean one concrete source state synthesized as part of the experiment.
Some worlds are historical Git states. Others are controlled source states produced specifically to separate candidate meanings.
In both cases, a world is something executable programs can actually run against.
Once there are several candidate programs and several worlds, I can run every candidate on every world and record what actually happens.
One such execution is a coordinate:
program P7
×
world W12
That coordinate is simply one cell in an experiment: program P7 executed against source state W12.
The cell can record several different facts:
matched?
changed?
exact?
semantic predicate true?
Those questions are separate. A program can match without changing anything useful, or change source without producing the desired result. A semantic analysis can also answer a question about the same source independently of the transformation.
The important shift is that the experiment is not asking a model whether P7 “seems more general.”
Synthesis creates situations in which the distinction can become observable, and execution collects the observation.
Once there are many programs and many worlds, the experiment starts to look like a small database.
This part is easy to make more mysterious than it is, so it helps to be explicit about what the axes mean.
Suppose synthesis has produced two candidate programs:
P0 = only change record.oldField
P1 = change any receiver.oldField
and four worlds:
W0 = original source
W1 = synthesized world with a different receiver
W2 = synthesized world with two possible targets
W3 = synthesized world where the migration already happened
The rows are programs.
The columns are worlds.
A cell is therefore one concrete execution:
row P0
column W1
means:
run program P0 on world W1
Now choose one yes/no question, for example:
Did this program actually change the source?
The matrix for that one question might be:
worlds
W0 W1 W2 W3
┌────────────────────
program P0 │ 1 0 1 0
program P1 │ 1 1 1 0
Read across a row and you see the behavior of one candidate program across several synthesized source states.
Read down a column and you see how several competing programs behave on the same world.
The 1 in row P1, column W1 means:
P1 changed W1.
The 0 in row P0, column W1 means:
P0 was executed on W1 and did not change it.
That column is useful precisely because synthesis constructed W1 to expose a difference that the original example could not.
AttuneFinite stores several relations over the same coordinates. A relation is just one question answered everywhere it was measured.
One relation can represent:
changed?
Another can represent:
exact?
Another can represent:
semantic predicate true?
The program rows and world columns stay the same. Only the question represented by the bits changes.
There is also an important third state:
?
Unknown.
Suppose a program/world coordinate was never executed. It would be wrong to store 0, because 0 means the experiment actually asked the question and observed “no.”
So conceptually a relation may look like:
W0 W1 W2 W3
┌───────────────────
P0 │ 1 0 ? 0
P1 │ 1 1 1 ?
? means no observation exists for that program/world pair.
Not asking a question is not evidence that the answer is no.
Internally, this can be represented with two bitsets: one saying which coordinates are known, and another containing the Boolean value for the coordinates that are known.
Now suppose the question is:
On which measured worlds do P0 and P1 behave differently?
Take the changed? rows:
P0 1 0 1 0
P1 1 1 1 0
and XOR them:
P0 XOR P1
1 0 1 0
1 1 1 0
-------
0 1 0 0
Only W1 differs.
The experiment has converted a qualitative question—
Where do these candidate meanings disagree?
—into a very small computation:
XOR the rows
The number of measured disagreements is:
popcount(P0 XOR P1) = 1
popcount means “count the number of 1 bits,” so the result says there is exactly one world in this example where the two programs exhibit different changed? behavior.
That gives a tiny algebra over experimental evidence:
A AND B
where are both facts true?
A XOR B
where do they disagree?
popcount(A)
how many coordinates are true?
A == B
are the measured Boolean patterns identical?
Earlier versions of Attune used DuckDB. This could absolutely be implemented as SQL.
I stopped doing that here for two less profound reasons. First, I wanted to explore Mojo. Second, AttuneFinite was an attempt to make the research instrument small enough that I could understand almost all of it at once.
SQL provides a general query system. For these experiments I mostly wanted ordered coordinates, known bits, value bits, AND, XOR, equality, and popcount.
This is not a better database. It is a deliberately small experimental representation.
Once a relation is represented as Boolean observations, 64 values fit in one UInt64. A long row can therefore be stored as a short sequence of machine words rather than a table of individual Boolean objects.
That makes some of the implementation pleasantly boring:
@always_inline
def operation[op: Int, width: Int](
a: SIMD[DType.uint64, width],
b: SIMD[DType.uint64, width],
) -> SIMD[DType.uint64, width]:
comptime if op == 0:
return a & b
elif op == 1:
return a ^ b
else:
comptime assert op == 2
return pop_count(a ^ b)
The SIMD here is not the interesting part. It simply lets several UInt64 words be processed together.
What matters is that the compressed representation remains traceable.
It is always possible to walk from:
bit 37 = 1
back to:
program P7
×
world W12
×
question: changed?
That is the balance I wanted: synthesize many hypotheses and counterexamples, test them cheaply, and still be able to explain exactly what every bit means.
The representation became much more useful when I stopped thinking only about synthesized worlds.
A historical repository state can also be treated as a world.
In one of the smaller AttuneFinite studies, the frozen Documenso corpus contains 2,418 states along its first-parent history. “First-parent” means following the mainline parent through merge commits rather than traversing every branch of the Git DAG, giving one ordered sequence of repository states.
If the columns are now those historical states, a row can represent one frozen executable program through time.
The axes become:
rows = executable programs
columns = ordered historical repository worlds
For one program, the row might look like:
repository history →
state C0 C1 C2 C3 C4 C5 C6 C7 C8 C9 C10
│ │ │ │ │ │ │ │ │ │ │
program P 0 0 0 1 1 1 1 1 0 0 1
Here the relation might ask:
Can this program make a concrete change in this repository state?
A 1 means the program was actionable at that point in history. A 0 means it was measured and was not.
This gives AttuneFinite two kinds of worlds in one representation:
synthesized worlds
source states constructed to expose differences now
historical worlds
source states that really occurred in Git history
The synthesized worlds are useful because they can deliberately target ambiguity. The historical worlds are useful because they were not designed around the current hypothesis.
Both become columns in relations over executable programs.
With several program rows, questions about historical behavior become the same kind of relation queries as the synthesized worlds.
When did a capability first become actionable?
Did it remain actionable for several commits?
Did it disappear and later return?
Do two syntactically different programs ever behave differently anywhere in measured history?
That last question gave me one of my favorite early results.
Two exact historical recurrence examples produced seven different Grit programs. Three were interpretations of the Prisma edit. Four came from a different edit in uploadDocument.ts, where a route changed from:
`${NEXT_PUBLIC_WEBAPP_URL}/documents/${createdDocumentIdFromBody}`
to:
`${NEXT_PUBLIC_WEBAPP_URL}/documents/${createdDocumentIdFromBody}/recipients`
The four generated programs ranged from rewriting the route fragment itself all the way out to the surrounding router.push(...) call.
These were genuinely different pieces of Grit source.
Across the measured historical states, though:
3 different Prisma programs
↓
1 observed historical behavior
4 different upload programs
↓
1 observed historical behavior
What this means is that, within each group, the rows in the historical behavior matrix were identical. The repository states that were measured never supplied a column where two programs in the same group behaved differently.
That is not universal semantic equivalence.
There may be some source state that was never measured where the programs would diverge immediately. The historical evidence only supports a narrower statement:
Across the repository states measured so far, no behavioral difference between these programs has been observed.
That is exactly the sort of claim I want the system to make. It is empirical and reversible.
And this is where synthesized worlds become useful again. If history has failed to distinguish two programs, AttuneFinite does not have to conclude that they are equivalent. It can instead synthesize a new world specifically intended to split their behavior.
History tells me what distinctions have naturally appeared.
Synthesis lets me ask for distinctions that history has not produced yet.
The uploadDocument example had another interesting temporal property. Its transformation became actionable 74 commits before the later human edit, and the target file remained byte-identical throughout that interval.
In other words, the historical row switched from 0 to 1; then 74 commits occurred elsewhere in the repository while the relevant local source stayed unchanged and the transformation remained possible. Eventually, a human performed exactly that operation.
This is not the same kind of contextual transfer as the running-total example at the beginning. The local target file did not evolve.
What it provides is a different kind of evidence: an executable maintenance opportunity can become available in repository history before a human eventually takes it.
The historical relation becomes a small map of executable possibility through time.
There is an important negative result here.
The simple product story would be to keep mining edits, accumulate more programs, and eventually build a huge automatic maintenance library.
That is not what happened.
In one frozen experiment, the library grew to 96 unique learned program texts over 14,512 accepted-edit coordinates.
Those 14,512 coordinates are the accepted edits in the frozen population — the worlds across which each program’s MATCH, REWRITE, and EXACT observations were measured. Grouping the 96 program texts by those observation patterns collapsed them into only 8 observed behavioral classes.
Different source code for a rule therefore did not necessarily buy a new measured capability. Many programs behaved identically everywhere the experiment had looked.
More importantly, exact future coverage reached two recurring families early and stayed at two while the program library continued growing.
That killed the naive version of the story for me.
I do not think AttuneFinite demonstrates that simply accumulating more deterministic rules steadily approaches autonomous repository maintenance.
The more interesting result is about applicability and discrimination.
When should a capability apply?
When do two plausible interpretations actually behave differently?
What synthesized world would expose that difference?
What historical world has already exposed it?
What evidence explains why one behavior is desirable and another is not?
Different programs can collapse to the same observed behavior. A capability can become actionable before a human uses it. Synthesis can construct source states that expose distinctions history has not shown yet, and semantic observations can sometimes explain those distinctions.
That is a much better research surface than simply counting how many rules have accumulated.
There is still a larger question underneath all of this.
A programming language permits an absurd number of possible programs. A repository permits an absurd number of possible changes.
Why should a tiny transformation extracted from one old edit tell me anything about another?
Two ideas from software research helped me form a hypothesis.
On the Naturalness of Software showed that real source code occupies a much smaller and more predictable region of the space permitted by a programming language. Human-written software is repetitive in ways that statistical models can exploit: names recur, structures recur, APIs recur, and idioms recur.
On the Localness of Software sharpened that observation. Some of that regularity becomes especially strong when the model is conditioned on nearby or repository-local context.
Repositories develop vocabularies. Packages develop conventions. Subsystems develop habits.
Those papers study distributions of code.
The extrapolation is mine:
Why wouldn't edits be local too?
A repository can theoretically accept an astronomical number of changes. Maybe the changes it actually accepts occupy a much smaller space.
Migrations recur. The same API conversion appears in several places. Reviewers repeatedly enforce the same architectural boundary. Two separate analytics features can eventually acquire the same running-total shape.
The 2023 and 2025 Documenso functions are clearly different programs. One counts newly created users; the other counts distinct active users over a rolling window.
But both eventually reach the same small representation boundary where monthly database rows become chart data, and both acquire the same additional field.
Perhaps repositories develop not only a local language of code, but a local language of change.
Under that framing, predicting the future sounds a little less supernatural.
The claim is not that arbitrary future code can be known in advance. It is that some later edits may be additional samples from a distribution of changes whose structure is already partially visible in the repository's past.
That is also the bridge back to machine learning.
A large pretrained model can know a great deal about programming in general. What it does not automatically know is the local distribution of acceptable change in a particular repository.
Repository history is evidence about that distribution.
Synthesized worlds provide another kind of evidence. Instead of waiting for the repository to naturally produce every interesting distinction, synthesis can construct valid local source states that ask whether a candidate rule has learned too much, too little, or the wrong thing.
History supplies observations the experiment did not design.
Synthesis supplies questions the history did not happen to ask.
Behavioral disagreement is useful, but sometimes I want to understand what the disagreement means.
Consider a synthesized world like:
const record = {
oldField: "seed",
newField: "seed",
};
const distractor = {
oldField: "other",
newField: "other",
};
record.oldField;
distractor.oldField;
A literal rule can change only record.oldField. A captured rule can potentially change an oldField access on another receiver too.
The world was synthesized precisely because that source state gives the programs somewhere to diverge.
Grit can therefore tell me that the candidates behave differently. But it cannot, by itself, tell me whether the difference corresponds to something semantically important.
Maybe the important distinction is not the spelling record versus distractor.
Maybe it is what those receiver expressions actually refer to.
That is a semantic question.
This is where CodeQL became useful.
CodeQL is a query language for asking structural and semantic questions about source code. Rather than rewriting source like Grit, it can resolve relationships such as which declaration a variable reference points to.
One of the queries looks roughly like this:
import javascript
from
PropAccess access,
VarAccess receiver,
Variable binding
where
// Find expressions shaped like:
//
// something.oldField
//
access.getPropertyName() = "oldField" and
access.getBase() = receiver and
// Ask which lexical variable declaration
// the receiver actually resolves to.
receiver.getVariable() = binding
select
receiver.getLocation().getStartLine(),
receiver.getLocation().getStartColumn(),
binding.getName(),
binding.getADeclaration().getLocation().getStartLine()
Grit asks:
What does this executable transformation do?
CodeQL asks:
What semantic relationship exists in this source?
Neither one declares the other correct. They are separate instruments looking at the same synthesized or historical world from different angles.
One small experiment makes that relationship concrete.
Six Grit programs were selected along with twelve controlled TypeScript inputs. Those controlled inputs are synthesized worlds: source states constructed to exercise particular distinctions.
Each program was executed on the worlds where it applied, and the resulting source was then analyzed independently with CodeQL.
That produced 24 integrated program/world cases.
Some worlds contained one relevant target:
import { PrismaClient } from "@documenso/prisma";
Others deliberately introduced multiplicity:
import { PrismaClient } from "@documenso/prisma";
import {
PrismaClient as SecondaryPrismaClient,
} from "@documenso/prisma";
The second world contains two relevant old targets rather than one.
The CodeQL side then asked a deliberately weak question:
// Does at least one old target exist?
//
// This intentionally says nothing about whether
// there is exactly one target.
boolean eligible(string family, string path) {
metamorphicStudyCell(_, _, family, path, _, _) and
if beforeCount(family, path, "old") > 0
then result = true
else result = false
}
The key condition is:
beforeCount(...) > 0
It asks only whether one or more old targets exist before the transformation.
I then compared that semantic predicate with what Grit had actually done:
| Grit output | eligible | ineligible | | --- | ---: | ---: | | desired exact output | 14 | 0 | | undesired output | 4 | 3 |
The filter retained all 14 desirable transformations. It rejected three of seven undesirable ones.
Four bad transformations still survived.
The reason was useful.
A world with:
beforeOld = 1
and one with:
beforeOld = 2
both satisfy:
beforeOld > 0
The query understood presence.
It did not understand multiplicity.
That is exactly where several overly broad transformations survived.
I like this result because the verifier fails in a useful way. Instead of an opaque number such as:
confidence = 0.73
the failure has a structure:
question:
does a target exist?
failure:
there are two targets
missing information:
multiplicity
next question:
how many?
The failed verifier points toward the next verifier.
That feels much closer to learning than a scalar score.
The important thing is not that CodeQL magically supplied the correct rule. It did not.
Synthesis created worlds where ambiguity could become visible. Grit exposed behavioral differences on those worlds. Then a weak semantic question failed in a structured way, and the pattern of its failures exposed information the next question needed to represent.
This is where the teaching analogy mostly stops feeling like an analogy.
Suppose two interpretations remain possible.
Keep both.
Use synthesis to construct a world where they should disagree.
Run both programs on that world.
If their behavior separates but the reason is unclear, ask a semantic question.
If that question rejects three failures but misses four, inspect those four and ask a better question.
If synthesized worlds stop revealing anything new, let Git history provide source states that were never constructed around the hypothesis.
The process looks roughly like:
accepted edit
↓
synthesize executable interpretations
↓
deterministic filtering
↓
several interpretations survive
↓
synthesize worlds that distinguish them
↓
behavioral observations
↓
semantic questions
↓
freeze
↓
future historical worlds
There is nothing individually intelligent about most of those operations.
That is part of what interests me.
The model can remain responsible for the open-ended part: generating code, proposing explanations, or suggesting hypotheses.
The surrounding system has a different job. It can remember what happened, preserve unresolved alternatives, synthesize source states that expose disagreement, collect mechanical evidence, and avoid paying for the same human judgment twice.
Reinforcement learning depends on some way of deciding which behavior should be reinforced. In coding, that usually means turning parts of software development into scores, rewards, verifiers, or other training signals.
The verification bottleneck is not unique to coding agents, but software is an unusually nice place to attack it.
There are compilers, tests, type systems, static analyses, syntax trees, build graphs, and years of repository history. Unlike many domains, it is often possible to intervene directly on the object, execute the intervention, and inspect what happened.
It is also possible to synthesize new source states cheaply.
That matters because supervision does not have to consist only of labels on naturally occurring examples. If several hypotheses all explain one human correction, the system can generate new valid programs specifically designed to make those hypotheses disagree.
Those synthesized worlds become experiments.
That gives a route from fuzzy human judgment toward increasingly mechanical supervision.
Imagine a coding agent proposes a patch and a human rejects one part of it. Instead of treating that correction as disposable conversational context, the surrounding system can ask:
What executable hypotheses explain this correction?
Where do those hypotheses disagree?
What world could expose that disagreement?
Which distinctions does this repository care about?
Can existing semantic tools explain the distinction?
Does this lesson survive later history?
That process can manufacture supervision the repository never explicitly stored.
A rename might provide evidence that two representations correspond to the same action. A canonical helper can reveal that superficially similar syntax should be treated differently in different contexts. A synthesized world can expose that a candidate rule generalized across the wrong receiver. A later accepted edit can show that an old interpretation survived contact with future repository history.
These are potential training signals.
They are not perfect reward, and they are not universal truth. They are pieces of evidence produced from human attention that was already spent, combined with experiments that software makes unusually cheap to construct.
That changes the economics.
A human review that only fixes one patch has value once. A human review that also seeds executable hypotheses, synthesized counterexamples, semantic predicates, or training relations can continue paying dividends.
Supervision should accumulate.
I do not want to make coding agents less generative. That would throw away one of their most interesting properties.
I want a model to retain the ability to do something strange and occasionally better than what I would have written. What I want to improve is the environment around that freedom.
A powerful base model arrives with general programming ability. Then the repository can begin teaching it the local part.
Not through one enormous prompt containing every convention anyone has ever written down, but through accumulated evidence about what the codebase has accepted, rejected, repeated, and distinguished.
Some of that evidence comes from history.
Some of it can be synthesized.
History says:
Here is what developers actually accepted later.
World synthesis asks:
What would happen if this local assumption changed?
Together they provide a way to learn from both naturally occurring development and deliberately constructed counterexamples.
Different repositories should produce different evidence. Different teams should eventually be able to produce different agents even when they begin with the same base model.
One becomes unusually good at a particular database layer. Another internalizes a team's API boundaries. Another learns which abstractions a repository consistently rejects. Another recognizes the migration that has been unfolding for six months.
That is a much more interesting future to me than one universal coding personality dropped unchanged into every repository.
AttuneFinite does not build that future. It gives me a small place to test whether some of its prerequisites exist.
Can an accepted edit become an executable lesson?
Can several possible lessons be represented simultaneously?
Can synthesis construct worlds that distinguish them?
Can semantic tools explain some of those distinctions?
Can later repository history test what survived?
In a few cases, the answer has been yes.
Sometimes an old executable lesson even produces exactly what a developer later changes.
That sounds a little like predicting the future. In the broad sense, it obviously is not. AttuneFinite cannot tell me what feature Documenso will build next, which architecture a team will choose, or what arbitrary code a developer will write.
The interesting claim is much smaller.
Repository history appears to contain recurring, executable fragments of human judgment. Synthesis can turn one accepted edit into multiple hypotheses and then generate source worlds that expose where those hypotheses disagree. Later history can provide examples that none of those hypotheses were allowed to see.
Underneath that result, the machinery is wonderfully ordinary: Git history, TypeScript, small executable rewrites, syntax trees, synthesized source states, Boolean relations, bit operations, semantic queries, and future commits.
When I was teaching, I did not want to give a smart kid every answer. I wanted them to retain the freedom to do weird things.
What I wanted them to learn was how to remain confused without becoming stuck: how to research, construct an experiment, notice when an explanation failed, and ask a better question.
Eventually, I wanted them to stop needing me for the same correction.
That is roughly what I want from agent coworkers too.
The base model makes the agent capable.
The repository teaches it how things are done here.