How Cribelo works

Cribelo reads the diff of a branch and gives back three things: the lines that still need a human, the facts the diff does not show, and the order to read the files in. No model runs behind it, so the same change gets the same answer on every machine.

From the diff to the plan

The change is the diff between the merge base of your branch and your working tree. Generated files, lock files and binaries are set aside. The rest is parsed before and after the change with the tree-sitter parsers of AST Metrics, then goes through three passes.

Hunks

The first pass decides what is left to read. Every hunk of the diff gets one nature. The first rule that applies wins, and the nature sets how many lines you read.

A move is matched function by function: same tokens, same tokens once identifiers are normalized, or 80 % of common fingerprints, the method of RefDiff. Then it is carried line by line. A line of the moved hunk that was not in the original function, or that names something the original never named, is new, and you read it. A file whose every hunk is mechanical never enters the reading list. It is listed under Skip with its reason: moved code only, renames only, formatting only.

Read closely, or glance at

The code lines left are then split by what holds them. A line in a function that gained no decision and stays simple (cyclomatic complexity at most 3 after the change) is glanced at. So is a line outside every function, a declaration, a field, a constant, unless it branches itself. A line in a function that gained a branch, or in a function that was already complex, is read closely. The check of a rename and the look at where a block went are glances too. Reading time counts twenty lines read closely a minute, sixty glanced at.

Facts

The second pass looks for what the diff does not show. A fact is something a reader can check in a minute. Cribelo computes the import graph of the repository before and after the change and compares them, asks git for the history of each touched function, and reads your coverage report. None is a prediction. At most one fact of each kind per file.

Order

The third pass puts the files in reading order. The files to read are grouped. Two files are in one group when one imports the other, or, in one directory, when one calls a function the other declares. Inside a group, a file comes before the files that use it: you read the definition before its uses, the rule of Baum and colleagues (2017). Each test comes right after the file it exercises, and a test that sits next to changed code is matched in its own directory only. Groups carrying a cycle, a signature change, a fix record, a first dependency or a clone come first, then the best score, then size.

Where the score comes in

Four things shape the plan. Each one settles only what the previous ones left open, and the score comes last.

1. NaturesWhat is read at all, and how many lines of each hunk. A file whose hunks are all mechanical never enters the list, whatever its score.
2. FactsWhich group comes first: a cycle, a signature change, a fix record, a first dependency or a clone pulls its group to the top.
3. DependenciesThe order inside a group: the definition before its uses, each test right after the file it exercises.
4. ScoreWhat is left: which files fit a budget, the tie between two groups, and the high, medium and low levels shown in the table. Ten public signals, weights you can recalibrate on your own history.

Under a budget

cribelo --budget 30m fills the time with the best-ranked files first, at twenty lines read closely a minute and sixty glanced at, and adds a test only while it fits. What does not fit is listed under Skip as ranked lower, with everything it knows about it.

The plan

cribelo prints the plan in the terminal. cribelo describe prints the same in Markdown, ready for the pull request description. The GitHub Action posts one comment per pull request and updates it on each push. Here is the plan of pull request #196 of AST Metrics, trimmed to its first files.

$ cribelo
cribelo · 34 files changed vs origin/main (merge-base b2f83dc)

Read 11 of 34 changed files, about 43 min.

                                                           lines to read closely               lines to glance at
   1  internal/analyzer/library_reach.go             new   30-35, 66-265                       16-26, 39-57
   2  internal/analyzer/library_conformance_test.go  test  98-140                              25-93
   3  internal/analyzer/library_reach_test.go        test  10-63, 85-144                       67-80
   4  internal/engine/util.go                              85-112
   5  internal/analyzer/file_dependency_analyzer.go        97-101, 122-127, 141, 152, +4 more  22-27
   6  internal/engine/php/deps/resolver.go           new   25-43, 57-69                        15-22, 46-51, 75-76
   7  internal/engine/php/deps/resolver_test.go      test  11-54
   8  internal/command/who_uses_command.go           new   52-241                              25-48
      … 3 more

Why these files
                                   complexity  used by    commits 90d      clone  test changed
   1  library_reach.go             ▮▮▮▯▯ 17    ▮▮▮▮▮ 83   ▮▮▯▯▯ 3                 ✓
   4  util.go                      ▮▮▯▯▯ 10    ▮▮▮▮▮ 111  ▮▯▯▯▯ 2, 1 fix          ✗
   5  file_dependency_analyzer.go  ▮▮▮▮▮ 44    ▮▮▮▮▮ 83   ▮▮▯▯▯ 3                 ✓
   6  resolver.go                  ▮▮▯▯▯ 9     ▮▯▯▯▯ 1    ▮▯▯▯▯ 1          84%    ✓
   8  who_uses_command.go          ▮▮▮▯▯ 18    ▮▮▯▯▯ 3    ▮▮▯▯▯ 3, 1 fix          ✓
      … 2 more

Totals  672 lines to read closely, 272 to glance at (20 and 60 lines a minute)
Set aside in the whole change: 11 import hunks.

Worth knowing  true by construction, not predicted
  internal/analyzer/aggregator.go  internal/analyzer depends on internal/engine/php/deps for the first time
  internal/engine/typescript/deps/resolver.go  internal/engine/typescript/deps depends on internal/engine/typescript/module for the first time
  internal/analyzer/library_reach.go  15 of 161 changed lines not run by any test, according to pr196-coverage.out
  internal/command/who_uses_command.go  49 of 162 changed lines not run by any test, according to pr196-coverage.out
  internal/engine/util.go  25 of 25 changed lines not run by any test, according to pr196-coverage.out

Skip (23)
  11 ranked lower · 8 tests only · 4 no measurable change    cribelo skip

The first table says where to read: for each file, the line ranges to read closely and the ones a glance is enough for. In the pull request comment, each range opens the diff on those lines. Under it, each column is a measure you can check: cyclomatic complexity of the touched functions, files that import this one, commits and fix commits in the last ninety days, similarity with another function, a test changed with the file.

A gate on what is measured

Nothing here fails a build on its own. In the Action, fail-over: 1h fails the job when reading the whole change would take more than an hour, every file counted, budget or not: the change is too large for one reviewer. fail-on: cycle, signature fails it when the change introduces an import cycle or changes an exported signature that other files call; any kind of fact can be named. The job says why in one line, such as "reading time 1 h 17, over the 1 h allowed". The score never closes a gate: a level is an estimate, a gate needs a measure.

How the relevance of Cribelo was measured

A reading map cannot be scored by a benchmark, so Cribelo was run on history. Twenty-eight open-source repositories in seven languages (Go, Rust, Python, TypeScript, PHP, Java, C#), their last 2,500 merges each: every merge is replayed against its parent with cribelo backtest, as if Cribelo had been there, and the plan it would have printed is kept. Four questions follow, each with the measure that answers it.

How much of a change is mechanical?

For every replayed change, the share of changed code lines still to read once moves, renames, formatting and imports are set aside, and the share of files where nothing is left to read.

On an ordinary month of pull requests, 8 % of the changed lines and 14 % of the files need no reading. The share climbs on the changes that drown a reviewer. In ruff, the pull request that splits types/class.rs changes 10,595 lines: 8,424 are moved, 3,678 are left to read, and git sees no rename. In phpstan, 2,636 lines land in LegacyTypeSpecifier.php and 124 are left to read.

Does the glance rule point at the right lines?

A reading rule can be checked without a reader, with the future: a function whose new lines were touched by a later bug fix deserved a close read. The same repositories, their last 1,500 merges, replayed function by function: 130,998 touched functions, 2,549 of them repaired later. The table reads: among the functions that gained no decision and ended at complexity 2 or below, 1.2 % were repaired later; among those that gained three decisions or more and ended above 10, 7.3 %.

The rule Cribelo ships, no decision gained and complexity 3 or below after the change, glances at 52 % of the lines inside touched functions. The functions it glances at were repaired 1.3 % of the time, the ones it reads closely 2.8 %. A third of the functions repaired later sit in the glanced half, which is why a glance is a fast read and never a skip. The reach of a file did not help: functions in files no other file imports were repaired 2.5 % of the time, functions in files imported by twenty or more 1.4 %. A test touched in the same change made no difference either, 2.1 % against 1.9 %. With the lines outside any function, the glanced share of a change runs from a third in caddy and the GitHub CLI to nine tenths in spring-boot and bitwarden.

Do the facts land where reviewers looked?

Four sets of 25 pull requests with human review comments, from prometheus, pydantic, cargo and vite, replayed at the commit of the first comment. 17 % of the changed files received a comment. The table counts, for each kind of fact, how many of the files it was stated on received one: a fact that lands on commented files far more often than 17 % of the time points at what reviewers care about.

Clone, fix history and unused API are far above the 17 % base rate, signature above it. Cycle, dependency and library are facts about the structure; reviewers do not comment on them file by file, and they are not there to predict that.

Does the order put the commented files first?

On the same hundred pull requests, share of the commented files among the first three files of the list: Cribelo 45 %, alphabetical order 33 %, largest files first 51 %. The order of Cribelo puts the definition before its uses so that the reader understands what follows; this proxy does not measure understanding.

Not measured

• The reading time, twenty lines a minute, is a convention taken from RAID (Brito and Valente, 2021), not a measurement.
• Hunk natures were checked by hand on samples in every language, then on twelve recent pull requests of ruff, hugo, efcore, Avalonia, jellyfin, spring-boot, keycloak, symfony, django, tokio and pydantic. Checked by one person, not measured against a labelled set.
• Coverage facts were tested on fixtures and on one real run (AST Metrics pull request #196 with its Go profile): no repository of the bench ships a coverage report at the replayed commit.

Limits

Cribelo does not find bugs and never says a file is safe. It says what is left to read, what it knows for sure about the structure, and in which order to read. It does not know your layering rules either: a dependency from the domain to the infrastructure is a first dependency like any other. The score that settles the budget and the tie between two groups comes from ten signals (lines, complexity, coupling, fixes, churn, tests, clones); its weights are public, and cribelo calibrate learns them from your own history. Every rule, threshold and weight is in docs/how-it-is-computed.md.