ansible/ansible/f327e65d11bb905ed9f15996024f857a95592629 · vba6da65a0f3baefda7a058ebbd0a8dcafb8512f5

Fix for ansible/ansible@f327e65d

Verdict

11.3%
Lineman saved $0.0352
Baseline cost$0.3118
Lineman cost$0.2767
Baseline turns17
Lineman turns14
Baseline duration1m 18s
Lineman duration52.0s
Lineman version0.1.0

Quality - SWE-bench Docker Harness

Baseline
Not yet evaluated
Lineman
Not yet evaluated

Analysis

(narrative skipped)

Baseline

Turns17
Tool calls16
Cache read388,753
Cache write33,648
Output tokens4,598
Total cost$0.3118

Lineman

Turns14-18%
Tool calls13-19%
Cache read357,952-8%
Cache write34,014+1%
Output tokens2,779-40%
Total cost$0.2767

Tool Call Timeline

#Baseline toolCharsLineman toolCharsΔ
1Read/private/var/folders/04/qmf7f1qx7nj2_2mczkd4msth0000gn/T/bench-baseline-instance_ansible__ansible-f327e65d11bb905ed9f15996024f857a95592629-vba6da65a0f3baefda7a058ebbd0a8dcafb8512f5/ISSUE.md662linemanread_file ISSUE.md955+44%
2Grepkeyword.*collection|collection.*keyword|iskeyword|python.*reserved645Grepkeyword|python.*keyword|iskeyword9.7k+1410%
3GrepFQCN|fqcn|fully.qualified.collection1.3kGrepvalidate.*collection|collection.*name.*valid|FQCN654-49%
4Grepiskeyword|keyword|reserved196linemanread_file_context lib/ansible/galaxy/dependency_resolution/dataclasses.py — iskeyword _is_fqcn collection name validation48-76%
5Grepvalidate|keyword|reserved|python16linemanread_file_full lib/ansible/galaxy/dependency_resolution/dataclasses.py5.5k+34050%
16Edit/private/var/folders/04/qmf7f1qx7nj2_2mczkd4msth0000gn/T/bench-baseline-instance_ansible__ansible-f327e65d11bb905ed9f15996024f857a95592629-vba6da65a0f3baefda7a058ebbd0a8dcafb8512f5/lib/ansible/utils/collection_loader/_collection_finder.py344
Total11.1k23.2k+109%