No regression does not mean the agent is safe
A pull request can show no regression while the agent configuration is still full of findings. Here is what a baseline, a score change, and an exact base comparison actually tell you.
The check can be green for the wrong reason
A pull request adds one MCP server. The Foo Guard check comes back with no regression. The reviewer treats that as a pass and merges.
The baseline it compared against already allowed a shell tool, a broad HTTP client, and a credential in the config. The new commit did not make that worse. It also did not make it acceptable.
“No regression” answers a narrow question: did this change move the scored findings in the wrong direction relative to the comparison you configured? It does not answer whether the agent should run.
A green pull-request result beside findings that were already in the baseline
Pull request check
No regression
Compared with the approved baseline.
Still in the baseline
CriticalAG-013 Shell or Command Execution
HighAG-014 Arbitrary HTTP Requests
CriticalAG-004 Embedded Credential
A baseline is a snapshot you chose
A repository baseline stores an approved completed scan. Later scans compare score movement and finding rule IDs with that snapshot so a regression is visible.
Approving the snapshot records what you are willing to compare against. It does not erase the findings inside it. A low score can stay a low score, commit after commit, and every one of those commits can still report no regression.
Read the score and the findings on the scan itself. A security score starts at 100 and loses fixed points for each finding’s severity. A higher score means fewer scored findings. A score of 100 means no scored findings on that file. It is not proof the agent is safe when it runs.
Ask what this pull request changed
On a pull request, Foo Guard can compare the head commit with the exact base revision GitHub reported for that pull request. Both revisions are scanned by the same deterministic rules. The check and the scan detail show the base and head commits, the grades and scores, and counts of new, existing, and resolved findings.
Existing findings were already in the base. New findings showed up in the head. Resolved means a finding from the base was absent in the head scan. Absence in that comparison is not proof someone fixed the underlying risk, and a renamed file can look like a resolution plus a new finding.
The comparison is information for the reviewer. Repository policy, the required check, and the stored baseline are separate decisions. When the base cannot be read, the scan scope changed, or the revisions moved during analysis, Foo Guard leaves the comparison unavailable. It does not fill those gaps with a claim of zero new findings.
Use the check for the decision you actually configured
Require the Foo Guard Security check when a high finding, or whatever threshold you set in repository policy, should block the merge. Use the baseline when you want to notice a regression from a snapshot you approved. Use the pull-request comparison when you want to see what this change introduced against that exact base.
Then still read the findings. A quiet diff on a dangerous config is a quiet diff. The agent’s permissions are whatever the file still declares.