feat(ci): add contributor reputation check workflow by imran-siddique · Pull Request #1520 · github/awesome-copilot

imran-siddique · 2026-04-27T05:09:12Z

Summary

Add automated contributor reputation screening on PR/issue open events to detect coordinated inauthentic contribution patterns (e.g., credential-laundering campaigns, spray-and-pray governance issues).

How it works

Checks out AGT's contributor reputation scripts via sparse-checkout
Runs profile check (account age, repo patterns, spray detection)
Runs credential audit (checks for merged-PR-to-citation pipelines)
Posts a comment and adds a label on MEDIUM/HIGH risk contributions
Skips bot accounts (dependabot, github-actions, copilot-swe-agent)

Why this matters

Multiple AI agent framework repos have been targeted by coordinated campaigns that:

Spray low-effort governance issues across 30+ repos in days
Get trivial PRs merged, then cite them as credentials in other repos
Use foundation submissions to build credibility for product placement

This workflow helps maintainers catch these patterns early.

Dependencies

Uses scripts from microsoft/agent-governance-toolkit (stdlib-only Python, no pip install needed)
Runs on pull_request_target and issues events only
No secrets beyond the default GITHUB_TOKEN

Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com

Add automated screening for coordinated inauthentic contributions on PR/issue open events using AGT contributor reputation tools. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

github-actions

⚠️ This PR targets main, but PRs should target staged.

The main branch is auto-published from staged and should not receive direct PRs.
Please close this PR and re-open it against the staged branch.

You can change the base branch using the Edit button at the top of this PR,
or run: gh pr edit 1520 --base staged

Copilot

Pull request overview

Adds a new GitHub Actions workflow to automatically screen new PR/issue authors using the Agent Governance Toolkit (AGT) and to surface elevated-risk contributors via comments and labels.

Changes:

Introduces .github/workflows/contributor-check.yml triggered on pull_request_target and issues opened events.
Runs two AGT Python checks (profile + credential audit) and computes an overall risk level.
Posts a PR/issue comment and applies a needs-review:<RISK> label for MEDIUM/HIGH outcomes.

Copilot · 2026-04-27T05:12:32Z

+      - name: Checkout AGT scripts
+        uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
+        with:
+          repository: microsoft/agent-governance-toolkit


This workflow checks out and executes Python from an external repo (microsoft/agent-governance-toolkit) in a pull_request_target context with write permissions. To reduce supply-chain risk, pin the checkout to a specific immutable ref (commit SHA) and consider setting persist-credentials: false so the token isn’t written into the checked-out repo’s git config.

Suggested change

repository: microsoft/agent-governance-toolkit

repository: microsoft/agent-governance-toolkit

ref: 0123456789abcdef0123456789abcdef01234567 # replace with a trusted AGT commit SHA

persist-credentials: false

Copilot · 2026-04-27T05:12:33Z

+            case "$1" in HIGH) echo 3 ;; MEDIUM) echo 2 ;; LOW) echo 1 ;; *) echo 0 ;; esac
+          }
+          p=$(risk_to_num "${{ steps.profile.outputs.risk }}")
+          c=$(risk_to_num "${{ steps.credential.outputs.risk }}")
+          max=$p; [ "$c" -gt "$max" ] && max=$c
+          case "$max" in 3) r="HIGH" ;; 2) r="MEDIUM" ;; *) r="LOW" ;; esac


The profile / credential steps convert any script failure or malformed JSON into risk=UNKNOWN, but risk_to_num maps unknown values to 0, which then makes the overall risk LOW. This means the workflow silently fails open (no label/comment) when the checks break; consider treating UNKNOWN as at least MEDIUM (or failing the job) so the workflow doesn’t miss suspicious contributions due to transient errors.

Suggested change

case "$1" in HIGH) echo 3 ;; MEDIUM) echo 2 ;; LOW) echo 1 ;; *) echo 0 ;; esac

}

p=$(risk_to_num "${{ steps.profile.outputs.risk }}")

c=$(risk_to_num "${{ steps.credential.outputs.risk }}")

max=$p; [ "$c" -gt "$max" ] && max=$c

case "$max" in 3) r="HIGH" ;; 2) r="MEDIUM" ;; *) r="LOW" ;; esac

case "$1" in

HIGH) echo 3 ;;

MEDIUM|UNKNOWN) echo 2 ;;

LOW) echo 1 ;;

*) echo 2 ;;

esac

}

p=$(risk_to_num "${{ steps.profile.outputs.risk }}")

c=$(risk_to_num "${{ steps.credential.outputs.risk }}")

max=$p; [ "$c" -gt "$max" ] && max=$c

case "$max" in 3) r="HIGH" ;; 2) r="MEDIUM" ;; 1) r="LOW" ;; *) r="MEDIUM" ;; esac

Copilot · 2026-04-27T05:12:33Z

+          body="<!-- agt-contributor-check -->
+          $icon **Contributor Reputation Check: $risk risk**
+
+          | Check | Risk |
+          |-------|------|
+          | Profile | $profile |
+          | Credential audit | $cred |
+
+          Maintainers: please review this contributor before merging.
+          *Automated check powered by [AGT](https://github.com/microsoft/agent-governance-toolkit).*"


The multi-line body string is indented inside the bash script, so the generated Markdown comment will include leading spaces on most lines. In GitHub-flavored Markdown, 4+ leading spaces turns content into a code block, which will likely prevent the table/bold formatting from rendering. Consider building the body with a heredoc (no indentation) or otherwise stripping leading whitespace.

Suggested change

body="

$icon **Contributor Reputation Check: $risk risk**

| Check | Risk |

|-------|------|

| Profile | $profile |

| Credential audit | $cred |

Maintainers: please review this contributor before merging.

*Automated check powered by [AGT](https://github.com/microsoft/agent-governance-toolkit).*"

body=$(cat <<EOF



$icon **Contributor Reputation Check: $risk risk**

| Check | Risk |

|-------|------|

| Profile | $profile |

| Credential audit | $cred |

Maintainers: please review this contributor before merging.

*Automated check powered by [AGT](https://github.com/microsoft/agent-governance-toolkit).*

EOF

)

Copilot · 2026-04-27T05:12:33Z

+        env:
+          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        run: |
+          username="${{ steps.author.outputs.username }}"


username is assigned but never used in this step; removing it would reduce noise and avoid implying it influences the comment/label behavior.

Suggested change

username="${{ steps.author.outputs.username }}"

aaronpowell · 2026-04-28T00:25:51Z

Please don't contribute new workflows and policies without discussing with the maintainers first.

aaronpowell · 2026-05-01T01:04:30Z

+  pull_request_target:
+    types: [opened]
+  issues:
+    types: [opened]


Have you experimented with running this on discussions too? Would that be useful to do?

aaronpowell · 2026-05-01T01:06:40Z

+      - name: Checkout AGT scripts
+        uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
+        with:
+          repository: microsoft/agent-governance-toolkit
+          sparse-checkout: scripts
+          path: agt


Do we have to clone the repo? Is it not possible to install the CLI with pip?

If we have to clone the repo, it might be a good idea to pin a tag or release, so that if there is drift in the script it doesn't result in this action failing.

aaronpowell · 2026-05-01T01:08:07Z

+          profile="${{ steps.profile.outputs.risk }}"
+          cred="${{ steps.credential.outputs.risk }}"
+
+          if [ "$risk" = "HIGH" ]; then icon="🔴"; else icon="🟡"; fi


Should low risk not have a separate icon?

aaronpowell · 2026-05-01T01:09:30Z

+          body="<!-- agt-contributor-check -->
+          $icon **Contributor Reputation Check: $risk risk**
+
+          | Check | Risk |


Ah, my comment on the discussion is probably irrelevant - I wrote that before reviewing the PR and I was assuming the PR was going to be using the action and Python script the action runs from the repo, not a custom script here.

aaronpowell · 2026-05-01T01:10:57Z

+            --repo "${{ github.repository }}" \
+            --json > /tmp/cred.json 2>/tmp/cred.log
+          set -e
+          risk=$(python -c "import json; print(json.load(open('/tmp/cred.json'))['risk'])" 2>/dev/null || echo "UNKNOWN")


It only extracts the risk property from the JSON, is there anything else from the JSON that might be useful to extract and display?

aaronpowell · 2026-05-01T01:15:32Z

+            --repo "${{ github.repository }}" \
+            --json > /tmp/cred.json 2>/tmp/cred.log
+          set -e
+          risk=$(python -c "import json; print(json.load(open('/tmp/cred.json'))['risk'])" 2>/dev/null || echo "UNKNOWN")


Suggested change

risk=$(python -c "import json; print(json.load(open('/tmp/cred.json'))['risk'])" 2>/dev/null || echo "UNKNOWN")

risk=$(cat '/tmp/cred.json' | jq '.risk // "UNKNOWN"')

Can simplify with jq rather than loading a Python environment for that.

aaronpowell · 2026-05-01T01:16:14Z

+            --username "${{ steps.author.outputs.username }}" \
+            --json > /tmp/profile.json 2>/tmp/profile.log
+          set -e
+          risk=$(python -c "import json; print(json.load(open('/tmp/profile.json'))['risk'])" 2>/dev/null || echo "UNKNOWN")


Suggested change

risk=$(python -c "import json; print(json.load(open('/tmp/profile.json'))['risk'])" 2>/dev/null || echo "UNKNOWN")

risk=$(cat '/tmp/profile.json' | jq '.risk // "UNKNOWN"')

Can simplify with jq rather than loading a Python environment for that.

aaronpowell · 2026-05-01T01:16:35Z

+            --username "${{ steps.author.outputs.username }}" \
+            --json > /tmp/profile.json 2>/tmp/profile.log
+          set -e
+          risk=$(python -c "import json; print(json.load(open('/tmp/profile.json'))['risk'])" 2>/dev/null || echo "UNKNOWN")


It only extracts the risk property from the JSON, is there anything else from the JSON that might be useful to extract and display?

github-actions Bot and others added 2 commits April 20, 2026 05:09

chore: publish from staged

63d08d5

feat(ci): add contributor reputation check workflow

acd23a9

Add automated screening for coordinated inauthentic contributions on PR/issue open events using AGT contributor reputation tools. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

imran-siddique requested a review from aaronpowell as a code owner April 27, 2026 05:09

Copilot AI review requested due to automatic review settings April 27, 2026 05:09

github-actions Bot requested changes Apr 27, 2026

View reviewed changes

Copilot started reviewing on behalf of imran-siddique April 27, 2026 05:09 View session

Copilot AI reviewed Apr 27, 2026

View reviewed changes

aaronpowell closed this Apr 28, 2026

aaronpowell reopened this May 1, 2026

aaronpowell requested a review from dvelton as a code owner May 1, 2026 00:58

aaronpowell reviewed May 1, 2026

View reviewed changes

	risk=$(python -c "import json; print(json.load(open('/tmp/cred.json'))['risk'])" 2>/dev/null \|\| echo "UNKNOWN")
	risk=$(cat '/tmp/cred.json' \| jq '.risk // "UNKNOWN"')

	risk=$(python -c "import json; print(json.load(open('/tmp/profile.json'))['risk'])" 2>/dev/null \|\| echo "UNKNOWN")
	risk=$(cat '/tmp/profile.json' \| jq '.risk // "UNKNOWN"')

Conversation

imran-siddique commented Apr 27, 2026

Summary

How it works

Why this matters

Dependencies

Uh oh!

github-actions Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Apr 27, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 27, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 27, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 27, 2026

Choose a reason for hiding this comment

Uh oh!

aaronpowell commented Apr 28, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants