Reasoning over Code Quality and Security in GitHub Pull Requests

Introduction

本指南说明如何将 OpenAI 推理模型集成到您的 GitHub Pull Request (PR) 工作流中，以自动审查代码的质量、安全性和企业标准合规性。通过在开发过程早期利用 AI 驱动的见解，您可以更早地发现问题，减少手动工作量，并在整个代码库中保持一致的最佳实践。

Why Integrate OpenAI Reasoning Models in PRs?

• 通过自动检测代码异味、安全漏洞和样式不一致来节省代码审查时间。 • 在整个组织中强制执行编码标准，以实现一致、可靠的代码。 • 为开发人员提供关于潜在改进的及时、由 AI 驱动的反馈。

Example Use Cases

• 审查者希望在合并前获得有关新代码更改安全性的反馈。 • 团队寻求强制执行标准编码指南，确保整个组织的代码质量一致。

Prerequisites

1. Generate an OpenAI “Project Key”

  1. 转到 platform.openai.com/api-keys 并单击以创建新的密钥。
  2. 将令牌安全地存储在 GitHub 存储库的密钥中，命名为 OPENAI_API_KEY。

2. Choose Your OpenAI Model

使用 OpenAI Reasoning Models 对代码更改进行深入分析。从最先进的模型开始，并根据需要优化您的提示。

3. Select a Pull Request

  1. 确认您的存储库已启用 GitHub Actions。
  2. 确保您有权配置存储库密钥或变量（例如，用于您的 PROMPT、MODELNAME 和 BEST_PRACTICES 变量）。

4. Define Enterprise Coding Standards

将您的标准存储为存储库变量 (BEST_PRACTICES)。这些可能包括： • 代码样式和格式 • 可读性和可维护性 • 安全性和合规性 • 错误处理和日志记录 • 性能和可伸缩性 • 测试和质量保证 • 文档和版本控制 • 可访问性和国际化

5. Define Prompt Content

构建一个元提示来指导 OpenAI 进行安全、质量和最佳实践检查。包括：

  1. 代码质量和标准
  2. 安全性和漏洞分析
  3. 容错和错误处理
  4. 性能和资源管理
  5. 分步验证

鼓励 OpenAI 提供彻底的逐行审查，并附带明确的建议。

Create Your GitHub Actions Workflow

此 GitHub Actions 工作流在每次针对主分支的拉取请求时触发，并包含两个作业。第一个作业收集所有已更改文件的差异（排除 .json 和 .png 文件），并将这些更改发送给 OpenAI 进行分析。OpenAI 提出的任何修复建议都将包含在 PR 的评论中。第二个作业根据您定义的企业标准评估 PR，并返回一个 Markdown 表格，总结代码对这些标准的遵守情况。您可以通过更新提示、模型名称和最佳实践等变量来轻松调整或优化工作流。

name: PR Quality and Security Check

on:
  pull_request:
    branches: [main]

permissions:
  contents: read
  pull-requests: write

jobs:
  quality-security-analysis:
    runs-on: ubuntu-latest
    steps:

      - name: Check out code
        uses: actions/checkout@v3
        with:
          fetch-depth: 0  # Ensure full history for proper diff

      - name: Gather Full Code From Changed Files
        run: |
          CHANGED_FILES=$(git diff --name-only origin/main...HEAD)
          echo '{"original files": [' > original_files_temp.json
          for file in $CHANGED_FILES; do
            if [[ $file == \*.json ]] || [[ $file == *.png ]]; then
              continue
            fi
            if [ -f "$file" ]; then
              CONTENT=$(jq -Rs . < "$file")
              echo "{\"filename\": \"$file\", \"content\": $CONTENT}," >> original_files_temp.json
            fi
          done
          sed -i '$ s/,$//' original_files_temp.json
          echo "]}" >> original_files_temp.json

      - name: Display Processed Diff (Debug)
        run: cat original_files_temp.json

      - name: Get Diff
        run: |
          git diff origin/main...HEAD \
            | grep '^[+-]' \
            | grep -Ev '^(---|\+\+\+)' > code_changes_only.txt
          jq -Rs '{diff: .}' code_changes_only.txt > diff.json
          if [ -f original_files_temp.json ]; then
            jq -s '.[0] * .[1]' diff.json original_files_temp.json > combined.json
            mv combined.json diff.json

      - name: Display Processed Diff (Debug)
        run: cat diff.json

      - name: Analyze with OpenAI
        env:
          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
        run: |
          DIFF_CONTENT=$(jq -r '.diff' diff.json)
          ORIGINAL_FILES=$(jq -r '."original files"' diff.json)
          PROMPT="Please review the following code changes for any obvious quality or security issues. Provide a brief report in markdown format:\n\nDIFF:\n${DIFF\_CONTENT}\n\nORIGINAL FILES:\n${ORIGINAL_FILES}"
          jq -n --arg prompt "$PROMPT" '{
            "model": "gpt-4",
            "messages": [
              { "role": "system", "content": "You are a code reviewer." },
              { "role": "user", "content": $prompt }
            ]
          }' > request.json
          curl -sS https://api.openai.com/v1/chat/completions \
            -H "Content-Type: application/json" \
            -H "Authorization: Bearer ${OPENAI_API_KEY}" \
            -d @request.json > response.json

      - name: Extract Review Message
        id: extract_message
        run: |
          ASSISTANT_MSG=$(jq -r '.choices[0].message.content' response.json)
          {
            echo "message<<EOF"
            echo "$ASSISTANT_MSG"
            echo "EOF"
          } >> $GITHUB_OUTPUT

      - name: Post Comment to PR
        env:
          COMMENT: ${{ steps.extract_message.outputs.message }}
          GH_TOKEN: ${{ github.token }}
        run: |
          gh api \
            repos/${{ github.repository }}/issues/${{ github.event.pull_request.number }}/comments \
            -f body="$COMMENT"
  enterprise-standard-check:
    runs-on: ubuntu-latest
    needs: [quality-security-analysis]

    steps:

      - name: Checkout code
        uses: actions/checkout@v3
        with:
          fetch-depth: 0  # ensures we get both PR base and head

      - name: Gather Full Code From Changed Files
        run: |
          # Identify changed files from the base (origin/main) to the pull request HEAD
          CHANGED_FILES=$(git diff --name-only origin/main...HEAD)

          # Build a JSON array containing filenames and their content
          echo '{"original files": [' > original_files_temp.json
          for file in $CHANGED_FILES; do
            # Skip .json and .txt files
            if [[ $file == \*.json ]] || [[ $file == *.txt ]]; then
              continue
            fi

            # If the file still exists (i.e., wasn't deleted)
            if [ -f "$file" ]; then
              CONTENT=$(jq -Rs . < "$file")
              echo "{\"filename\": \"$file\", \"content\": $CONTENT}," >> original_files_temp.json
            fi
          done

          # Remove trailing comma on the last file entry and close JSON
          sed -i '$ s/,$//' original_files_temp.json
          echo "]}" >> original_files_temp.json

      - name: Analyze Code Against Best Practices
        id: validate
        env:
          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
        run: |
          set -e
          # Read captured code
          ORIGINAL_FILES=$(cat original_files_temp.json)

          # Construct the prompt:
          #  - Summarize each best-practice category
          #  - Provide a rating for each category: 'extraordinary', 'acceptable', or 'poor'
          #  - Return a Markdown table titled 'Enterprise Standards'
          PROMPT="You are an Enterprise Code Assistant. Review each code snippet below for its adherence to the following categories: 

          1) Code Style & Formatting
          2) Security & Compliance
          3) Error Handling & Logging
          4) Readability & Maintainability
          5) Performance & Scalability
          6) Testing & Quality Assurance
          7) Documentation & Version Control
          8) Accessibility & Internationalization

          Using \${{ vars.BEST_PRACTICES }} as a reference, assign a rating of 'extraordinary', 'acceptable', or 'poor' for each category. Return a markdown table titled 'Enterprise Standards' with rows for each category and columns for 'Category' and 'Rating'. 

          Here are the changed file contents to analyze:
          $ORIGINAL_FILES"

          # Create JSON request for OpenAI
          jq -n --arg system_content "You are an Enterprise Code Assistant ensuring the code follows best practices." \
                --arg user_content "$PROMPT" \
          '{
            "model": "${{ vars.MODELNAME }}",
            "messages": [
              {
                "role": "system",
                "content": $system_content
              },
              {
                "role": "user",
                "content": $user_content
              }
            ]
          }' > request.json

          # Make the API call
          curl -sS https://api.openai.com/v1/chat/completions \
            -H "Content-Type: application/json" \
            -H "Authorization: Bearer $OPENAI_API_KEY" \
            -d @request.json > response.json

          # Extract the model's message
          ASSISTANT_MSG=$(jq -r '.choices[0].message.content' response.json)

          # Store for next step
          {
            echo "review<<EOF"
            echo "$ASSISTANT_MSG"
            echo "EOF"
          } >> $GITHUB_OUTPUT

      - name: Post Table Comment
        env:
          COMMENT: ${{ steps.validate.outputs.review }}
          GH_TOKEN: ${{ github.token }}
        run: |
          # If COMMENT is empty or null, skip posting
          if [ -z "$COMMENT" ] || [ "$COMMENT" = "null" ]; then
            echo "No comment to post."
            exit 0
          fi

          gh api \
            repos/${{ github.repository }}/issues/${{ github.event.pull_request.number }}/comments \
            -f body="$COMMENT"

Test the Workflow

Commit this workflow to your repository, then open a new PR. The workflow will run automatically, posting AI-generated feedback as a PR comment.

For a public example, see the OpenAI-Forum repository’s workflow: pr_quality_and_security_check.yml.

pr_quality_and_security_check.png

workflow_check.png