GPTReview - Integrate Your Own AI-Based Code Reviewer Using GitHub Actions - Part 2

Jul 24, 2024 by Atulpriya Sharma

In the previous blog post, we created our own AI-based code reviewer using GPTScript and a chat-based interface. Using GPTScript, we used AI to review the changed files in an open pull request and provide feedback.

That was a good example to start with. However, when most of your software development workflow is automated, you’d also want the code review process to be automated as part of the CI/CD pipeline.

In this blog post, we’ll integrate our AI-based code reviewer, built using GPTScript, into a CI/CD pipeline using GitHub actions.

GPTReview in GitHub Actions

Making automated code reviews part of your CI/CD workflows is crucial, as it helps developers get feedback faster. GitHub is one of the most widely used tools, and GitHub actions are perfect for building your CI/CD workflows.

Pre-requisites

  • An OpenAI API Key.
  • GitHub repository.

Writing the GPTScript

The first step is to create the GPTScript. Since GPTScript is written primarily in natural language, writing the script to perform code review is easy. Below is the gptreview.gpt file:

Name: Code Reviewer
Description: A tool to help you perform code review of open PRs
Context: learn-gh
Tools: sys.exec, sys.http.html2text?, sys.find, sys.read, sys.write
Args: PR_URL: The GitHub PR_URL

You have the gh cli available to you. Use it to perform code review for a pr from the $(repo) provided.
Perform the following steps in order:
1. Identify the files changed in the pull request ($PR_URL) using the pr number and perform a diff.
    1. Analyze the complete code of each identified file and perform a detailed line by line code review.
    2. Repeat the process for each changed file in the pr.
3. Share your review comments separately for each file.
4. In a new line write "Code: Approved" or "Code: Require Changes" based on the review comments.
---

Name: learn-gh
Description: A tool to help you learn gh cli

#!/usr/bin/env bash
echo "The following is the help text for the gh cli and some of its sub-commands. Use these when figuring out how to construct new commands.  Note that the --search flag is used for filtering and sorting as well; there is no dedicate --sort flag."

gh --help
gh repo --help
gh pr --help
gh pr checkout --help
gh pr diff --help

Let us understand what the script does:

  • The script takes a "--PR_URL" parameter, which is the URL for the pull request. It takes this from the environment variable set by the GitHub action.
  • It identifies the files changed in the pull request and performs a code review for the changed file.
  • Based on the code review, it writes "Code: Approved" or "Code: Require Changes"

You can find this GPTScript in our examples folder.

Writing GitHub Action

The next step is to create your GitHub action. Setting up a GitHub action for your repository is fairly simple. For detailed steps, follow the GitHub Actions quickstart guide.

Configuring Environment Variables

For the GitHub action to be able to execute our GPTScript, we need to set up an environment variable for the OpenAI API key. To do this, navigate to the “Settings” tab from your repository page and open “Actions” located under “Security and variables”.

Click on the “New Repository Secret” button, and on the new screen, provide a name and value. In this case, the name is “OPENAI_API_KEY” and the value will be your OpenAI API Key.

GH-env-variable.png

Once you’ve configured the GitHub action, update the yaml file with the following:


name: PR Review with GPTScript

on:
  pull_request:
    types: [opened, synchronize, reopened]

jobs:
  pr_review:
    runs-on: ubuntu-latest
    steps:
    - name: Checkout repository
      uses: actions/checkout@v2

    - name: Get PR Details
      id: pr_details
      run: |
        PR_URL=$(jq -r '.pull_request.html_url' $GITHUB_EVENT_PATH)
        PR_NUMBER=$(jq -r '.pull_request.number' $GITHUB_EVENT_PATH)
        PR_FILES=$(jq -r '.pull_request.changed_files' $GITHUB_EVENT_PATH)
        echo "PR_URL=${PR_URL}" >> $GITHUB_ENV
        echo "PR_NUMBER=${PR_NUMBER}" >> $GITHUB_ENV
        echo "PR_FILES=${PR_FILES}" >> $GITHUB_ENV

    - name: Install GPTScript
      run: curl https://get.gptscript.ai/install.sh | sh

    - name: Run GPTScript for Code Review
      id: run_gptscript

      env:
        OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
        GH_TOKEN: ${{ secrets.GH_TOKEN }}
      run: |
        {
        echo 'REVIEW&lt;<EOF'
        gptscript codereview.gpt --PR_URL=${{ env.PR_URL }}
        echo EOF
        } >> "$GITHUB_ENV"   

    - name: Post Review Comment
      env:
        GITHUB_TOKEN: ${{ secrets.GH_TOKEN }}
      run: |
        gh pr comment ${{ github.event.pull_request.number }} --body "$REVIEW"

    - name: Set PR Status Fail
      if: contains(env.REVIEW, 'Require Changes')
      run: |
       echo "Code Requires Changes"
       exit 1

    - name: Set PR Status Pass
      if: contains(env.REVIEW, 'Approved')
      run: |
       echo "Code Approved"

Let’s understand this GitHub action:

  • The script triggers when any PR is opened, reopened, or synchronized.
  • It checks out the code and configures the environment variables with things like PR number, URL, and the changed files.
  • It then installs GPTScript, executes the GPTScript file, and performs a code review of the changed files.
  • Based on the review comments, it sets the REVIEW variable based on which the workflow is marked as pass or fail.

Executing the GPTScript

To execute this workflow, create a new PR with some code changes. Here we will add a readme.md file with the following content.


This is a readme file.

codereview-gh-action-1.png

This will trigger the GitHub action.

codereview-gh-action-2.png

GPTScript checks the details of the PR and realises that there was a new README.md file added. It then reviews the contents of the file. Since we only had one line in the file, it doesn’t convey anything about the repository. Hence, GPTScript shares detailed review comments for the file mentioning that the file needs more details.

Hence the code review fails in this case, so the workflow's status is set to fail.

codereview-gh-action-3.png

The action updates the status in the PR and posts a comment with the analysis.

Next, we create a new PR, this time for a Python file. We add comments to the functions and try-catch blocks wherever necessary. This time, when the GitHub action is triggered, GPTScript analyses the code, and the review passes. The workflow run's status is marked as pass, and a comment with the review comment is posted.

codereview-gh-action-4.png

Using a few lines of natural text, we built GPTReview and performed code reviews for PRs. Integrating that with GitHub actions made it a part of a workflow. So, every time there’s a new PR, GPTReview will perform the code review and update the status.

Summary

This guide gave an overview of how you can use GPTScript to create an AI-based code review tool and integrate it with GitHub actions. Making the code review process part of your development workflows is crucial as it enables faster feedback and removes the scope for manual errors.

In the next part of this tutorial, we’ll integrate this into a CI/CD pipeline using Jenkins.

Until then, check out GPTScript and try building AI-enabled applications with ease. If you’re new here or need guidance, join the GPTScript Discord server and get all your queries answered.

This tutorial is part of a series on GPTReview, an AI code reviewer built on GPTScript. Check out parts 1 & 3 below.