RevAI: Let AI Review it First!

RevAI: Let AI Review it First!

Sadra Yahyapour's photo
Sadra Yahyapour
·

5 min read

Table of contents

As a maintainer, do you need an AI that reviews your pull requests right before you check out the changes or even merge them?! Well, here it is.

RevAI is a GitHub Action that enables you to create a pipeline on each pull request opened on your repositories. Underneath, it uses MindsDB to handle the requests and LLM interactions.

Case Scenario

Consider someone has branched off a new development line from your repository and has changed a file called file.py. Now, he opens a pull request from him/feature to you/main.

Right when he opens the PR, RevAI gets triggered and starts reviewing his changes.

Ultimately, it leaves a comment that shows what AI thinks about the changes that have been made.

As you can see, during this PR, there was only one file (file.py) that got changed and RevAI has reviewed the file based on the prompt that I had given earlier. We'll go through this process later on.

Customization

As the maintainer of the repository, you have a full control over the factors and criterias that RevAI is going to review the code based on. You can train the model based on any prompting statement that you want. You can also change the commenting template as well.

Setting Up

In this section, we'll talk about the steps that need to be taken to have an instance of RevAI on your own repositories.

1. Create a MindsDB account and train a GPT model

Before we begin..

  1. Create an account on OpenAI

  2. Create a chatGPT instance

  3. Generate an access token from the dashboard and keep it somewhere safe

Log into MindsDB dashboard. Create an instance and run the following SQL snippet.

CREATE MODEL mindsdb.gpt_model
PREDICT response
USING
engine = 'openai',
api_key = '<TOKEN>',
model_name = 'gpt-3.5-turbo',
prompt_template = 'review the {{text}} based on clean-code principles and pep rules then rate it from 1 to 10 and put it in the "score" field. Put your thoughts about it in one sentence in the "message" field.

Respond with no formatting, but in the following structure:

{
    "score": int,
    "message": str
}';

Replace <TOKEN> with your OpenAI token. You can also put whatever prompting statement you want, but in this case, make sure to refer to {{text}} as the source code content inside your prompt and your prompt should end with..

Respond with no formatting, but in the following structure:

{
    "score": int,
    "message": str
}

It's quite obvious that I'm asking AI to review the code and score it based on clean-code principles and pep rules. I assume that it takes only Python files, but you can set any prompting that meets your needs.

2. Set the secrets

Navigate to your repository. Go to Settings > Secrets and variables > Actions. Click on New repository secrets. You need to add two secrets here. One should be named EMAIL that contains your MindsDB's account email and the other one is PASSWORD that is valued to your account's password.

Your repository secrets panel should look like this..

3. Set up the template and workflow

Create .github/comment-template.md file. Put the following markdown snippet as the commenting template.

| 📂 **File**   | 💬 **Comment** | 🏆 **Score**    |
| :-----------: |---------------| :-------------: |
| `{{ .file }}` | {{ .message }} | {{ .score }}/10 |

You can also modify this template and design the structure that you want RevAI to put its thoughts and reviews in.

Don't forget to mention {{ .file }}, {{ .message }}, and {{ .score }} placeholders in your comment-template.md file.

This is a quick look of the template that we designed..

Make a new workflow by creating .github/workflows/review.yml file. Put the following YAML configuration in review.yml file.

name: RevAI Reviewing

on:
  pull_request:
    branches:
      - 'main'

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
      - name: Checking out
        uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Get changed files
        id: changed-files
        uses: tj-actions/changed-files@v41

      - name: Use RevAI
        id: revai
        uses: lnxpy/revai@0.1.0
        with:
          email: ${{ secrets.EMAIL }}
          password: ${{ secrets.PASSWORD }}
          files: ${{ steps.changed-files.outputs.all_changed_files }}

      - name: Render template
        id: template
        uses: chuhlomin/render-template@v1.4
        with:
          template: .github/comment-template.md
          vars: |
            file: ${{ steps.revai.outputs.file }}
            message: ${{ steps.revai.outputs.message }}
            score: ${{ steps.revai.outputs.score }}

      - name: Create comment
        uses: peter-evans/create-or-update-comment@v3
        with:
          issue-number: ${{ github.event.number }}
          body: ${{ steps.template.outputs.result }}

This workflow gets triggered whenever someone opens a new pull request to your main branch. To have a brief look at what this workflow does, we can sum them up into a few sequential jobs.

  1. It checks out to the commit state where the PR is.

  2. Gets a list of changed files.

  3. Uses RevAI and sends the changed files as well as your MindsDB credentials.

  4. Renders the commenting template with the results that RevAI has provided.

  5. Finally, comments out the review.

You can also set (a) default reaction(s) to RevAI's comments.

      - name: Create comment
        ...
        with:
          ...
          reactions: |
            heart
            hooray
            laugh

Tech Stack

  • PyAction: Creating GitHub Actions in Python. (more..)

  • MindsDB: AI Development Cloud Platform (more..)

Conclusion

With the massive growth of AI and LLMs, we'll be seeing a huge improvement in our CI and even CD cycles. There will be no need for us (humans) to review or point out any approval for minimal changes. We have AI!

Special thanks to Hashnode and MindsDB for sorting up this cool hackathon. 🍻

MindsDBHackathonmindsdbAIGitHubci-cdautomationcodereviewgenerative aiopenaichatgptllm