Integration meets automation: Simplifying PRs with OpenAI

Code reviews are crucial for effective development workflows, ensuring quality, maintainability, and adherence to best practices. They allow developers to assess solutions from multiple perspectives, uncover potential issues, and address overlooked edge cases.

However, solo developers often lack a second opinion, making it difficult to apply this practice consistently. Even in large teams, tight deadlines and heavy workloads can lead to rushed or superficial reviews, increasing the risk of undetected bugs or suboptimal code.

This is where AI can be a valuable ally. By automating aspects of pull request (PR) reviews, AI tools can identify potential issues, suggest improvements, and maintain a high standard of code quality without burdening developers with additional work.

In this post, we’ll explore how you can use AI for code review and integrate AI-powered pull request reviews into your GitHub workflow,  enhancing efficiency and code quality while reducing the need for manual effort.

Building a Custom GitHub Action

To make AI-assisted PR reviews more easily reusable across projects, we packaged the functionality into a custom GitHub Action. This action automates the process by fetching the PR diff, analyzing the code using OpenAI’s API, and posting the review comments back to GitHub.

The action requires several inputs, including API keys and repository details, to ensure secure authentication and correct PR identification. Additionally, it supports optional parameters for customizing AI behavior, such as selecting the GPT model or setting token limits for responses. Here’s the action.yml file that defines the GitHub Action:

runs:
  using: 'composite'
  steps:
    - name: Create temp directory
      run: mkdir -p ${{ github.action_path }}/tmp
      shell: bash

    - name: Extract PR diff
      run: |
        git diff ${{ github.event.pull_request.base.sha }} ${{ github.event.pull_request.head.sha }} > ${{ github.action_path }}/tmp/pr_diff.txt
        echo "DIFF_FILE_PATH=${{ github.action_path }}/tmp/pr_diff.txt" >> $GITHUB_ENV
      shell: bash

    - name: Make script executable
      run: chmod +x ${{ github.action_path }}/scripts/analyze-code.sh
      shell: bash

    - name: Analyze code with OpenAI
      env:
        OPENAI_API_KEY: ${{ inputs.OPENAI_API_KEY }}
        GITHUB_TOKEN: ${{ inputs.GITHUB_TOKEN }}
        REPOSITORY: ${{ inputs.REPOSITORY }}
        PR_NUMBER: ${{ inputs.PR_NUMBER }}
        DIFF_FILE_PATH: ${{ env.DIFF_FILE_PATH }}
        GPT_MODEL: ${{ inputs.GPT_MODEL }}
        MAX_TOKENS: ${{ inputs.MAX_TOKENS }}
      run: ${{ github.action_path }}/scripts/analyze-code.sh \ 
        $OPENAI_API_KEY $GITHUB_TOKEN $REPOSITORY $PR_NUMBER \ 
        $DIFF_FILE_PATH $GPT_MODEL $MAX_TOKENS
      shell: bash

After passing these inputs, the action triggers a Bash script that handles the entire process in the background. The script:

• Extracts the PR diff.
• Sends the diff to OpenAI’s API for analysis.
• Receives AI-generated feedback.
• Posts the feedback as a comment on the pull request using the GitHub API.

This automation ensures consistent, efficient, and high-quality PR reviews while allowing developers to focus on more critical tasks.

Leveraging OpenAI API for AI code review

The OpenAI API is a powerful tool capable of generating insights, identifying code smells, and suggesting improvements. In our workflow, it plays a central role by analyzing PR changes and providing meaningful review comments.

To make this integration seamless, we need to craft a Bash script that:

• Prepares the AI prompt – Includes clear instructions and the extracted PR diff.
• Sends the request to OpenAI – Calls the API with a specified GPT model and token limit.
• Processes the AI response – Extracts a concise summary of the AI’s feedback.

This function sends a prompt along with the PR diff to OpenAI, allowing the AI to generate relevant feedback that will later be posted on GitHub.

# Function to send the prompt to OpenAI API and get the response
call_openai_api() {
  local openai_api_key="$1"
  local full_prompt="$2"
  local gpt_model="$3"
  local max_tokens="$4"

  local messages_json
  messages_json=$(jq -n --arg body "$full_prompt" '[{"role": "user", "content": $body}]')

  curl -s -X POST "https://api.openai.com/v1/chat/completions" \
    -H "Authorization: Bearer $openai_api_key" \
    -H "Content-Type: application/json" \
    -d "{\"model\": \"$gpt_model\", \"messages\": $messages_json, \"max_tokens\": $max_tokens}"
}

GPT-4o-mini, which is used as a default model here, is an excellent choice for developers looking for an affordable yet high-quality AI-powered code review assistant. It strikes a balance between cost-effectiveness and intelligent analysis, making it ideal for teams that need quick, reliable feedback without the high expense of premium models. For this purpose, it is important to note that reviews from mini models are often faster and more concise, as they prioritize efficiency and are perfect for PR comments that need to be clear and to the point.

Using GitHub API to post an automated review

Once the AI generates feedback, the next step is to post the review as a comment on the PR using the GitHub API. This ensures that developers can access the AI insights directly within the code discussion, streamlining collaboration. 

The function below formats the AI response and sends it as a PR comment:

# Function to post the summary as a comment on the pull request
post_summary_to_github() {
  local github_token="$1"
  local repository="$2"
  local pr_number="$3"
  local summary="$4"

  local comment_body
  comment_body=$(jq -n --arg body "$summary" '{body: $body}')

  curl -s -X POST \
    -H "Authorization: Bearer $github_token" \
    -H "Content-Type: application/json" \
    -d "$comment_body" \
    "https://api.github.com/repos/$repository/issues/$pr_number/comments"
}

Below is an example of how the AI-generated review appears on a pull request:

This blog post outlines the core steps of the GitHub Action, but the full implementation includes additional logic for error handling, input validation, and process automation. You can explore the complete source code on the GitHub Marketplace and easily integrate it into your repository. Check it out here: Analyze Pull Request with OpenAI.

Summary

We’ve explored how to automate pull request reviews using GitHub Actions and OpenAI API, reducing manual effort while maintaining code quality. By leveraging AI-powered insights, teams can:

• Streamline code review workflows
• Identify issues and improvements faster
• Ensure consistency across pull requests

As AI continues to evolve, automated code review assistants like this one will become even more powerful—helping developers write better software, faster.

Note: While this solution uses the OpenAI API, it’s crucial to be aware of and address privacy concerns when implementing such systems. Ensure that sensitive code is not inadvertently shared with external AI services. 

Have you tried incorporating AI into your code review process? We’d love to hear about your experiences and insights!