I used CodiumAI's PR Agent, and I love it

Recently, I started developing a PyTorch-esque library for educational purposes to know what goes into making a deep-learning library. I wanted to build this in an open-source way. But having not moderated any large open-source project before, I wanted some help with it. I have seen that moderating any medium to large project can be extremely difficult with a large influx of Pull Requests, and reviewing and testing them could take a lot of time and effort.

However, I stumbled across this tool by CodiumAI. It’s a PR Agent that provides multiple AI commands that work within GitHub on pull requests. So, I decided to try it out as this can be very useful if this project does explode 😁. Now, CodiumAI is well-known for its VSCode extension that provides tools to automate writing tests for your codebase. They also have a chat tool that allows you to explain your code, generate code, and prepare it for PR.

In this blog, I want to outline some of the AI agent's best features and commands and see how it differs from another similar tool on the market, GitHub CoPilot.

Availability

The best part about CodiumAI’s PR agent is that it is available right now. You can use it on any public repository by just commenting with @CodiumAI \command. This summons the AI agent that looks at the Pull Request along with the commits and the code to help provide insights and improve the quality of the code. For the folks and enterprises that use platforms other than GitHub, CodiumAI works seamlessly with other platforms such as BitBucket and GitLab, unlike the PR tool offered by GitHub CoPilot, which only works with GitHub. Also, I couldn't even try GitHub CoPilot as it has a waitlist and might not even be available for individual developers.

Moreover, you can quickly deploy your private PR Agent for private repositories by following the simple instructions here. Being completely open-source, you can run the PR agent with any of the below methods if you want to integrate it into your private repository and workflow. For example, with GitHub, you can use the PR Agent in any of the following ways.

• Run as a GitHub Action
• Run as a polling server
• Run as a GitHub App
• Deploy as a Lambda Function
• AWS CodeCommit

But, to demonstrate the features, I will stick to a public repository and use the Agent there.

Usage

Once you have set up the Agent, using it becomes very easy. With a few simple but powerful tools, you can create a better description of your PR, generate docstrings, get reviews and improvement suggestions, and also ask questions related to the PR. You might wonder why you can't just perform these actions in the IDE. Well, as this is a PR, the code in question may not have been written by you. Therefore, receiving insights and suggestions directly in GitHub is much easier. However, if you are the one who is going to make a PR, I highly recommend that you use CodiumAI’s Chat plugin on VS Code to write excellent PRs.

I have made an example Pull Request to my public repository. This pull request adds a MultiLayerPerceptron class to the Neural Network Library. Let’s look at some of these tools and how they work on the PR. You can check out this PR where I have used these commands.

The /describe Command

The /describe command is a powerful tool that can significantly improve the way a pull request (PR) is described. It allows you to provide a comprehensive and standardized description for your PR, especially when the original description is lacking in clarity or detail.

When you use the /describe command, it automatically generates a title for your PR based on the changes made, smartly identifies the type of the PR (e.g., bug fix, feature addition, documentation update), and writes a concise but informative description of the changes. It also provides an organized list of relevant files, categorized by their labels, making it easier for reviewers to navigate and understand the scope of the changes.

By using the /describe command, you can ensure that your PR is presented in a clear and professional manner, enhancing the overall quality of your contributions. It saves you time and effort in manually crafting a detailed description and helps maintain a consistent format across different PRs in your project.

The /review Command

The /review command is a powerful tool that allows you to conduct a thorough and comprehensive review of the PR. By using this command, you can gain valuable insights and information about the PR that can help you in making informed decisions.

The output generated by the /review command includes various important aspects of the PR that you need to be aware of. These include:

• Main Theme of the PR: This provides you with a clear understanding of the main objective or purpose of the PR.
• Summary of the PR: A concise summary of the key points and changes made in the PR.
• Type of PR: This indicates the type of PR, which can be helpful in categorizing and organizing your review process.
• Presence of relevant tests: One crucial aspect of a PR is the inclusion of relevant tests. Using the /review command, you can determine whether any tests have been added. In this particular case, no tests are present.
• Estimated Effort to review the PR: This provides an estimation of the effort required to review the PR, giving you an idea of the time and resources you need to allocate for the review process.
• Identification of security concerns: Security is a critical aspect of any code review. The /review command helps in identifying and highlighting any potential security concerns that may exist in the PR.

By utilizing the /review command, you can save a significant amount of time, as it offers a quick initial review of the PR, allowing you to focus on the most important aspects and address any major concerns promptly. Additionally, this command also provides valuable feedback on the overall quality of the code and suggests ways to make improvements, enabling you to enhance the overall codebase and deliver a higher-quality product.

The /improve Command

The /improve command is a powerful tool that analyzes code within a pull request (PR) and provides valuable suggestions to enhance the quality and readability of the code. In addition to offering recommendations for improving code performance, this command allows you to easily commit the suggested changes directly within the PR environment. This streamlined process eliminates the need to manually pull the code base, make fixes locally, and then push the changes again. By leveraging the /improve command, you can save significant time and effort while ensuring the code meets the highest standards of quality and efficiency.

The /add_docs Command

/add_docs command is by far my favourite one. The agent goes through the PR code and creates useful docstrings for the functions and class descriptions. I can see myself using this a lot as writing documentation is not something that I am very good at.

As a programmer, I often find myself struggling with writing documentation for my code. It's not that I don't understand the importance of good documentation, but rather that I find it tedious and time-consuming. That's why the /add_docs command might quickly become my go-to feature whenever I need to write documentation for my code.

The beauty of this command lies in its simplicity. All you need to do is run it, and the agent will automatically scan the PR code and create useful docstrings for the functions and class descriptions. This saves me a lot of time and effort, and I can focus on other aspects of my work.

One of the reasons I love this feature is that it's incredibly accurate. The generated docstrings are always relevant and informative, and I don't need to worry about missing any important details. Similar to the /improve command the changes can be directly committed to the code in place.

Overall, I think the /add_docs command is an excellent tool for programmers who want to create high-quality documentation for their code without spending too much time on it. I can see myself using this feature a lot in the future.

The /ask Command

The /ask command is a remarkable feature that I have come to appreciate. It enables you to ask the agent specific questions related to the PR, and it offers precise answers as comments. This command comes in handy when you need to get clarifications or insights into specific aspects of your project.

For instance, I used the /ask command to inquire about the input size required for my training data in the new MultiLayerPerceptron. The agent's response was incredibly accurate and helped me to fine-tune my project. It also found out that my input is a transpose of the data that I am loading in another part of my codebase, which was quite surprising.

Question: what are the expected input shapes for the X and y matrices

Answer:

The expected input shapes for the X and y matrices are as follows:

• X: The input data, X, should be a 2D numpy array where each row represents a sample and each column represents a feature. So, the shape of X should be (n_samples, n_features).
• y: The labels, y, should be a 1D numpy array where each element represents the label of the corresponding sample in X. So, the shape of y should be (n_samples,).

However, please note that in the train method, the X and y matrices are transposed for the training process. This means that within the train method, X is expected to be of shape (n_features, n_samples) and y is expected to be of shape (n_classes, n_samples) after one-hot encoding.

Additionally, the /ask command can be particularly useful when you encounter bugs or errors during PR testing. Instead of spending hours trying to figure out what went wrong, you can directly ask the agent about the issue, and it can offer relevant suggestions for how to fix the problem. This can save you a lot of time and effort and help you deliver high-quality projects more efficiently.

In summary, the /ask command is a powerful tool that can help you to get the most out of your PR projects. It provides accurate insights and suggestions, saving you a lot of time and effort when you encounter issues.

Comparison to GitHub CoPilot

Availability

As mentioned before, CodiumAI’s PR Agent is available right now and is free to use for individual developers. There are paid plans that offer more features.

Also, the PR agent works on other platforms other than GitHub which can be useful for a lot of people.

Open-source Community

CodiumAI’s PR Agent is completely open-source, and this makes it more transparent and reliable. The company is also transparent about how the tool works and there are new features coming regularly. Conversely, GitHub CoPilot is not opensource and is heavily integrated only in Microsoft products.

Commands and Features

CodiumAI’s PR Agent gives you access to a lot of powerful commands. You get more control over what exact information you get or tasks that you want the agent to do.

Final Takeaway

I must say, I'm quite impressed with the power of this tool. I can see it becoming a staple in all future Pull Requests. However, the only drawback at the moment is its reliance on OpenAI's GPT model. While this doesn't bother me personally, it may be a concern for those who prefer to use an open-source AI model. According to their GitHub page, CodiumAI plans to add support for using your own model instead of OpenAI, so that's something to look forward to. Additionally, there are more new commands on their roadmap to further improve the experience. All in all, I'm excited to integrate this tool into my workflow.

---

Want to connect?

🌍My Website

🐦My Twitter

👨My LinkedIn