Announcing AppMap for GitHub - Runtime Code Reviews for Every Pull Request

#ruby #java #python #node

Most of us have a love-hate relationship with code reviews. We rely on code reviews to ensure that the code we send to production is clean, performs well, and is free of security flaws. But code reviews can also add a lot of overhead and delays to shipping code.

Some people suggest that code reviews should be eliminated altogether. Well, I'm not in that camp, and here's why: Unexpected code behavior and deep runtime defects are responsible for a staggering 40% of performance problems¹ and 50% of security concerns² in software development. But, I do think that it's important to make code reviews easier for developers, and at AppMap we've created AppMap for CI to help do that.

“Security and performance reviews are part of the process on any engineering team, and ours is no different. It can take months to find issues and weeks to fix problems using static analysis and code reads. With AppMap, that time is reduced to minutes.”

-Padraig Coffey, CEO at Zartis.

If you know AppMap already, you know us for our VS Code and JetBrains extensions that automatically generate complete, accurate, interactive diagrams of your code. So now, with AppMap for CI, we've built on the solid foundation of AppMap to make code reviews easier, better, and faster.

The first CI platform we support is GitHub Actions. Our solution combines an AppMap GitHub App with GitHub Actions. When you build your project, the AppMap Analysis Action analyzes AppMaps from before and after each code change. It's able to identify critical performance, security, and stability issues and annotate specific lines of code in the pull request. It also performs deep analysis of test case failures and web service API changes, giving you a big head start on code review. And by the way, the performance and security flaws that AppMap finds are not discoverable by static analysis tools - because modern code is too dynamic for static analysis tools to understand.

“When we initially adopted AppMap, we were only using it for visualizing N+1 queries locally. We later integrated it into our CI run, and we are finding the GitHub comments posted by AppMap to be very useful. The OpenAPI integration, in particular, was a surprise: it’s nice to see an automatically generated summary of what endpoints have changed in a PR."

-Paul Kuruvilla, CTO at CodeCrafters.

Your data stays with you

We know that your code is sensitive, and you don't want it to be transmitted to 3rd parties. All AppMaps are stored as artifacts in your GitHub build system and are sent directly to your browser when you view them. The data only resides in two places - the GitHub Actions environment, and your machine. Learn more at appmap.io/security.

A tour of AppMap's "runtime code review"

✅ AppMap issues a summary report of its runtime analysis as a comment on every PR. The PR comment includes four key categories:

Failed tests

Identifies the root cause of test failures with detailed insights and direct links to the line of code which caused the error. AppMap also includes source diffs and behavioral sequence diffs for efficient debugging.

API Changes

Detects API route differences, including new, deleted, or modified routes, as well as alterations in response attributes like body, content, and descriptions.

Security flaws

Uncovers issues such as broken or missing access control, deprecated cryptography, improper session management, and missing authentication. AppMap focuses on the runtime security flaws that dominate the OWASP Top 10, and can't be reliably detected by static analysis.

Performance problems

Identifies performance problems like N+1 SQL queries, very complex queries, RPC anti-patterns, and plain old slow code.

Example - Performance problems

Here's an example of a code change that introduces four new performance problems.

The AppMap runtime code review includes details about the flaws detected, a stack trace, and links to the offending source code.

AppMap highlights specific code changes that are contributing to the flaw. It's able to focus on the specific code changes that are relevant to the newly introduced problem.