Automatically Generate and Update OpenAPI Documentation

The OpenAPI specification, which is formerly known as Swagger Specification, is a community driven open standard, programming language-agnostic interface description for HTTP APIs. This allows both humans and computers to discover and understand the capabilities of a service without requiring access to the source code.

AppMap is an open source tool used by tens of thousands of developers to visualize their application’s runtime behavior. Unlike static analyzers, AppMap records the details of how an application runs, and it uses that information to find design flaws, security holes, and performance issues.

Because AppMap records how an application executes at runtime, it can see and record all of the API calls processed including the schema of each request and response. Using the AppMap CLI (or the VS Code or JetBrains extension) we can automatically output OpenAPI specifications for an app, with zero work on the part of the developer.

While it’s often recommended to create your OpenAPI docs first and then build out the corresponding API endpoints from the spec, we know that modern software development is often messier. Developers are often getting tossed into old projects with limited documentation and creating these docs by hand is often error prone and annoying to keep up to date as the code changes.

AppMap currently supports Ruby, Java, Python, and Javascript projects and the libraries are MIT licensed.

Install AppMap

The easiest way to get up and running with AppMap is by installing the VS Code or JetBrains extension which walks users through the process end to end. But if you are not currently using one of those code editors or just love to live on the command line, you can also install the Appmap client locally.

The AppMap client is written in Javascript which makes it easy to run across a variety of platforms. You’ll need Node.js installed or alternatively you can grab pre-built binaries for @appland/appmap on Github. I’ll be using a sample ruby on rails application from the Ruby on Rails Tutorial, 6th Edition book. You could also

Since this is a Ruby project - I can just add one line to the top of my Gemfile to include AppMap recording in development and test.

gem 'appmap', :groups => [:development, :test]

Then run bundle install to download and install AppMap into your project.

It’s important to add this before any other gems as we want the AppMap gem loaded first so that it can have the opportunity to observe the other gems involved in the application.

You can also run the @appland/appmap project installer command via npx which lets you easily execute node package binaries. The installer will identify the language of your project and setup the libraries correctly if you don’t want to do it manually.

$ npx @appland/appmap install -d /Users/dev_user/repos/sample_app_6th_ed
? Multiple project types were found in /Users/dev_user/repos/sample_app_6th_ed. Select one to continue. (Use arrow keys)
❯ Bundler
  yarn

Because this Ruby on Rails project has both a Gemfile and a package.json file the installer just needs to know if it will install the Ruby library or the Javascript one. Since this is a Ruby project, we’ll select bundler. This will just automate adding the gem correctly to your Gemfile and validating its setup correctly.

? Multiple project types were found in /Users/dev_user/repos/sample_app_6th_ed. Select one to continue. Bundler
Installing AppMap agent for ....
? AppMap is about to be installed. Confirm the details below.
  Project type: Bundler
  Project directory: /Users/dev_user/repos/sample_app_6th_ed
  Git remote: origin    git@github.com:land-of-apps/sample_app_6th_ed.git (fetch)
  Ruby version: 3.0.2p107
  Gem home: /Users/dev_user/.asdf/installs/ruby/3.0.2/lib/ruby/gems/3.0.0

  Is this correct? Yes
✔ Installing AppMap...
✔ Validating the AppMap agent...

Create AppMaps

Now with AppMap installed we need to record our application, and there are a few ways to do that which will allow AppMap to actually see all of the API requests that we’ll use to populate the OpenAPI Specification file.

Test Case Recording

AppMap integrates with many popular testing frameworks, so if you can run your tests to interact and test all your API endpoints. This sample ruby application has extensive test coverage so I can simply run tests and AppMap will generate one AppMap per test.

$ bundle exec rails test
AppMap minitest recording is enabled because RAILS_ENV is 'test'
Running via Spring preloader in process 42539
Started with run options --seed 58694

  67/67: [=========================================================================================================================================================] 100% Time: 00:00:04, Time: 00:00:04

Finished in 4.21250s
67 tests, 282 assertions, 0 failures, 0 errors, 0 skips

Remote Recording

A remote recording is a great way to record your application if you don’t have robust test case coverage OR if you wanted to record a sequence of HTTP requests or other types of user interactions.

You can remote record your application by running it locally, in a docker container, or in a development environment. Since AppMap records everything in a request, include parameters and variables, we do NOT recommend recording production applications as you may inadvertently record sensitive data.

To record, first run your application, in my example i’ll run the rails server command.

$ bundle exec rails server
=> Booting Puma
=> Rails 6.0.4.1 application starting in development
=> Run `rails server --help` for more startup options
AppMap remote recording is enabled because RAILS_ENV is 'development'
AppMap requests recording is enabled because RAILS_ENV is 'development'
Puma starting in single mode...
* Puma version: 5.3.2 (ruby 3.0.2-p107) ("Sweetnighter")
*  Min threads: 5
*  Max threads: 5
*  Environment: development
*          PID: 48707
* Listening on http://127.0.0.1:3000
* Listening on http://[::1]:3000
Use Ctrl-C to stop

We see in the output that remote recording is enabled. Now we can run the record command on the command line to start the recording process.

$ npx @appland/appmap record remote
✔ Checking if the AppMap agent is available at http://localhost:3000/

   ╭─────────────────────────────────────────────────────────╮
   │                                                         │
   │   AppMap agent is available at http://localhost:3000/   │
   │                                                         │
   ╰─────────────────────────────────────────────────────────╯


? Press enter to start recording

When you are ready to start recording, press enter, and then interact with your application by clicking around a web UI or issuing API calls to various endpoints. Try to keep recordings focused on a specific interaction to keep the AppMap from getting unnecessarily large.

Complete the recording by pressing enter again and the AppMap will open in your browser (or within VS Code or JetBrains if you are using those editor extensions).

? Press enter to stop recording

   ╭────────────────────────────────────────────────────────╮
   │                                                        │
   │   Recording has finished, with 908856 bytes of data.   │
   │                                                        │
   ╰────────────────────────────────────────────────────────╯

? Choose a name for your AppMap: User Login
✔ Saving recording to User Login.appmap.json in directory tmp/appmap/remote

   ╭──────────────────╮
   │                  │
   │   AppMap saved   │
   │                  │
   ╰──────────────────╯

Opening tmp/appmap/remote/User Login.appmap.json
? What would you like to do next: (Use arrow keys)
❯ Reopen tmp/appmap/remote/User Login.appmap.json
  Exit

Request Recording

Another way to generate AppMaps is by recording each individual request. This is currently supported for Ruby on Rails applications with upcoming support for Python Django applications as well.
For this - simply run your application and AppMap request recording will be enabled by default, interact with your application and AppMap will generate a single AppMap per request and open them similar to above.

$ ls -lt tmp/appmap/requests/ | head
total 6800
drwxr-xr-x  19 dev_user  staff     608 Oct  5 14:22 1664994122_863181_http_localhost_3000_users_1
-rw-r--r--   1 dev_user  staff  478242 Oct  5 14:22 1664994122_863181_http_localhost_3000_users_1.appmap.json
drwxr-xr-x  19 dev_user  staff     608 Oct  5 14:22 1664994122_821723_http_localhost_3000_microposts_295
-rw-r--r--   1 dev_user  staff  125491 Oct  5 14:22 1664994122_821723_http_localhost_3000_microposts_295.appmap.json
drwxr-xr-x  19 dev_user  staff     608 Oct  5 14:22 1664994120_190062_http_localhost_3000_users_1
-rw-r--r--   1 dev_user  staff  473555 Oct  5 14:22 1664994120_190062_http_localhost_3000_users_1.appmap.json
drwxr-xr-x  19 dev_user  staff     608 Oct  5 14:21 1664994118_6177652_http_localhost_3000_users
-rw-r--r--   1 dev_user  staff  316685 Oct  5 14:21 1664994118_6177652_http_localhost_3000_users.appmap.json

Generate an OpenAPI Spec

Now that we have AppMaps of our API requests, we can run a single command to output the OpenAPI v3 supported specification file and use with other tools such as Hoppscotch, which is an open source API development ecosystem, or Postman which is another popular commercially available API tool.

npx @appland/appmap openapi --output-file openapi.yml

You can now take this file and import it into Hoppscotch as a new collection by navigating to the collections button on the right hand side.

Select on the “Import/Export” icon.

Choose to import from OpenAPI

And then finally, select the OpenAPI file we created in the previous step.

Now you can use these tools to more easily interact with your APIs, test new or updated features, and finally export these OpenAPI specifications to your users so that they will know how to interact with your application as well.

You can learn more about other features of AppMap by reviewing our developer documentation or you can join the developers in our community Slack group.