Andrew Owen

Posted on May 21, 2022 • Edited on Dec 19, 2022

Creating a developer portal with no budget

#devportal #swagger #redoc #hugo

When I signed up to dev.to, I skim read the terms and conditions and missed the crucial point that articles must contain the full text and not just the links. My own site is built on Hugo, which I used to create my first dev portal and the source text is in Markdown. So for most articles I've been able to copy the original across. I've lost syntax highlighting and Mermaid diagram support, but the crucial info is there. However, in the case of this article, which was originally a three part series, neither editing the Markdown nor copying and pasting from the published site worked. Therefore, this is an abridged version of the original (which you can find on my personal dev blog: Byte High, No Limit.

In this article, I'll outline how to create a fully featured dev portal for your Swagger or OpenAPI 3.0 content without spending a dime.

Part I: Backend

A dev portal is just a website that presents API (and other) docs to developers. It can be as simple as a static web page or as complex as you want to make it. In this series, we'll make use of the following back-end technologies:

Static Site Generators
Source control
Containerization
Automation

In addition, you'll inevitably spend a fair amount of time using a browser. For better or worse, Chrome is the new standard.

Static Site Generators

Hugo

Hugo is a type of web server known as a static site generator (SSG). It's lightweight and fast and can serve dynamic and static content. You can test content locally before deploying it. For OpenAPI content, we'll use the command line version of ReDoc to convert Swagger JSON files to a static HTML page that will be served by Hugo.

Install CLI tools (macOS)

Homebrew is a package manager. From the Terminal, enter /usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
Git provides source control. From the Terminal, enter git and follow the instructions to install Xcode.
Hugo is a static site generator. From the Terminal, enter brew install hugo.
NPM is a package manager for JavaScript. From the Terminal, enter brew install nodejs. ReDoc renders an OpenAPI file into a static HTML page. From the Terminal, enter npm install -g redoc-cli@0.13.2. You can leave off the @<version_number>, but I've found npm doesn't always get the latest version if you do. Ensure you are using version 0.9.8 or later:redoc-cli --version\.
(Optional) Swagger2PDF converts an OpenAPI JSON file into a static PDF document. From the Terminal, enter npm install swagger-spec-to-pdf.

Install CLI tools (Windows)

Requirements

Powershell 3 (or later). Powershell 5 is already installed in Windows 10.
.NET framework 4.5 (or later).

Instructions

Download and install Scoop.
Git provides source control. From PowerShell, enter scoop install git.
Hugo is a static site generator. From PowerShell, enter scoop install hugo.
NPM is a package manager for JavaScript. From PowerShell, enter scoop install nodejs.
ReDoc renders an OpenAPI file into a static HTML page. From PowerShell, enter npm install -g redoc-cli0.13.2. You can leave off the @<version number>, but I've found npm doesn't always get the latest version if you do. Ensure you are using version 0.9.8 or later:redoc-cli --version.
(Optional) Swagger2PDF converts an OpenAPI JSON file into a static PDF document. From PowerShell, enter npm install swagger-spec-to-pdf.
cURL is a command line data transfer tool. From PowerShell, enter scoop install curl.

Test a local copy of the dev portal

Navigate to the folder where you have cloned the Git repository.

From the command line, enter hugo server.
Copy the URL from the console output and paste it into the Chrome browser.

Add Google Analytics to a static HTML page

Paste the following text after the <head> tag:
<script async src="https://www.googletagmanager.com/gtag/js? id=<yourID>"></script>

Source control

Git

If you are generating comments from code, you'll be working directly in the software repository. Typically, this will be a Git repository. You should familiarize yourself with the processes for creating branches, creating pull requests, resolving conflicts and merging changes. There are a number of commercial hosting options including GitHub, GitLab and Bitbucket. Here we'll look at Bitbucket.

Get the URL from Bitbucket

In Chrome, navigate to the required repository.
In the left icon menu, click Clone.
Copy the URL.

Clone the repository and create a local branch

Open PowerShell / Terminal and navigate to where your local repository folder. Example: ~/development/.
Enter git clone and paste the path you copied from Chrome. This creates a local copy of the repository.
In Visual Studio Code, open the repository's folder.
In the lower left corner of the window, click Branch.
Enter a name in the box ( feature/apidocs-<api class name>) and select the branch to base it on. Example: development

You can now make your changes.

Publish changes

In Visual Studio Code, in the left icon menu, click Source Control.
Enter a short description of why you made the changes in the box.
Click the Commit (tick) icon. If prompted to configure Visual Studio Code to automatically stage changes, you should do so.
In the lower left corner of the window, click Publish Changes.

Create a pull request

In Chrome, navigate to the Bitbucket dashboard.
From the Repositories menu, select the repository you are working with.
From the left icon menu, click Create pull request.
Select your local repository from the Source list.
Confirm the Destination repository (typically, development) and click Continue.
Enter a Description. This should contain any additional explanation for the change.
Attach a copy of the ReDoc HTML file.
Select Reviewers. Typically, this is pre-populated. You can start typing a name to find a user. Ensure the product owner is included.
Click Create.

You'll receive email notifications when the status of the pull request changes. Example: When the change is approved.

Resolve conflicts

In your local branch: git pull origin master.
Merge incoming code changes while retaining doc changes.
git commit -am "<your commit message>"
git push

Merge changes

In Chrome, navigate to the pull request and click Merge.

Delete branch

After the change is successfully merged, from the left icon menu, click Branches.
Locate the branch you were working in and from the Actions menu, select Delete

branch.

Click Delete to confirm your action.

Edit text in Bitbucket

While not directly related to APIs, you may on occasion have to modify text contained in a file in a Git repository stored in Bitbucket. You must have a Bitbucket account to edit the files.

Navigate to the file in Bitbucket.
Click Edit.
Make your changes.
Click Commit.
(Optional) Enter a title in the Commit message box.
Select the Create a pull request for this change check box.
Click Commit.
(Optional) Enter a Branch name. Example: feature/uitext.
Click Create pull request.
1. Enter a Description.
2. Click Create.

When your pull request has been reviewed and there has been at least one successful build, providing that there are no merge conflicts, you can merge your change.

Navigate to the pull requests.
Locate your pull request and click its hyperlink.
Click Merge.

If the merge was successful, you should now delete your branch.

Navigate to the branches.
Click your branch's Actions button and select Delete branch, then click Delete.

Issue tracking

Track documentation tasks in Jira

Standalone documentation tasks should have the issue type Documentation.
Developer tasks that require documentation should have the label Documentation.
Documentation should be part of the definition of done.

Containerization

Containers are an operating system-level virtualization technology. Sun Microsystems' Solaris Zones were an early implementation in 2004. But the technology was popularized by Docker containers, release in 2013, and Kubernetes container management, released the following year.

You don't need containerization if you're using a hosting solution such as Netlify. However, if you are hosting your own site, then you should at least consider using containers.

Install Docker

If you deploy your dev portal in a Docker container, you can try it locally before you build your automation tool chain with Docker Desktop.

After installation, you can run Docker images locally. Example: docker run -d --name rabbitmq -p 15672:15672 -p 5672:5672 bitnami/rabbitmq:latest

Create a Dockerfile for Hugo

The easiest way to deploy the Hugo static site is in a Docker image, as defined by a Dockerfile:

FROM defines the base image (in the example Alpine, a lightweight Linux distribution).
COPY copies files and folders to the Docker image.
ARG specifies arguments for the Docker build command.
RUN executes commands.
EXPOSE informs a user about the ports used.
CMD specifies the component and its arguments to be used by the image.

Start Docker on login (macOS)

Some APIs may require middleware, for example a local RabbitMQ instance. It can be convenient to have Docker start the middleware when you log in. For example, on macOS:

Open the Automator application.
Select the Application type and click Choose.
In the Actions menu, select Utilities > Run Shell Script.
In the Run Shell Script section, select /bin/bash from the Shell menu.
In the box, enter:

cd /usr/local/bin

while (! ./docker stats --no-stream ); do

sleep 10

done

./docker start rabbitmq
From the File menu, click Save.
Navigate to the Applications folder (or your user Applications folder).
Enter StartRabbit in the Save As box and click Save.
Open System Preferences and click Users & Groups.
1. Select your user and click Login Items.
2. Click Add (+).
3. Navigate to the location where you saved the StartRabbit application and select it.
4. Click Add.
5. Close System Preferences.

The script will run the next time you log in. The path has to be changed because /usr/local/bin is not part of the path for startup scripts. The script waits for the Docker daemon to start before starting the RabbitMQ container. It does this by querying Docker until it gets a response. A spinning cog is displayed in the right menu while the script is running.

If you are deploying more than one Docker instance, that's the perfect time to start using Kubernetes.

Automation

You should automate your deployments so that when your code changes, your API docs are updated. When software is developed using continuous integration and deployment, it helps if the docs are deployed that way too. A popular open source solutions for this is Jenkins.

Part II: Tools

When I wrote the original version of this guide, I was an API writer creating docs for an event-driven e-commerce solution. Event driven architecture (EDA) is something I'll talk more about in a future post. Development was done primarily in .NET and API docs were generated using Swashbuckle (also a good topic for a future post). This is known as the code first or bottom up approach. The more your code grows, the harder it is to maintain consistent APIs with this approach.

In my view, you should create your API specification and code against that. But for the purposes of this guide, I'll assume that you don't have any say in that, and you just have to get the API docs into the hands of developers. You may not even have an API schema, in which case you should create one in Postman, based on whatever API documentation you have available to you.

IDE

Visual Studio Code

If you're using windows and writing API docs directly in the code, you may be using Visual Studio. If you are not using Windows, then the next best thing is Visual Studio Code.

Before you set up your dev environment, ensure you have installed the CLI tools.

Set up a .NET dev environment

Download and install the Visual Studio Code editor.
Download and install the .NET Core SDK.
Create a folder called development where you want to store your local repositories.
In BitBucket Server, go to the repository you want to work with and click the clone icon from the left toolbar.
Copy the HTTP address.
In the PowerShell / Terminal, navigate to the development folder.
Enter git clone and paste the HTTP address to complete the line.

Build the OpenAPI 2.0 (Swagger) JSON file

This assumes you have a working .NET dev environment and have correctly configured Swashbuckle:

In PowerShell / Terminal, navigate to the src folder in your local repository.
Enter dotnet build.
Navigate to the folder where the DLL was built and enter dotnet run.
In Chrome, navigate to the URL shown in the build output. Example: http://127.0.0.1:5000.
Click the /swagger/v1/swagger.json link to download the JSON file.

Plug-ins

Some recommended plug-ins for Visual Studio Code include:

• Better TOML by bungcip

• C/C++ by Microsoft

• C# by Microsoft

• Code Spell Checker by Street Side Software

• GitLens by Eric Amodio

• Markdown Preview Enhanced by Yiyi Wang

• Prettier - Code formatter by Esben Petersen

Preview API doc changes

Requirements

• You can create tasks to automate processes in Visual Studio Code.

• Each repository requires its own set of tasks.

• You must add .vscode/ to the .gitignore file to any repository you intend to use tasks with.

• Separate scripts are required for macOS and Windows.

Convert Swagger JSON to ReDoc HTML

Enter: redoc-cli bundle <filename>.json.

Running scripts

• After you have configured the scripts for a given repository, you can run them from Visual Studio Code.

• Press command+shift+B (macOS) or ctrl+shift+B (Windows) to run the build task.

• The ReDoc HTML file is opened in Chrome.

• If the server takes a long time to start, the ReDoc task will fail. After the server has started, run the ReDoc task from the Terminal menu.

Example Visual Studio Code tasks JSON file

This file should be placed inside the .vscode folder in the Git repository and .vscode added to the .gitignore file.

tasks.json

    {
      "version": "2.0.0",
      "tasks": [
        {
          "label": "build",
          "command": "dotnet",
          "type": "process",
          "args": [
            "build",
            "${workspaceFolder}/src/Dev.Example.Com.Myproject/Devg.Example.Com.Myproject.csproj"
            ],
            "problemMatcher": "$msCompile"
          },
          {
            "label": "Swagger",
            "type": "shell",
            "command": ".vscode/swagger.sh",
            "windows": {
            "command": ".vscode\\swagger.cmd"
          },
          "presentation": {
            "reveal": "always",
            "panel": "new"
          }
        },
        {
          "label": "ReDoc",
          "type": "shell",
          "command": ".vscode/redoc.sh",
          "windows": {
            "command": ".vscode\\redoc.cmd"
          },
          "presentation": {
            "reveal": "always",
            "panel": "new"
          },
         "problemMatcher": []
       },
       {
          "label": "Build API Docs",
          "dependsOn": ["Swagger", "ReDoc"],
          "group": {
            "kind": "build",
            "isDefault": true
          },
        }
      ]
    }

Example scripts (macOS)

redoc.sh

    cd ~/
    rm redoc.jsonrm redoc-static.html

    # wait for server to start
    echo waiting for service to start
    sleep 10

    curl -o redoc.json http://localhost:62512/swagger/v1/swagger.json
    redoc-cli bundle redoc.json
    open -a "Google Chrome" redoc-static.html

swagger.sh

    export CONSUL=[http://consul.dev.example.com](http://consul.dev.example.com "http://consul.dev.example.com")
    cd src
    dotnet build --source [http://dev.example.com/myproject/](http://dev.example.com/myproject/ "http://dev.example.com/myproject/")
    cd Dev.Example.Com.Myproject
    dotnet run --urls=http://localhost:62512

Example scripts (Windows)

redoc.cmd

    cd ~/
    del redoc.json
    del redoc-static.html

    # wait for server to start
    echo waiting for service to start
    sleep 10

    curl -o redoc.json

swagger.cmd

    set CONSUL=http://consul.dev.example.com
    cd src
    dotnet build --source http://dev.example.com/myproject/
    cd Dev.Example.Com.Myproject
    dotnet run --urls=http://localhost:62512

Part III: Writing

Most of this relates specifically to writing in-line docs directly in C# code using Swashbuckle. However, if you are using a different approach you can skip ahead to the section on Markdown.

Swashbuckle

Autogenerated Swagger / OpenAPI 3.0

Swashbuckle is a framework for ASP.NET Core that converts code comments into Swagger or OpenAPI 3.0 docs. If your developers write in C# and they're not already using a top-down API methodology, it's well worth a look.

Document API methods with multiple domain models

Some APIs use domain models to perform more than one task using a single API method.

Swagger shows the various domain models as different options in the Request Body Schema. However, it always shows the payload for the first domain model, regardless of the domain model selected.

For these API methods, the schema should be documented in a separate HTML page with a relative link, served by the static site generator.

Documentation for these API methods is declared in a private class as follows:

    postPath.Post.Summary = "Short name";
    postPath.Post.Description = "Long description";

The string must be contained on a single line. If you need to insert a carriage return you must use the HTML <br/> tag.

The long description should contain the following text:

The request body schema shown applies only to the first listed command.

This should be followed by a relative link to a markdown page describing the various commands.

Create API docs in VS Code

With Swashbuckle, API documentation is created as comments in the C♯ source code (.cs files). These comments are automatically converted to Swagger / OpenAPI 3.0 docs when the application is built. Most content can be written in markdown.

Tag API docs in the code

To make it clear in the code that the comments will be public facing, you can use XML comment format tags around the other comments:

    /// <!--apiddocs-->
    /// ...
    /// <!--/apiddocs-->

API group descriptions

To enable a summary for a group, the Startups.cs file must include the following:

    if (File.Exists(xmlPath))
    {
        c.IncludeXmlComments(xmlPath, true);
    }

API methods

Typically, methods are associated with a particular API. However, some methods are inherited and must be edited separately. Documentation can be added using these tags:

summary (plain text): A short description of the method. If no summary is provided, the API name is used.
remarks: A description of the method.
param: A short description of the input parameters.
response: A short description of the response code.

Example:

    /// <!--apidocs-->
    /// <summary>
    /// Find product by SKU.
    /// </summary>
    /// <remarks>
    /// Find a product where the \`sku\` is known.
    /// </remarks>
    /// <param name="sku">Stock keeping unit.</param>
    /// <response code="400">Bad request.</response>
    /// <!--/apidocs-->
    [SwaggerOperation(Taqs = new[] { "Products" })]
    [HttpGet(Name = GetProductBySkuRoute)]
    ...

Response parameters

Typically, responses for a given API are grouped in a single file. Documentation can be added using summary and example tags. You can find parameters in a project by searching files for [JsonProperty. The example tags are always converted to a single string so they should only be used with the following types:

enum (example: DateTimeOffset)
GUID (globally unique identifier)
string

Where you need to provide an example for other types, add a carriage return in the last line of the summary, followed by Example: . Don't use example tags for arrays. If you use an array example in the summary, you must escape the first square brace (\[, ]).

Example:

    /// <!--apidocs-->
    /// <summary>
    /// Stock keeping unit.
    /// </summary>
    /// <example>
    /// 6502_RICE_1KG
    /// </example>
    /// <!--/apidocs-->
    [JsonProperty{required = Required.Always}]
    public string Sku { get; set; }

Required parameters

When parameters are required, you can mark them with [Required], but the .cs file will need a using System.ComponentModel.DataAnnotations; declaration at the beginning of the file.:

If there is already a [ JsonProperty(Required = Required.DisallowNull)] then do not use [Required].

Example:

    ...
    /// <!--/apidocs-->
    [Required]
    public bool AddProductToBasket { get; set; }

Commenting out comments

If you need to prevent the comments being converted to XML so that you can see what the method name is in the ReDoc output, you can use XML comment syntax:

    /// <!-- <summary>
    /// Stock keeping unit.
    /// </summary> -->

In XML, certain characters must be escaped or the XML will fail to build from the comments (without warnings). For example, ampersands (&) and angle brackets, (<, >). If you need to use an ampersand with code font style, you must use HTML <code> tags. If you use the markdown backtick (\`) the escaped ampersand will not be converted to a single character.

Markdown

All tagged comments can contain plain text. Some can also contain markdown. Any that can contain markdown can also contain HTML. However, the way markdown and HTML are rendered will vary depending on the tag.

Mermaid

Liquid didn't seem to like this section. Here's my dev.to article on Mermaid: https://dev.to/aowen/creating-diagrams-with-mermaid-1lbj

Troubleshooting

Create a PDF from OpenAPI JSON

On occasion you may be asked to produce a version of your dev portal as a PDF file. Distilling an interactive website into a PDF is impossible. Most browsers will not even let you print the site in its entirety. However, you can use Swagger2PDF to generate a simple PDF from an OpenAPI JSON file:

swagger2pdf -s swagger.json

Ensure new features and changes are documented

Doc teams typically cover the work of multiple development teams, creating developer and user docs. It is not always possible to attend every scrum meeting and sprint review.

To ensure developer and user docs are delivered as close as possible to the sprint in which features are delivered, it is essential that the team receives timely notification.

There are three ways to do this in Jira:

Create a Documentation sub task on a ticket.
Create a Documentation task and associate it with a ticket.
Add the Documentation label to a ticket.

You should configure a doc team kanban board to display tickets that meet any of these criteria. You can then watch a ticket to track its progress. Let your developers know how to create doc tickets:

You can request new documentation for existing features by creating a Documentation task.
The doc team will prioritize documentation requests to best meet customer needs.
Requests for documentation that are made other than using Jira will be given the lowest priority.

Fix a broken API

Things to try:

Does the start of the .cs file have the following:

using Swashbuckle.AspNetCore.SwaggerGen;
Are there incorrectly declared types?

Type = "integer" / Type = "boolean"
If the comments are absent from the JSON is the DocumentationFile property set in the .csproj file (should be set for debug and release). Example:

bin\Debug\netcoreapp2.2\Your.ApiGateway.xml

Convert OpenAPI YAML to JSON

ReDoc requires OpenAPI 3.0 / Swagger 2.0 source in JSON format to create a static HTML page. Typically, auto-generated content, such as that produced by Swashbuckle, is already in JSON format. However, hand edited API schema are typically created in YAML.

Navigate to https://editor.swagger.io/. Alternatively, download the Swagger Editor to use keep the content inside the corporate network.
Select File > Import File.
Fix any critical schema errors.
Select File > Convert and save as JSON.

Convert Swagger 2.0 to OpenAPI 3.0 to resolve schema errors

ReDoc requires schema in OpenAPI 3.0 format. If you provide a Swagger 2.0 JSON file, it will attempt to do the conversion itself, but schema errors may cause the conversion to fail. Ideally you should fix the schema errors. However, if you require a quick approximation of how the API will appear you can try using the Mermade online converter.

Force iFrame link to open in parent window

ReDoc produces a static HTML page. The easiest way to add it to a Hugo site while keeping the navigation in the header is to use an iFrame. A better approach would be to ingest the page and dynamically recreate the whole thing. But that would require work by an experienced Hugo developer.

The quick and dirty approach is:

Add <base target="_parent"> in the <head> section of the ReDoc HTML file.
Put the HTML pages in Hugo's static folder.
Create a markdown page in Hugo's content folder:

    ---
    ---

Prevent API method links going to the wrong place

In the API docs, clicking a method link in the left pane in ReDoc should take the user to the appropriate method. However, when developers reuse Operation IDs, the link will instead take them to the first instance of that Operation ID. Developers must either use Operation IDs that are unique across the entire API gateway, or use no Operation IDs at all.

Remove .DS_Store files from a Git repository

In the shell, navigate to the root of the repository.
Enter find . -name .DS_Store -print0 | xargs -0 git rm -f --ignore-unmatch .
Commit changes.

Install Java 8

Oracle changed the license for JDK 8, effective 16 April 2019. You can still download it, but if in doubt, you can use penJDK 8 instead (https://www.azul.com/downloads/zulu/).

Run Windows apps on Intel Macs

Because API doc tools make heavy reliance on UNIX-land command line tools, it may be preferable to use a Mac. However, traditional technical writing tools may be Windows-only, for example MadCap Flare. The easiest way to run Windows programs on an Intel Mac is with Parallels. However, Intel Macs are going away. I'll cover the options for running Windows programs on an M1 Mac in a future post.

Run a local Rabbit MQ

Some API gateways have RabbitMQ as a build dependency. Because the remote server is not always available, you may need to run a local instance of RabbitMQ. You can use Docker to do this from the command prompt:

docker run -d --name rabbitmq -p 15672:15672 -p 5672:5672 bitnami/ rabbitmq:latest

Navigate to http://localhost:15672/ and create a user called rabbituser with the password rabbituser as an administrator.
Click the rabbituser user and in the Virtual Host section, then click Set permission.
In the source code, locate the appsettings.Development.json file and change all instances of guest to rabbituser .

To restart the service:

docker start rabbitmq

To remove an old Rabbit MQ:

docker system prune -a

As a final thought, you really need a writing style guide. If you don't have one you should create one. I'll cover that topic in a future post.

Part I: Backend

Static Site Generators

Hugo

Install CLI tools (macOS)

Install CLI tools (Windows)

Requirements

Instructions

Test a local copy of the dev portal

Add Google Analytics to a static HTML page

Source control

Git

Get the URL from Bitbucket

Clone the repository and create a local branch

Publish changes

Create a pull request

Resolve conflicts

Merge changes

Delete branch

Edit text in Bitbucket

Issue tracking

Track documentation tasks in Jira

Containerization

Install Docker

Create a Dockerfile for Hugo

Start Docker on login (macOS)

Automation

Part II: Tools

IDE

Visual Studio Code

Set up a .NET dev environment

Build the OpenAPI 2.0 (Swagger) JSON file

Plug-ins

Preview API doc changes

Requirements

Convert Swagger JSON to ReDoc HTML

Running scripts

Example Visual Studio Code tasks JSON file

tasks.json

Example scripts (macOS)

redoc.sh

swagger.sh

Example scripts (Windows)

redoc.cmd

swagger.cmd

Part III: Writing

Swashbuckle

Autogenerated Swagger / OpenAPI 3.0

Document API methods with multiple domain models

Create API docs in VS Code

Tag API docs in the code

API group descriptions

API methods

Example:

Response parameters

Example:

Required parameters

Example:

Commenting out comments

Markdown

Mermaid

Troubleshooting

Create a PDF from OpenAPI JSON

Ensure new features and changes are documented

Fix a broken API

Things to try:

Convert OpenAPI YAML to JSON

Convert Swagger 2.0 to OpenAPI 3.0 to resolve schema errors

Force iFrame link to open in parent window

Prevent API method links going to the wrong place

Remove .DS_Store files from a Git repository

Install Java 8

Run Windows apps on Intel Macs

Run a local Rabbit MQ

Read next

Explore use of agents to comment you code

🌿 Why RabbitMQ is Faster than Apache Kafka and When to Use Each

Rspec

Exploring the Components of Blockchain Systems