Over the last year I have been given the chance to work with some amazing people and we all have been developing micro-services that expose RESTful APIs.

Through out all this year we had to integrate with other APIs, such as RESTful, NVP - Name-Value Pair, etc, and both internal or external 3rd parties.

Making sure the project runs without any major incident is hard work. It involves a lot of people from a lot of departments and their ability to share information is the most vital part before developing the API.

In no way am I stating that these are the right approaches to take when designing an API. I am just sharing what I have learned the hard way.

So here are some points I find important to keep in mind when given a new task, aka, build a new micro-service.

• Understand the API dependencies;
  • a dependency can be an internal API, external API or an AWS service you need to integrate with, etc...
• Understand if those dependencies provide all the functionality the API requires (both tech and business);
  • does it fit the purpose;
  • how well it is documented/supported;
  • is it too strict;
  • how flexible regarding unforeseen/future changes.
• Understand what the API MUST deliver;
  • current deliver;
  • future deliveries (although in an agile environment this is a fuzzy topic).
• An API contract;
  • a contract is an agreement between two or more parties that develop/consume the API.
• API documentation;
• API testing.

All above topics can be grouped into 3 main categories:

1. API dependencies
2. API specification
3. API development

API dependencies

As I stated above, this is where typically you will have the business and technical requirements and you or your team start brainstorming/spiking on the best course of action.

This means, understanding what are the API dependencies and their added value.

From personal experience, you always miss some edge cases so it is a good practice to use UML sequence diagrams to help you structure how your endpoints will behave, such as request headers, payloads, responses, etc.

What is an UML sequence diagram?

An UML sequence diagram describes how operations are carried out in an interactive format. They are time based and they show the order of those interactions. They also specify all the participants in the workflow.

A visual interpretation helps you keep track of the workflow and define happy paths as well as errors from any 3rd party API and how your own API should deal with that information.

How can we as a developer take advantage of sequence diagrams? By using tools such as PlantUML or mermaidJS which allows us to generate diagrams from textual representations.

A simple example with PlantUML (taken from the official website):

@startuml
Alice -> Bob: Authentication Request
Bob --> Alice: Authentication Response

Alice -> Bob: Another authentication Request
Alice <-- Bob: Another authentication Response
@enduml

which will generate the next image:

This is a cool feature because it can be control versioned.

I find that mermaidJS is still a little behind of PlantUML in terms of integrations and functionalities but they are both powerful tools and I have used both in different contexts. You should use the one that best fits your needs.

If you use Confluence, there is a nice plugin for PlantUML.

API specification

After you have defined the diagrams, the next step is to start drafting the contract.

This contract should be "signed off" in a standard specification that most developers can be familiarised with. Luckily the OpenAPI Specification has been here for a while.

The specification is written in yaml and it can also be control versioned.

Once again and from personal experience, the drafted contract may suffer small to medium changes. Which is normal, it's a contract where more than one team is involved and feedback is always a good thing.

Always be open to suggestions but don't forget that your team owns the API.

Discussion is healthy and allows us to see different angles to achieve the same goal.

Keep in mind that your API may also suffer changes in the future which may impact production environments. So think wisely on versioning your API being that through path versioning such as /v1, etc. Or by headers such as GitHub's example Accept: application/vnd.github.v3+json.

If you are like me and your OCD kicks in when the topic "API versioning" comes to the table then read this interesting post about Evolvable APIs from Fagner Brack - To Create An Evolvable API, Stop Thinking About URLs.

API development

It's time to take all the value we gained before to start implementing it into code. Just make sure to follow the contract and protect the micro-service against unexpected 5xx popping into production.

But depending on the language you choose to code, a big part of the development is testing - unit, functional, etc...

With the right tools you can prepare functional test scenarios by using Postman or Insomnia.

Postman has a neat feature which is called Newman where you can run a collection against a file to check if your endpoints follow the contract.

At this point I had shared tools that can be version controlled along with the current code. Making it easy to keep all of them synced.

Demoing with an example

Nothing is better than an "almost real" example to demonstrate everything described above.

This example is based on making capture, void and refund transactions, given an authorization identifier.

an authorization identifier means that we locked some amount from the payment method used by a customer.

Fictional requirements

• capture the authorization;
  • charge the account an amount lower or equal than the locked funds in a specified currency;
  • returns a transaction identifier for possible refund.
• void the authorization;
  • release the locked funds.
• refund the account;
  • providing a valid transaction identifier;
  • returns a refund transaction identifier.
• all above actions MUST be validated against another fictional internal API;
  • validate that accountId is linked to the authorizationId.
• all above actions MUST have a required X-Api-Key header;
  • for security reasons.
• all above actions SHOULD have an X-Correlation-Id header.
  • for keeping track of workflows.

Let's name the micro-service as process-transactions.

Planning with sequence diagrams

From the previous fictional requirements we can define 3 participants:

• USER - The API consumers;
• MS - The API micro-service;
• API - The API consumed.

A draft of the diagram should resemble as the following image:

Tooling for PlantUML

Consider the following file structure.

./images
./plantuml
├── capture.puml

Where capture.puml has the following content.

@startuml

participant "USER" as A
participant "MS" as B
participant "API" as C

title //process-transactions// micro-service capture workflow

rnote left A
**headers**
  X-Api-Key<font color="red">*</font> //<string>//
  X-Correlation-Id //<string>//
end note

activate A
A -> B: **POST** ""/capture/:authorizationId""

rnote left A
**payload**
  accountId:<font color="red">*</font> //<string>//
  amount:<font color="red">*</font> //<number>//
  currency:<font color="red">*</font> //<string>//
end note

rnote left B
**headers**
  X-Api-Key<font color="red">*</font> //<string>//
  X-Correlation-Id //<string>//
end note

activate B
B -> C: **POST** ""/validate""

rnote left B
**payload**
  accountId:<font color="red">*</font> //<string>//
  authorizationId:<font color="red">*</font> //<string>//
end note

alt success request

rnote right B
**headers**
  X-Correlation-Id //<string>//
end note

activate C
B <-- C: ""**200** OK""

rnote right B
**payload**
  success: //true//
end note

|||

B -> B: capture amount
activate B
deactivate B

rnote right A
**headers**
  X-Correlation-Id //<string>//
end note

A <-- B: ""**200** OK""

rnote right A
**payload**
  transactionId: //<string>//
end note

|||

else failure request

rnote right B
**headers**
  X-Correlation-Id //<string>//
end note

B <-- C: ""**200** OK""
deactivate C

rnote right B
**payload**
  success: //false//
end note

rnote right A
**headers**
  X-Correlation-Id //<string>//
end note

A <-- B: ""**422** UNPROCESSABLE ENTITY""
deactivate B

rnote right A
**payload**
  error: //true//
  reason: Conditions could not be met
end note

|||

end
deactivate A

@enduml

We can use the package node-plantuml for generating the sequence diagram as an image.

• npm install node-plantuml
• puml generate -s -o ./images/capture.svg ./plantuml/capture.puml

Now we have a version controlled file that describes our /capture endpoint.

Writing the contract in the OpenAPI Specification

PlantUML gives us a pretty good view of what the capture endpoint expects as a request and responses.

Remember that at this point the micro-service logic is still a black-box, and it should remain that way for now.

We are trying to achieve a contract that does what business is expecting.

It's also expected that all the dependencies of the micro-service regarding 3rd/internal parties APIs are clear on their purposes and which of their endpoints suits our needs.

In our capture endpoint we assume some generic response. But we could be calling x number of endpoints if needed before the capture replies with anything.

Anyways, the OpenAPI is defined as a yaml file with all the specifications.

But if we have a few endpoints and a lot of responses, it might be useful to have separate files for each section of the Specification.

Ultimately this will ease the burden of maintaining the specification.

Organising the contract structure

Updating the above file structure.

./images
├── capture.svg
./plantuml
├── capture.puml
./open-api
├── components
│   ├── headers
│   │   └── x-correlation-id.yaml
│   ├── headers.yaml
│   ├── parameters
│   │   ├── authorization-id.yaml
│   │   └── x-correlation-id.yaml
│   ├── responses
│   │   ├── capture-200.yaml
│   │   └── capture-422.yaml
│   ├── responses.yaml
│   ├── schemas
│   │   ├── capture-200.yaml
│   │   ├── capture-422.yaml
│   │   └── capture.yaml
│   └── schemas.yaml
├── components.yaml
├── index.yaml
├── info.yaml
├── paths
│   └── capture.yaml
└── paths.yaml

Instead of using the normal '#/components/...', ref is a relative link to the file, which after the compile step will be properly OAS "reffed".

Content of ./open-api/index.yaml:

openapi: 3.0.2
tags:
  - name: capture
info:
  $ref: './info.yaml'
paths:
  $ref: './paths.yaml'
components:
  $ref: './components.yaml'
security:
  - X-Api-Key: []

Content of ./open-api/paths.yaml:

/capture/{authorizationId}:
  post:
    $ref: './paths/capture.yaml'

Content of ./open-api/paths/capture.yaml:

summary: Capture an amount
tags:
  - capture
operationId: capturePost
parameters:
  - $ref: '../components/parameters/authorization-id.yaml'
  - $ref: '../components/parameters/x-correlation-id.yaml'
requestBody:
  content:
    application/json:
      schema:
        $ref: '../components/schemas/capture.yaml'
responses:
  '200':
    $ref: '../components/responses/capture-200.yaml'
  '422':
    $ref: '../components/responses/capture-422.yaml'

Tooling for OAS

We can use the package swagger-cli for generating the compiled Specification file.

• npm install swagger-cli
• swagger-cli bundle -o open-api.yaml --type yaml open-api/index.yaml.

And the full specification:

openapi: 3.0.2
tags:
  - name: capture
info:
  version: 1.0.0
  title: Process Transactions Micro-service
  description: 'Capture, void and refund an account.'
paths:
  '/capture/{authorizationId}':
    post:
      summary: Capture an amount
      tags:
        - capture
      operationId: capturePost
      parameters:
        - name: authorizationId
          description: Authorization Id which allows to capture the locked funds
          in: path
          required: true
          schema:
            type: string
        - name: X-Correlation-Id
          description: Correlation Id to keep track of workflow
          in: header
          schema:
            type: string
      requestBody:
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/capture'
      responses:
        '200':
          $ref: '#/components/responses/capture-200'
        '422':
          $ref: '#/components/responses/capture-422'
components:
  securitySchemes:
    X-Api-Key:
      type: apiKey
      in: header
      name: X-Api-Key
  responses:
    capture-200:
      description: Capture of funds has succedeed
      headers:
        X-Correlation-Id:
          $ref: '#/components/headers/X-Correlation-Id'
      content:
        application/json:
          schema:
            $ref: '#/components/schemas/capture-200'
    capture-422:
      description: Capture did not succeed
      headers:
        X-Correlation-Id:
          $ref: '#/components/headers/X-Correlation-Id'
      content:
        application/json:
          schema:
            $ref: '#/components/schemas/capture-422'
  schemas:
    capture:
      type: object
      required:
        - accountId
        - amount
        - currency
      properties:
        accountId:
          type: string
        amount:
          type: number
          example: 9.99
        currency:
          type: string
          example: EUR
    capture-200:
      type: object
      properties:
        transactionId:
          type: string
          description: The transaction id which will allow to refund
    capture-422:
      type: object
      properties:
        success:
          type: boolean
          default: false
        reason:
          type: string
          example: Conditions could not be met
  headers:
    X-Correlation-Id:
      schema:
        type: string
security:
  - X-Api-Key: []

Testing against the API

Now that we defined diagrams and the contract is settled, we are ready to implement it.

Let's say you have a server up and running with all of the requirements in place.

Wouldn't it be better to have a Postman collection based on the OAS instead of manually creating it?

Converting OpenAPI to Postman Collection

We can use the package openapi-to-postmanv2 for generating the Postman collection.

• npm install openapi-to-postmanv2
• openapi2postmanv2 -s open-api.yaml -o postman-collection.json -p it will generate the a Postman collection almost pre-filled.

Obvious you will need to fill the blanks such as the X-Api-Key and alike.

Conclusions

Building the Postman collection through the OpenAPI Specification will help find breaches in the development.

Just keep in mind that changes to the contract often happen during development or even when all the functionality is being tested.

Hope this workflow helps in any way possible to fasten and tighten the development.

What are your thoughts? Please share your experience and all constructive feedback is welcome!

Links

Specifications

• UML Specification
• OpenAPI Specification

Tools

• PlantUML
• mermaidJS
• Postman
• Insomnia
• Newman
• Swagger Editor
• LiveUML
• node-plantuml
• swagger-cli
• openapi-to-postmanv2