Very Nordic Problem: In search of great skiing track

Finland is the happiest country on Earth. It has endless forests and lakes. During wintertime, most of the territory has snow. People are using this combo of cold conditions and excellent services. Cross-country skiing is becoming very popular. However, I noticed that the municipality should prepare the forest tracks properly using heavy machinery.

As a ski track user, I would like to be notified when the tracks are fresh and cleaned. If there were maintenance in the morning, the ice on the trails would be an obstacle for comfortable skating-style skiing. In theory, I would like to avoid it.

Map of the tracks called Outdoor Exercise Map shows the tracks' state. If I want to get the latest data about maintenance, there is the only way - go to the website and check. My motivation behind making the Ski Track Notifier is to stop doing these repetitive tasks. I decided to solve the issue with my favorite technologies: AWS, CDK, Serverless. All the code examples in this article focus on Typescript. The project code is on GitHub.

Architecture

The idea behind the application is to use as many managed services as possible. It will mean that the price will be only for the invocations. AWS Lambda is the primary business logic executor. The app calls it periodically to check the Helsinki region Service Map API.

The flow is going as follows. The EventBridge rule calls AWS Lambda every 30 minutes. The function checks the API and filters out the ski tracks which were not ready inside the time interval of 60 minutes. There is an SNS topic to gather the PublishCommands from Lambda. It has an e-mail subscription as a notification target. Here is the architecture of the solution.

Tooling

I decided to make development and deployment as easy as possible. It is a small project which should be free for hosting and fast to put into production. This section will speak about the technologies used for the application. I would start from the cloud resources to the actual business logic.

Cloud Resources

Infrastructure as Code and CI/CD pipeline allow me to make my application available in minutes from the 'master' branch. I used CDK with CDK Pipelines for provisioning resources. I talked about the CI/CD process more in this article.

Because I am working alone on this project, there is no need for a multi-branch setup. There are two CloudFormation stacks in the project. One is responsible for CI/CD pipeline made with the CodePipeline service. Another stack is for implementation. It consists of Lambda, SNS, permissions, and EventBridge Trigger Rule in a nutshell.

Project Structure

I used the monorepo approach, where the implementation and the infrastructure instructions live in the same codebase. Beforehand there were two issues with this approach:
Two 'package.json' files in the source folder and the 'infra' one for dependencies created a dependency mess. It means that there will be two package-lock and two folders with node_modules.

Packaging lambda code dependencies meant having an extra step of the building and packing it. One could do it via lambda layer or external bundler like webpack.

Thanks to CDK team, now there is a construct called NodejsFunction. It solves precisely these problems by simplifying the build process and code structure. According to the creators, it uses an esbuild that should be 'extremely fast'. I tried it, and it blew my mind with simplicity. My function resource looks like this with just an entry point to generate a valid bundled js file with the tree-shaking. This approach will pack exactly this function without having a whole folder in the cloud.

const getSkiTrackStateHandler = new aws_lambda_nodejs.NodejsFunction(this, 'MyFunction', {
      entry: path.join(__dirname, '../../src/functions/get-ski-track-state.ts'),
    });

One will run only four commands to complete building a valid CloudFormation template. The instructions generated from this array:

const buildCommands = [
 'npm i', 
 'npm run build', 
 'npm run test', 
 'npx cdk synth'
];

Business Logic

I mainly used the TDD (Test-Driven Development) approach to develop the implementation. My idea was to decouple the API call made with Axios GET request from the actual filtering. Keeping this in mind helps to test it faster. I could create a fixture with the sample ski tracks. For the tests, I used jest.

The API returns a JSON object. Every ski track has a key called observations. It is an array of objects that should have property each.

      observations: [
        {
          unit: 54322,
          id: 545294,
          property: 'ski_trail_condition',
          time: '2022-01-07T23:18:48.129560+0200',
          expiration_time: null,
          name: {
            fi: 'Hyvä',
            sv: 'Bra',
            en: 'Good condition',
          },
          quality: 'good',
          value: 'good',
          primary: true,
        },
        {
          unit: 54322,
          id: 545293,
          property: 'ski_trail_maintenance',
          time: '2022-01-07T23:18:48.129560+0200',
          expiration_time: null,
          name: {
            fi: 'Kunnostettu',
            sv: 'Iståndsatt',
            en: 'Maintenance finished',
          },
          quality: 'unknown',
          value: 'maintenance_finished',
          primary: false,
        }
]

I need to have ski_trail_condition with the good value. Also, there is ski_trail_maintenance. It should have value with the maintenance_finished. One more condition is to have it in the last hour. For the time comparisons, there is a library called luxon.

Example

Here is the screenshot of the letter sent to the e-mail. It has the track's name, parsed date, and address. Surprisingly, maintenance data comes with the delay to the API. That is why it is essential to have this ratio between running each lambda and checks of the track update age. After couple of experiments it was like 30/60 minutes.

Future work

Of course, there are more insights about the tracks in the API. For example, 'valaistu' means the availability of artificial light on the tracks. Also, there is a distance shown in kilometers.

Now the application supports only one subscriber. In the future, it would be great to have multiple subscribers, the ability to choose municipality, and track choice of the main focus. Also, it would be nice to connect something like an Amazon Lex-powered bot for multiple channels.

One other idea is to send notifications in batches. The current implementation has the drawback of sending multiple e-mails if there are recent maintained tracks. Today, there is no separate check that the notification about a particular object has been sent to the subscriber.

It took a couple of evenings to create a small full-fetched application to notify myself about the changes found in the public API. With the power of serverless and event-driven architecture, one could deliver business value fast without worrying about servers, virtual machines, or containers. Thanks for reading! Follow me on twitter:

Igor Soroka

@grenguar

After watching cloud educational content on 2x speed, I noticed that every instructor sounds sad at normal playback.

10:21 AM - 17 Jan 2022