Getting started with Spring AI

I've been trying to get started on this sample ever since they announced SpringAI a few weeks ago and to be honest I wasn't sure what to expect. But having gotten my hands dirty with this first PoC I have to say I'm impressed.

Before we begin, the context of this post is about integrating AI services (OpenAI, Gemini, HuggingFace etc) into an existing application.

This is not about training models, nor is it about creating capabilities that generate ML offerings, purely integrating with the existing offerings out there.

With that, I'll begin...

AI over the years

I've been watching AI for the past 20 odd years and of course it comes in waves of new achievements but for some reason it never really became mainstream until recently. We now see this tidal wave of offerings that offer services from chat, image generation, Transcription, even pair programming so I will say we are seeing a more viable partnership with AI in the way we write and design applications.

Just about every cloud provider and startup is now creating capabilities that enable people to tailor how to incorporate AI into their own products to create a new approach to how you interact with their services.

Spring AI

Spring AI was announced a few weeks ago, but what exactly is it?

Spring AI is the framework that allows you to easily select an AI provider and seamlessly integrate it with your existing application stack.

Spring AI takes Spring Boot and incorporates the right interfaces to ensure you don't have to rewrite code each time you decide to switch AI providers.

Let's get started!

The Spring AI documentation has a brief example of how to get started, so we will make use of that and explore beyond what it offers.

Bootstrap Spring

As per the article, we will use Spring CLI (NOT TO BE CONFUSED WITH SPRING BOOT CLI!!!!) to get up and running.

$ spring boot new --from ai --name myai
Getting project from https://github.com/rd-1-2022/ai-openai-helloworld
Created project in directory 'myai'

The first you notice about the application is that it is a typical Spring Boot application. It comes with your typical Application.java entry point and also includes a RestController SimpleAiController.java

SimpleAiController.java

What I like about Spring AI is its simplicity. Let's take a quick look at the RestController it creates.

package org.springframework.ai.openai.samples.helloworld.simple;

import org.springframework.ai.chat.ChatClient;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;

import java.util.Map;

@RestController
public class SimpleAiController {

    private final ChatClient chatClient;

    @Autowired
    public SimpleAiController(ChatClient chatClient) {
        this.chatClient = chatClient;
    }

    @GetMapping("/ai/simple")
    public Map<String, String> completion(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
        return Map.of("generation", chatClient.call(message));
    }
}

As you can see, the most obvious aspect is the ChatClient. This is the interface that makes the calls to the provider, returning the results from your prompt.

If you look at the pom.xml you'll notice the only change from a standard SpringBoot application is the AI integration:

<dependency>
    <groupId>org.springframework.ai</groupId>
    <artifactId>spring-ai-openai-spring-boot-starter</artifactId>
</dependency>

This is the implementation specific classes that perform the integration with the OpenAI platform.

In order to use this API there are still a couple of steps required.

Up and Running

First you need to create an OpenAI account and add a balance.

Once you've signed up, navigate to Settings -> Billing -> Payment Methods and add a method of payment.

Next, go back to Settings -> Billing and add a small value, $5 is usually enough to start playing around with the LLMs.

Lastly, we are going to create an API key to allow our Spring AI application to execute requests. Navigate to API Keys and select + Create new secret key

In the dialog that is presented, give your key a name, select All permissions and click the Create secret key button.

You will then be presented with your secret key which you will use for your Spring AI application - copy it and put it somewhere safe.

Configuring our Chat API

In the application.properties file we will add 2 properties:

spring.ai.openai.api-key=<secret api key>
spring.ai.openai.chat.options.model=gpt-3.5-turbo

This is where we enable our application to:

1. perform requests against our OpenAI account
2. Make use of the gpt-3.5-turbo chat model

OpenAI offers a number of different models for various purposes, this is one of the flexible features around Spring Ai that makes it easy to integrate AI into your applications.

The OpenAI Model documentation lists the models that are available for use.

A list of the model prices is also available here

With these settings in place, you can start your application and begin sending prompts!

$ ./mvnw spring-boot:run
  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::                (v3.2.3)

2024-03-29T14:04:33.137-07:00  INFO 46846 --- [           main] o.s.a.o.samples.helloworld.Application   : Starting Application using Java 21.0.2 with PID 46846 (/Users/anthonyikeda/work/git/myai/target/classes started by anthonyikeda in /Users/anthonyikeda/work/git/myai)
2024-03-29T14:04:33.139-07:00  INFO 46846 --- [           main] o.s.a.o.samples.helloworld.Application   : No active profile set, falling back to 1 default profile: "default"
2024-03-29T14:04:34.316-07:00  INFO 46846 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat initialized with port 8080 (http)
2024-03-29T14:04:34.328-07:00  INFO 46846 --- [           main] o.apache.catalina.core.StandardService   : Starting service [Tomcat]
2024-03-29T14:04:34.328-07:00  INFO 46846 --- [           main] o.apache.catalina.core.StandardEngine    : Starting Servlet engine: [Apache Tomcat/10.1.19]
2024-03-29T14:04:34.521-07:00  INFO 46846 --- [           main] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring embedded WebApplicationContext
2024-03-29T14:04:34.522-07:00  INFO 46846 --- [           main] w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 1344 ms
2024-03-29T14:04:35.260-07:00  INFO 46846 --- [           main] o.s.b.a.e.web.EndpointLinksResolver      : Exposing 1 endpoint(s) beneath base path '/actuator'
2024-03-29T14:04:35.360-07:00  INFO 46846 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat started on port 8080 (http) with context path ''
2024-03-29T14:04:35.373-07:00  INFO 46846 --- [           main] o.s.a.o.samples.helloworld.Application   : Started Application in 2.587 seconds (process running for 3.021)

Just like any Spring Boot application you can see the application started on port 8080.

Running your favorite HTTP Client, let's send some prompts.

GET http://localhost:8080/ai/simple?message=Who%20was%20first%20to%20live%20in%20Pangea
Accept: application/json

The first organisms to live on Pangea were likely single-celled bacteria and algae that existed around 300 million years ago. As Pangea was a supercontinent that formed during the late Paleozoic era, it provided a unique environment for early life forms to evolve and thrive.

Success!

I agree, what we have created is a proxy or gateway to these services, but with minimal modifications, getting started too less than 30 minutes!