Happy Path: Adding speech recognition to Vaadin apps.

The previous blog post added wake word detection for a Vaadin application using Picovoice. That is a robust in-browser approach to creating always-listening apps. You might want to use that for full speech recognition, but also the draft standard web speech API also provides a way to voice-enable applications.

There are a few ways you can integrate web APIs. This time we look at how to use web speech API for speech recognition in Vaadin by integrating a ready-made add-on from Vaadin Directory.

Choose an add-on from Vaadin Directory

Head to vaadin.com/directory and search for "speech". Pick an add-on, such as "Voice Recognition", and click the install button on the top-right to see the different ways of installing: get the necessary dependencies, download, or in this case: create a new project.

Create a Vaadin project with the add-on

Click on "Create Project" and it starts downloading a Zip file with the full Maven project setup. Extract the zip and import it into your IDE. In IntelliJ, use New --> Project from Existing Sources....

Implement Speech Recognition in Vaadin UI

After opening the Vaadin project in your IDE, locate the main View class. I edited the "HelloWorldView.java" which has some sample code in it. Head back to the add-on page and copy and paste the sample code from the add-on's documentation into the constructor of the view class. Just replace the previous sample code.

Run the Application class in debug and start the development server. Because we are adding a new add-on, this might take a while, but be patient.

Click the start button in the UI and grant permission to use the microphone. Try to say something and see how well it performs. On the first try, the browser will ask your permission to access the mic. Remember, at the time being, the speech recognition API is only available in Chrome and Safari.

Customize and add your own commands

Now you can implement your custom application-specific functions. For example, modifying the code in the event listener to show a Vaadin standard Notification:

final String command = "show notification";
voiceRecognition.addResultListener(listener -> {
    String text = listener.getSpeech();
    if (text.contains(command)) {
        Notification.show(
            text.substring(text.indexOf(command)
                          +command.length()));
     }
});

Conclusion

That was a simple way to create a voice commands for Vaadin web application. While still limited by browser support, speech recognition opens up exciting possibilities for enhancing user experience in web applications. Share your thoughts and experiences in the comments below, and stay tuned for more.