Gergely Szerovay for This is Learning

Posted on May 2 • Originally published at aiboosted.dev

Create an AI prototyping environment using Jupyter Lab IDE with Typescript, LangChain.js and Ollama for rapid AI prototyping

#typescript #ai #chatgpt

Intro

This is the 2nd part of my article series describing my journey in AI-assisted development.

In the introductory part of this series, I summarized the recent changes that enable us to prototype and develop AI apps in Typescript easily.

Haven’t read the first part? Here is what this project is all about: my goal in the coming months, and probably years, is to experiment with the new AI models & tools and show you everything I learn along the way. I'm going to “build in public” a lot and show you the important milestones of my journey. As a result, you’ll be able to apply this knowledge in your own practice.

In this article, I show you how to install the Jupyter Lab IDE and all the necessary tools for our rapid prototyping environment. You’ll only need to set up this once, and I’ll explain everything step by step. After having this setup, you’ll be able to follow along with me and build all my future experiments. (So it’s worth the effort :) ).

For example, in the next parts of the article series, I'm going to explain you in more detail how I built the "Text reviewer app":

My workflow and tools

I use the following workflow for AI tool development:

a Typescript- and Jupyter Lab-based AI tool prototyping environment for quickly prototyping AI tools
I extend the code of the prototype, create a store from it. In React apps I use the Zustand React store, in Angular I use NgRx's SignalStore, in Jupyter Lab, I use Zustand Vanilla stores. These store solutions have a lot in common, so I can reuse the same selector, effect and updater code.
I build an Angular or React app or smart component that uses the store in an Nx monorepo. I use the Shadcn UI in React apps, and the Shadcn-based spartan/ui in Angular apps, so I can use almost the same app architecture both in Angular and React.

These are the most important tools of my Typescript-based AI tool prototyping environment:

Jupyter Lab web-based interactive development environment
Deno Typescript/Javascript kernel for Jupyter notebooks, it enables us to create Jupyter Notebooks in Typescript
Ollama for running LLMs locally
Langchain, a framework for developing applications powered by language models

I'm going to also explain how to install and what are the most important commands of the following package managers:

Chocolatey Windows software management solution, we use this for installing Python and Deno
Package installer for Python (pip), we use this for installing the Python-based packages, such as Jupyter Lab, and we'll use this for installing other Python-based tools like the Chroma DB vector database

If you follow the steps of this article, you'll be able to run the prototype of the “Text reviewer app” in your local Jupyter Lab IDE (downloadable Jupyter notebook), and you’ll also be able to start creating your own app prototypes based on my examples:

Sidenote: what are the benefits of using Jupyter notebooks with Typescript for prototyping?

Initially, when I started doing my experiments, I created a simple Angular app, and I wrote all the experimental AI code in the main component's constructor, and showed the output on the console. I found that this approach worked well with simple experiments, when I have a clear plan on how I want to build something, and I don't want to experiment with many different models, prompts or agent networks. However, in case of more complex experiments, this simple approach has its limitations:

The whole experimental code runs all at once, if I would like to split it up into multiple parts and run the parts separately, I need UI controls, or I have to comment/uncomment parts of the code. In Jupyter, I can split the code into multiple cells, and run all the cells, or some of the cells separately
Package management: in Deno, this is automatic, I just write an import statement, and Deno downloads and stores the NPM or Deno package automatically in the background. There is no need for the package.json or node_modules directory. Once a specific version of a package is downloaded by Deno, we can use it from all the Jupyter notebooks
We can simply create a copy of a notebook in Jupyter. If I experiment with a new AI tool, I can just clone my notebook and change for example the prompt or the model. Then I can run my notebooks in parallel, and compare the results side-by-side
In Jupyter, if I need customized agents with RAG (Retrieval-Augmented Generation, we use this technique for optimizing the output of an LLM by extending its knowledge by additional information external to its training source), I can directly access the local files containing the additional knowledge. We can also do this in an Angular / React app, but it's more challenging, as we need some kind of a server to access the local files.
The Typescript code we create with Jupyter/Deno is portable: we can use it in a frontend project, but it's also possible to use the code in the backend.I believe that for experimenting purposes, Jupyter and Deno are perfect, while in the prototyping phase, we should create an Angular or React app. These prototypes can effectively support a single user's or smaller team's work. However, when our goal is to turn the prototype into a real service, we need to move at least some parts of the code to the backend in order to keep the resources and costs under control (e.g. API access and fees).
And finally, the biggest benefit of using Jupyter notebooks in my opinion: it is like having an interactive article, you have the code itself, the output of the code and the explanations (in Markdown) in one place (next to each other), so it is perfect for e.g. communicating ML / AI concepts to other developers.

Now let’s get started with building our rapid prototyping environment! Remember, you’ll only need to set this up once :).

Setup: Chocolatey

Installing Chocolatey CLI (Windows)

As I mentioned, Chocolatey is a software management solution for Windows, we'll use this for installing Python and Deno. We need Python for installing and running the Jupyter Lab IDE, and Deno provides the Typescript language support for Jupyter Lab.

If you use Linux, skip these steps, and jump to the installing Python and Deno “On Linux" section.

To install Chocolatey, execute the following command from an elevated Command Prompt (source):

@"%SystemRoot%\System32\WindowsPowerShell\v1.0\powershell.exe" -NoProfile -InputFormat None -ExecutionPolicy Bypass -Command "[System.Net.ServicePointManager]::SecurityProtocol = 3072; iex ((New-Object System.Net.WebClient).DownloadString('https://community.chocolatey.org/install.ps1'))" && SET "PATH=%PATH%;%ALLUSERSPROFILE%\chocolatey\bin"

This command downloads and runs the remote install1.ps script on your computer, so you should check its contents to ensure safety before running it.

The most important Chocolatey CLI commands

choco list -localonly: list the locally installed packages
choco install PackageName: installs a package (you can browse the available packages in the Community Package Repository)
choco upgrade PackageName: upgrades a package
choco uninstall PackageName: uninstalls a package
choco outdated: list the locally installed packages that have a newer version

Setup: Installing Python and Deno

On Windows

In the elevated Command Prompt, run choco install python for installing Python. Then run the refreshenv command for refreshing the PATH and other environment variables. Now if you run python --version, it returns the version of Python.

Run python -m pip install -U pip for upgrading the Python package manager (pip).

Then install the Deno Typescript Kernel:

choco install deno

On Linux

On Linux, you can use your distribution's package manager for installing Python. Then run python -m pip install -U pip for upgrading the Python package manager (pip).

Then install the Deno Typescript Kernel and the Pnpm package manager:

curl -fsSL https://deno.land/install.sh | sh

You should check the contents of these remote script files to ensure safety before running them.

Deno installation

Creating a Python virtual environment (venv)

From this point on, the installation steps are very similar on both Windows and Linux, just use /home/YourUser/ instead of C:\Users\YourUser\ as your home directory.

A virtual environment contains a set of installed packages. I suggest creating a separate virtual environment for each Python-based project on your computer, so we create a new virtual environment for JupyterLab and Deno. You can create additional virtual environments later.

Open a Command Prompt (non-elevated), and create a new directory for the virtual environments: mkdir C:\Users\YourUser\venv, then create the new virtual environment with the following command: python -m venv C:\Users\YourUser\venv\jupyter-deno

We have to activate the virtual environment before we install packages into it with the C:\Users\YourUser\venv\jupyter-deno\Scripts\activate command. After we executed this script, the prompt changed to (jupyter-deno) C:\.

The most important Python package manager (pip) commands

pip list: list the locally installed packages
pip install PackageName: installs a package (you can browse the available packages in the Python Package Index)
pip install --upgrade PackageName: upgrades a package
pip uninstall PackageName: uninstalls a package
pip list --outdated: list the locally installed packages that have a newer version

Setup: Installing JupyterLab with the Deno Kernel

Inside the activated environment, we install Jupyter Lab: pip install jupyterlab, then we add the Deno kernel to Jupyter with the deno jupyter --unstable --install command.

Hidden files are not visible in Jupyter Lab by default, so for example we can't see the .env files in Jupyter Lab. We use the .env files e.g. for storing the key for OpenAI's API. To display these files in Jupyter Lab's file manager, we have to generate a new Jupyter config file with the following commands:

mkdir C:\Users\YourUser\.jupyter
cd C:\Users\YourUser\.jupyter
jupyter server --generate-config

The last command generates a C:\Users\YourUser\.jupyter\jupyter_server_config.py file. Open this file and set ContentsManager.allow_hidden = True. Restart Jupyter Lab, and now you can show or hide hidden files in Jupyter Lab using the Show Hidden Files menu item in the View menu.

As the last step of the installation process, create a directory for the Jupyter notebooks and start Jupyter Lab:

mkdir C:\Users\YourUser\jupyter
cd C:\Users\YourUser\jupyter
jupyter lab

Installing Ollama and a model

As I mentioned before, I use Ollama for running local LLM models.

You can download Ollama for Windows here and find the model library here. After installing Ollama, I suggest you to download the codellama:7b-code model, it's great for testing purposes: ollama pull codellama:7b-code. This is a small model, requiring about 4 GB of RAM. I use this model on my older laptop that has only 16 GB RAM and a dual core CPU, and it gives responses with an acceptable speed. I use this local model in the prototype of the “Text reviewer app”, too.

The most important Ollama commands:

ollama list lists the downloaded models
ollama pull ModelName:version downloads an LLM model
ollama serve runs the Ollama server in the background

Try out our first AI prototype with a local model

If you haven't started Jupyter Lab yet, activate your virtual env and start Jupyter Lab inside it with the jupyter lab command. You can open the UI in your browser: http://localhost:8888/. Start Ollama with ollama serve.

Download the notebook containing the app's prototype into the C:\Users\YourUser\jupyter directory. Select the downloaded notebook from JupyterLab's file browser (left panel on the screen), then select the "Clear Outputs of All Cells" from the "Edit" menu to remove all the saved outputs from the cell, then click on the "Run all cells" menu item from the "Run" menu. After some processing, the results of the comparison should be visible above the last cell:

What’s next

In the following parts of my article series, I'm going to show you:

How to use the Jupyter Lab IDE and the Deno Javascript/Typescript runtime that executes the Typescript code in a notebook
How the "Text reviewer app" prototype (screenshot above) works
The basics of LLMs and LangChain.js. (LangChain enables us to easily switch between the different LLM models and provider APIs, like the OpenAI platform API or the local Ollama API.)
How to create the state and store of the "Text reviewer app" based on the prototype, and using AI for code generation, and
How to build a React and an Angular app that uses the above mentioned store

Summary

Congrats on completing this little tutorial! I know that it wasn’t the most exciting process you have ever followed along. But trust me, now you have a powerful prototyping environment for experimenting with building AI tools. It’s like getting all the ingredients needed for a great meal (and building a kitchen at the same time :D ). In the following articles, we are going to start cooking, so stay tuned for the fun part!

In the next two articles, I'm going to explain how to use the JupyterLab IDE and the Deno Javascript / Typescript runtime, and I’m going to show you in detail how the “Text reviewer app” prototype works.

👨‍💻About the author

My name is Gergely Szerovay, I worked as a data scientist and full-stack developer for many years, and I have been working as frontend tech lead, focusing on Angular-based frontend development. As part of my role, I'm constantly following how Angular and the frontend development scene in general is evolving.

Angular has advancing very rapidly over the past few years, and in the past year, with the rise of generative AI, our software development workflows have also evolved rapidly. In order to closely follow the evolution of AI-assisted software development, I decided to start building AI tools in public, and publish my progress on AIBoosted.dev , Subscribe here 🚀

Follow me on Substack (Angular Addicts), Substack (AIBoosted.dev), Medium, Dev.to, X or LinkedIn to learn more about Angular, and how to build AI apps with AI, Typescript, React and Angular!

DEV Community