Click here to read original post
Continue
is an open-source alternative to GitHub Copilot
, this is an open-source AI coding assistant that allows to connect various large language models(LLMs) within VS Code
and JetBrains
to build custom code autocompletion and chat capabilities. It supports:
- Code parsing
- Code autocompletion
- Code optimization suggestions
- Code refactoring
- Code implementations Inquiring
- Documentation online searching
- Terminal errors parsing
and more. It assists developers in coding and enhancing their development efficiency.
In this tutorial, we are going to use Continue + GPUStack
to build a free GitHub Copilot locally, providing developers with an AI-paired programming experience.
Running Models with GPUStack
First, we will deploy the models on GPUStack
. There are three model types recommended by Continue
:
-
Chat model: select
llama3.1
, this is the latest open-source model trained by Meta. -
Autocompletion model: select
starcoder2:3b
, a highly advanced autocompletion model trained by Hugging Face. -
Embedding model: select
nomic-embed-text
, which supports a context length of 8192 tokens, it outperforms OpenAI ada-002 and text-embedding-3-small models for both short and long context tasks.
After deploying the models, you are also required to create an API key
in the API Keys section for authentication when Continue
accesses the models deployed on GPUStack
.
Installing and Configuring Continue
Continue
provides extensions for both VS Code
and JetBrains
. In this article, we will use VS Code
as an example. Install Continue
from the VS Code
extension store:
Once installed, drag the Continue
extension to the right panel to avoid conflict with the file explorer:
Then, select the settings button in the bottom-right corner to edit Continue
's configuration and connect to the models deployed on GPUStack
. Replace the sections for "models"
, "tabAutocompleteModel"
, and "embeddingsProvider"
with your own GPUStack-generated API Key:
{
"models": [
{
"title": "Llama 3.1",
"provider": "openai",
"model": "llama3.1",
"apiBase": "http://192.168.50.4/v1-openai",
"apiKey": "gpustack_f58451c1c04d8f14_c7e8fb2213af93062b4e87fa3c319005"
}
],
"tabAutocompleteModel": {
"title": "Starcoder 2 3b",
"provider": "openai",
"model": "starcoder2",
"apiBase": "http://192.168.50.4/v1-openai",
"apiKey": "gpustack_f58451c1c04d8f14_c7e8fb2213af93062b4e87fa3c319005"
},
"embeddingsProvider": {
"provider": "openai",
"model": "nomic-embed-text",
"apiBase": "http://192.168.50.4/v1-openai",
"apiKey": "gpustack_f58451c1c04d8f14_c7e8fb2213af93062b4e87fa3c319005"
}
}
Get to Use Continue
After configuring Continue
to connect to the GPUStack-deployed models, go to the top-right corner of the Continue
plugin interface and select Llama 3.1
model. Now you are able to use the features we mentioned at the beginning of this tutorial:
-
Code Parsing: Select the code, press
Cmd/Ctrl + L
, and enter a prompt to let the local LLM parse the code: -
Code Autocompletion: While coding, press
Tab
to let the local LLM attempt to autocomplete the code: -
Code Refactoring: Select the code, press
Cmd/Ctrl + I
, and enter a prompt to let the local LLM attempt to optimize the code:The LLM will provide suggestions, and you can decide whether to accept or reject them:
-
Inquire About Code Implementation: You can try
@Codebase
to ask questions about the codebase, such as how a certain feature is implemented: -
Documentation Search: Use
@Docs
and select the document site you wish to search for and ask your questions, enabling you to find the results you need:
For more information, please read the official Continue
documentation: https://docs.continue.dev/how-to-use-continue
Conclusion
In this tutorial, we have introduced how to use Continue + GPUStack
to build a free local GitHub Copilot, offering AI-paired programming capabilities at no cost to developers.
GPUStack
provides a standard OpenAI-compatible API, which can be quickly and smoothly integrated with various LLM ecosystem components. Wanna give it a go? Try to integrate your tools/frameworks/software with GPUStack
now and share with us!
If you encounter any issues while integrating GPUStack with third parties, feel free to join GPUStack Discord Community and get support from our engineers.
Top comments (0)