oam/knowledge base/ai/use a local llm for coding assistance in vscode.md

Use a local LLM for coding assistance in VSCode

1. Setup
2. Further readings
   1. Sources

Setup

1. Install Ollama and download the models you mean to use.
   Refer Ollama.

   brew install 'ollama'
   ollama pull 'llama3.1:8b'
   ollama pull 'qwen2.5-coder:7b'
   ollama pull 'nomic-embed-text:latest'
   

2. Install the Continue VSCode extension.

3. Configure the extension to use local LLMs only.

   1. Open the extension's sidebar.
   2. In the top-right, select Local Config from the dropdown menu.
   3. Eventually, tweak the configuration file.

Further readings

• Ollama
• Continue VSCode extension

Sources

• How to use a local LLM as a free coding copilot in VS Code