AI-Assisted Development

Software & Apps 2026-08-30

Shanios supports every major style of AI-assisted coding: cloud assistants (GitHub Copilot, Anthropic, OpenAI-backed tools), editor extensions (Continue, Cody, Cline, Kilo Code, JetBrains Junie), AI-native editors (Cursor, Devin Desktop, Zed), agentic terminal harnesses (OpenCode, Claude Code, aider, Gemini CLI, Codex CLI, Kilo CLI, Pi, OpenHands, Amp, Open Interpreter), and fully local models that never send your code anywhere. The immutable base stays untouched either way — all tooling lives in Flatpak, Nix, containers, AppImages, or your home directory.

Choosing an Approach

Style	Examples	Your code leaves the machine	Needs GPU
Cloud assistant extension	Copilot, Cody, Continue (API mode), Cline, Kilo Code, JetBrains Junie	Yes — snippets sent to vendor	No
AI-native editor	Cursor, Devin Desktop, Zed	Depends on configured provider/model	No
Agentic CLI / harness	OpenCode, Claude Code, aider, Gemini CLI, Codex CLI, Goose, Kilo CLI, Pi, OpenHands, Amp, Open Interpreter	Depends on configured provider/model	No
Local model + editor bridge	Continue/Ollama, Tabby, Cline (local mode)	No	Helpful (AMD/Intel iGPU works; NVIDIA best)
Fully local agentic CLI	OpenCode/aider pointed at Ollama or a local DeepSeek	No	Same as above

Cloud Assistants in Editors

VS Code installs as a Flatpak (org.visualstudio.code); JetBrains IDEs via Flathub. Extensions install inside the editor sandbox exactly as upstream documents them:

flatpak install flathub org.visualstudio.code
flatpak install flathub com.vscodium.codium   # telemetry-free alternative

GitHub Copilot — sign in via the extension; authentication and completions work inside the Flatpak sandbox with no extra configuration.
Continue / Cody — configure either for cloud APIs or point them at a local endpoint (below).
Kilo Code — Apache-2.0, forked from the Cline→Roo Code lineage (Roo Code itself was archived in May 2026). Installs from the VS Code Marketplace, or from Open VSX for VSCodium and other non-Microsoft builds — the official docs specifically list VSCodium among its supported Open VSX targets. BYOK across 400+ models, plus an MCP server marketplace for one-click tool integrations.
JetBrains Junie — JetBrains' own first-party agent (reached GA June 2026), distinct from Continue/Cody, distributed as a plugin across the whole JetBrains lineup (IntelliJ, PyCharm, WebStorm, GoLand, RubyMine, Rider, CLion, PhpStorm — all Flathub-installable on Shanios). Also ships a standalone CLI for terminal/CI use. Pricing is folded into JetBrains AI Pro/Ultimate, not sold separately.
API keys live in ~/.config / ~/.var/app/<app>/config under @home — they persist across updates and rollbacks.

Agentic CLIs that expect Node.js run cleanly inside a Distrobox container or via Nix:

# Distrobox route: full node environment, home shared
distrobox create --name dev --image archlinux:latest
distrobox enter dev
npm install -g @anthropic-ai/claude-code   # or: pip install aider-chat

# Nix route: single binaries in your profile
nix-env -iA nixpkgs.aider

AI-Native Editors

A separate track from editor extensions: whole IDEs built agent-first rather than agent-added.

Editor	What it is	Linux install
Cursor	VS Code fork, deepest first-party agent integration; free tier up to a $200/mo Max tier	AppImage — needs `libfuse2` on a fresh Ubuntu-based base
Devin Desktop (formerly Windsurf)	Cognition acquired and rebranded Windsurf in June 2026 after a prior OpenAI-acquisition deal collapsed; Pro moved to a flat $20/mo, replacing per-message token-credit billing	AppImage — prefer a distro `.deb`/`.rpm` where the vendor offers one, since the AppImage tarball doesn't self-update
Zed	Rust-based, GPU-accelerated editor. Its Agent Panel speaks the open Agent Client Protocol (ACP) — model-agnostic, and can drive an external CLI harness (Claude Code, Codex CLI, OpenCode) from inside the same panel, including multiple agents running in parallel on different parts of a codebase	Native Linux binary; ships its own edit-prediction model (Zeta2)

Both Cursor and Devin Desktop distribute as AppImages rather than distro packages on Linux — which fits Shanios well: an AppImage runs from anywhere in @home without touching the immutable root, same as any other portable binary on this OS. Neither needs anything beyond a normal desktop session; no container or Distrobox required unless you specifically want one for isolation (see Agent Sandboxing below).

Agentic CLIs & Harnesses

Terminal-native coding agents (harnesses) drive whole tasks — editing files, running commands, iterating on failures. In 2026 they are the fastest-growing category of dev tooling. They are ordinary userland binaries, so they install into ~/.local/bin (under @home, survives updates) or inside Distrobox without touching the immutable root:

Harness	Install on Shanios	Provider flexibility
OpenCode	`curl -fsSL https://opencode.ai/install \	bash `(lands in` ~/.local/bin`) or` npm i -g opencode-ai` in Distrobox	Most flexible — 75+ providers via Models.dev, plus local models through Ollama-compatible endpoints
Claude Code	`npm install -g @anthropic-ai/claude-code` (Distrobox/nvm)	Anthropic API; point `ANTHROPIC_BASE_URL` at compatible endpoints for other backends
Codex CLI	`npm i -g @openai/codex` (Distrobox/nvm)	OpenAI-first; custom providers (DeepSeek, Qwen, Kimi) via `~/.codex/config.toml` with `wire_api = "chat"`
Gemini CLI	`npm install -g @google/gemini-cli` (Distrobox/nvm)	Google API / OAuth free tier; Plan Mode for long-horizon tasks
aider	`nix-env -iA nixpkgs.aider` or `pip install aider-chat` (pipx/venv in home)	Git-native — auto-commits every change for reviewable history; any OpenAI-compatible endpoint
Goose	Binary from block/goose releases into `~/.local/bin`	Multi-provider including local Ollama
GitHub Copilot CLI	`gh extension install github/gh-copilot`	GitHub-native workflows
Kilo CLI	Terminal counterpart to the Kilo Code extension above; itself forked from OpenCode	Same MCP marketplace and 400+-model BYOK routing as the extension, in a shell
Pi	Build from source (TypeScript) — check the repo for the current published-package name before assuming an npm one-liner	Deliberately minimal — only four tools (read/write/edit/bash) and a sub-1,000-token system prompt; MIT-licensed, also embeddable as a TypeScript SDK
OpenHands (formerly OpenDevin)	Self-hosted, runs as a sandboxed container via Podman instead of Docker — follow the current official run command exactly rather than an older tutorial's, since the image path has moved at least once (`ghcr.io/openhands/...`, previously `all-hands-ai/...`)	MIT, 80k+ GitHub stars — one of the most-starred agents by that measure. Note: the dedicated `OpenHands-CLI` is no longer actively maintained; the project now points users at its "Agent Canvas" for a local, customizable setup instead of the standalone CLI
Amp	`npm i @ampcode/cli` (Distrobox/nvm) — the package was recently renamed from `@sourcegraph/amp`, check current docs if this drifts again	Sourcegraph's harness — pay-as-you-go, zero markup over provider pricing; free tier includes full agent access with no hard token cap
Open Interpreter	`pip install open-interpreter` (venv/pipx in home, or `uv pip install`), then run `interpreter`	65k+ GitHub stars, MIT-licensed. Natural-language interface to Python/JS/shell execution rather than a file-editing harness per se — strong story for local models (Llama/Mistral/Ollama) and non-coding tasks (document/spreadsheet editing) alongside code

A useful rule of thumb the ecosystem has converged on: the harness shapes workflow; the model decides quality. All of the above run happily on budget backends, so pick the tool whose ergonomics fit you and route it to whatever provider suits your budget and privacy needs.

Where the market actually stands (2026): GitHub Copilot leads on raw share (~40%, driven by enterprise/Microsoft bundling), but Claude Code already out-adopts Cursor in the workplace despite being newer, and was rated the single most-loved tool in JetBrains' most recent developer survey. Cursor remains the fastest-growing at 5M+ active users. By GitHub star count, OpenCode is the most-starred open-source agent (~199k) with OpenHands, Gemini CLI, and Codex CLI all in the same tier. In practice the average professional developer now runs 2.3 of these tools side by side rather than standardizing on one — a data point worth keeping in mind before over-indexing on any single recommendation in this page.

A different category worth knowing about: Warp, an AI-native terminal (not a harness itself) with full native Linux support since 2024 and a client that went dual MIT/AGPLv3 open-source in 2026. Its "Oz" feature is a cloud background-agent orchestrator that can launch and drive Claude Code, Codex CLI, or Warp's own agent from inside the terminal — worth a look if you'd rather manage multiple harnesses from one place than run each standalone.

Wiring a Harness to Any Provider

Every harness consumes OpenAI-compatible endpoints, which makes providers interchangeable. Worked example with DeepSeek — the identical two-line swap works for OpenAI, Gemini, GLM, Kimi, Qwen, or a self-hosted endpoint:

# Hosted provider example (DeepSeek shown; same shape for any compatible API)
export OPENAI_API_BASE=https://api.deepseek.com
export OPENAI_API_KEY=sk-...

# Or skip hosted entirely — local weights via Ollama
podman exec ollama ollama pull deepseek-r1:8b        # reasoning-style distill
podman exec ollama ollama pull deepseek-coder-v2:16b # coding-focused variant
podman exec ollama ollama pull qwen2.5-coder:7b      # alternative coding family
aider --model ollama/deepseek-coder-v2:16b

Budget-hosted APIs keep per-session costs to cents; local distills trade some capability for zero data egress. Nothing here is exclusive — the same machine can mix hosted providers for heavy tasks and local models for proprietary code, and switching providers is a two-line change because the harness layer does not care.

Free & Low-Cost Model Access

You can run a full agentic workflow in 2026 without paying for inference. Several routes, each with different trade-offs.

OpenCode Zen

Zen is OpenCode's curated model gateway (https://opencode.ai/zen/v1, OpenAI-compatible), with 90+ models total behind pay-per-use. Beyond that, it exposes a smaller rotating free tier — around 5 free models at any given time (community and vendor-promo coding models; names rotate — check the picker or curl https://opencode.ai/zen/v1/models), capped at 100 requests/day across all of them, no key of your own required.

Privacy caveat that matters: several Zen free-period models state that collected data may be used for model improvement. Fine for boilerplate and public code; avoid proprietary paths on those specific models.

Kilo Gateway

Kilo Code's direct equivalent to Zen — a free-model gateway with zero configuration, no API key required, and no sign-in needed to start. As of late 2026 it exposes 40+ free coding models (ByteDance Seed, Grok Code Fast, NVIDIA Nemotron, Arcee Trinity, and others), with anonymous (unauthenticated) use rate-limited to 200 requests/hour per IP — signing in raises the limit. Same rotation caveat as Zen: the exact model roster changes without much notice, so check the live list in Kilo's model picker rather than assuming a specific name is still available.

Free Tiers Built Into the Tools Themselves

Zen and Kilo Gateway are free models; this is a different thing — a free quota on the tool itself, independent of which model it's calling. Every harness and editor covered on this page has some story here, and they vary a lot in how genuinely usable and how permanent they are:

Tool	What's actually free	The catch
GitHub Copilot	2,000 code completions + 50 chat requests/month — permanent, no card, no trial expiry	The most durable free tier here; everything else on this list is either time-limited, rate-limited harder, or newer/more likely to change
Google Jules	15 fully autonomous agent tasks/day, 3 concurrent, on Gemini 2.5 Pro	Google-account gated; daily quota resets, doesn't roll over
Zed (Personal plan)	Free forever — unlimited local models and BYOK, no restriction	Only Zed's own hosted models require the $10/mo Pro plan (which bundles a $5 token credit)
Codex CLI	50 agentic coding messages/day + 1,000 terminal completions/month via a free ChatGPT account sign-in	Ties the CLI to a ChatGPT Free account, not a bare API key
Cursor (Hobby)	Community-reported ~2,000 completions + 50 slow "premium" requests/month	Cursor doesn't officially publish exact numbers — the pricing page just says "Limited"
Devin Desktop	Unlimited tab completions/inline edits, free, no card	Full agent sessions ("Devin Local") hit an undisclosed, tight daily cap fast
Gemini CLI	250 requests/day on Flash/Flash-Lite via an API key	The old free-OAuth tier (1,000 req/day, no key) was shut down in 2026 after third-party abuse — don't trust an older tutorial's numbers
Claude Code	No permanent free tier; new signups get ~$5 in API credit (roughly 1.67M Sonnet input tokens)	Notable exception: Anthropic's "Claude for Open Source" program gives qualifying maintainers 6 months of the $200/mo Max 20x tier entirely free
JetBrains Junie	30-day AI Pro trial on signup, then ~3 AI credits/30 days ongoing after it ends	Meaningful use beyond light chat needs a paid Pro/Ultimate plan
Amp	Free, ad-supported tier — no hard token cap, but rate-limited to roughly $10/day of usage value	Sponsored suggestions appear in non-code UI on the free tier
OpenHands (Cloud)	Free Individual tier, capped at 10 conversations/day	Beyond that: BYOK, or OpenHands' own at-cost (zero-markup) provider option
Goose	BYOK by default, no bundled credits	One exception: a one-time $10 credit if you authenticate via its Tetrate integration
aider	Nothing tool-native — strictly BYOK	Pairs naturally with any free model-provider tier from the table below

Worth featuring first if you're starting from zero budget: Google Jules' 15 free autonomous tasks/day is the most generous agentic free tier on this list; GitHub Copilot Free is the only genuinely permanent one; Zed's Personal plan gives full BYOK/local-model access with no restriction at all; and Anthropic's Claude for Open Source program is worth knowing about if you maintain a real open-source project.

Permanent Free Tiers (no credit card)

Provider	What you get	The catch
Google AI Studio (Gemini Flash)	Large daily request quota, 1M-token context, multimodal	Prompts may train models outside EU/UK; quotas cut before, verify current limits
Groq	Extremely fast inference on open models (Llama 70B class, GPT-OSS)	Rate-limited (RPM/daily caps); smaller context windows
Cerebras	High daily token volume (~1M tokens/day) on open models	Context length capped on free tier
NVIDIA NIM (build.nvidia.com)	100+ hosted open models against free build credits	Trial-oriented; session logging on some endpoints
Mistral La Plateforme ("Experiment")	Massive monthly token budget incl. Codestral for code	Requires opting into data training
OpenRouter `:free` models	30+ free variants behind ONE key, easy model swapping	Daily request cap tied to lifetime credit (~50/day below $10 top-up, ~1000/day at ≥$10)
GitHub Models	Frontier-class models via Azure endpoint + playground, GitHub account only	Rate-limited; evaluation use
Cloudflare Workers AI	Free daily allocation across many small models	Smaller models; edge-oriented

All speak the OpenAI dialect, so any harness or editor bridge on this page consumes them directly:

# Example: route aider through Groq
export OPENAI_API_BASE=https://api.groq.com/openai/v1
export OPENAI_API_KEY=gsk_...
aider --model openai/llama-3.3-70b-versatile

Stacking Tiers

Because each provider's limits are independent, common practice is stacking several free keys behind a router — OpenRouter (one key + failover, ~5% fee on paid usage), or self-hosted LiteLLM proxy in Podman if you want full control:

podman run -d --name litellm -p 4000:4000 \
  -v ~/.litellm/config.yaml:/app/config.yaml \
  ghcr.io/berriai/litellm:main --config /app/config.yaml

Point any harness at http://localhost:4000/v1 and it fails across free tiers automatically.

Honest expectations: free tiers change monthly, throttle mid-task, and never include frontier weights. They excel at boilerplate, tests, and learning; for hard multi-file refactors, metered paid inference remains the rational upgrade — the harnesses themselves stay free either way.

MCP: Wiring Agents into Tools

The Model Context Protocol is the standard way AI clients call external tools — adopted by Anthropic, OpenAI, Google, and Microsoft, and governed under the Linux Foundation since late 2025. If a harness supports MCP (Claude Code, OpenCode, Codex CLI, Gemini CLI, Cline, and most editors do), one server config unlocks integrations everywhere.

Developer-relevant first-party servers worth starting with:

Server	What your agent gains
GitHub (`github/github-mcp-server`)	Issues, PRs, code search without leaving the agent loop
Context7 (`upstash/context7`)	Up-to-date library docs injected at prompt time — cuts stale-API hallucinations
Filesystem / Git (reference servers)	Scoped repo access for non-repo-aware tools
Playwright / Puppeteer	Browser automation for testing web changes
Postgres	Read-only schema/data inspection during feature work

Local stdio servers are plain processes configured in your client's config — state lives in $HOME, so configs survive updates like everything else:

{
  "mcpServers": {
    "context7": { "command": "npx", "args": ["-y", "@upstash/context7-mcp"] }
  }
}

Security posture (the ecosystem's known weak spot): prefer first-party servers over registry finds, pin versions, scope credentials tightly, and treat random community servers with the same trust you would a curl-piped shell script. Prefer OAuth-backed remote (Streamable HTTP) servers for anything touching production systems.

Context Engineering

2026's named successor discipline to "prompt engineering," specific to agentic coding: the unit of control isn't the prompt anymore, it's the entire information environment an agent operates in — which files it sees, which rules apply, which history it's given, which tools it can call. The practice has converged on a few concrete habits:

Quality over quantity — a small, clean, relevant context paired with a weaker model consistently outperforms a huge, cluttered one paired with a stronger model. Trim AGENTS.md/context files as aggressively as you'd trim a diff.
Structured rules, not prose — short bullet rules plus an explicit ## Preferred / ## Avoid block with real code samples reads more reliably to an agent than a paragraph of guidance.
Freshness metadata — stamping context files with last_updated, owner, and scope lets an agent (or a human) judge whether a rule still applies before trusting it, the same instinct that already applies to any other stale documentation.

This is additive to, not a replacement for, the AGENTS.md and spec-driven practices below — it's the discipline of deciding what goes into those files in the first place.

Repo Instructions: AGENTS.md

The AGENTS.md format is the open standard for telling coding agents about your project — build commands, conventions, testing rules, no-go zones. It is governed under the Linux Foundation's Agentic AI Foundation, adopted by 60k+ repositories, and read natively by every major harness (Claude Code, Codex CLI, OpenCode, Cursor, Copilot). One file serves them all.

A minimal AGENTS.md at the repo root:

# Project Notes

## Build & Test
- `make build` — compile
- `make check` — lint + unit tests; must pass before any commit

## Conventions
- Shell scripts follow existing style; `bash -n` before committing
- Never edit files under keys/ — signed material only changes via release process
- Docs live in docs/; regenerate manifests after edits

## No-gos
- Do not modify GPG keyring files without maintainer approval

Keep it short and factual — agents re-read it every session, so bloat costs tokens and compliance. Vendor-specific extras (CLAUDE.md, .cursorrules) still work alongside it when you need tool-specific hints.

Spec-Driven Development

The 2026 methodology shift: instead of prompting an agent and hoping ("vibe coding"), write the specification first and let agents implement against it. GitHub's Spec Kit (Specify CLI: /specify → /plan → /tasks → /implement) and AWS's Kiro IDE formalize the loop, but the core practice needs nothing beyond Markdown discipline:

.specs/
├── requirements.md   # user stories + acceptance criteria (no architecture)
├── design.md         # stack decisions, constraints, trade-offs
└── tasks.md          # numbered ~2-hour tasks, test-first, agent-sized

The numbered task list matters most: an agent told "implement task 3: add the webhook deliveries migration" makes far better choices than one told "build the feature". Works identically across every harness listed above.

Async Cloud Agents

Beyond terminal harnesses running locally, 2026's other model is delegation: assign a task, an isolated cloud VM does the work, a draft PR comes back. These complement — not replace — local tools: delegate routine tickets while your desktop handles design work.

Service	Trigger	Returns
GitHub Copilot coding agent	Assign an issue / `@copilot`	Draft PR on its own branch
Google Jules	Web UI / `jules` issue label (free tier: limited daily tasks)	Plan → approval → PR
OpenAI Codex cloud	chatgpt.com/codex prompt or `@codex`	Diff/PR from sandboxed VM
Cursor cloud agents	Editor/mobile/ticket	PR plus run recording

Review discipline applies to all of them: read the diff, not the summary; never let an agent merge its own code; scope tasks tightly. On Shanios nothing special is required to consume their output — they open ordinary PRs against your hosted repos, which you then review and pull as usual.

AI Code Review

Beyond generation, 2026's other big trend is automated review. Three integration styles:

SaaS reviewers — CodeRabbit, Qodo, Greptile, Sourcery: connect your repo host, get AI summaries and line-level review comments on every PR. Nothing to install locally.
Self-hosted reviewer — PR-Agent (open source, from Qodo) runs as a container and reviews PRs on demand or via webhook:

``bash podman run --rm \ -e OPENAI_KEY=$OPENAI_API_KEY \ -e CONFIG.GIT_PROVIDER=github \ docker.io/codiumai/pr-agent:latest \ --pr_url=https://github.com/you/repo/pull/123 review ``

In-loop review — ask your existing harness: opencode "review my uncommitted diff for race conditions" works today with zero extra setup, and pairs naturally with the Btrfs snapshot safety net before letting an agent touch anything.

Local Models with Ollama

The AI & LLMs page documents full self-hosted inference stacks. For a developer workstation, the short path is the Ollama container with your GPU passed through:

# AMD / Intel GPUs: device passthrough works out of the box
podman run -d --name ollama -p 11434:11434 \
  --device /dev/dri \
  -v ~/.ollama:/root/.ollama \
  docker.io/ollama/ollama

# Pull coding/reasoning models (small enough for 8 GB RAM class machines)
podman exec ollama ollama pull qwen2.5-coder:7b
podman exec ollama ollama pull deepseek-r1:8b

Model storage lives in ~/.ollama under @home, so downloaded models survive OS updates and rollbacks without re-downloading.

Bridge Your Editor to It

Point Continue (VS Code/JetBrains) at the local endpoint:

{
  "models": [
    { "title": "Local Qwen Coder", "provider": "ollama",
      "apiBase": "http://localhost:11434", "model": "qwen2.5-coder:7b" }
  ]
}

Or use aider against it:

aider --model ollama/qwen2.5-coder:7b

Beyond Ollama

Ollama is the easiest local runtime, but not the only one:

# llama.cpp server — single binary, OpenAI-compatible endpoint
podman run -p 8080:8080 -v ~/.models:/models \
  ghcr.io/ggml-org/llama.cpp:server \
  -m /models/qwen2.5-coder-7b-q4_k_m.gguf --host 0.0.0.0

# vLLM — high-throughput serving when you have a big GPU (see GPU Containers)

Any of these exposes an OpenAI-shaped API, which is exactly what every harness and editor bridge in this page consumes. And for quick scripted questions outside an agent, the llm CLI (pipx install llm or nix-env -iA nixpkgs.llm) is a handy Swiss-army prompt tool with the same provider plugins.

Not Sure What Your Hardware Can Run?

llm-checker scans CPU/GPU/VRAM/RAM and ranks exactly which Ollama models fit your machine, per category:

# Inside a Distrobox with Node 18+ (or via nvm)
npm install -g llm-checker

llm-checker hw-detect                    # what hardware is detected
llm-checker recommend --category coding  # ranked picks + ready-to-run pull commands
llm-checker check                        # full compatibility analysis
llm-checker ai-run --category coding \
  --prompt "write a hello world in python"   # auto-selects a model, reports tokens/sec

It also exposes itself as an MCP server (npx --yes --package llm-checker llm-checker-mcp), so a coding agent can query your own hardware fit mid-conversation. Pair it with the Btrfs snapshot habit: check → snapshot → pull → test.

Hugging Face as Your Model Source

Hugging Face hosts virtually every open-weight model, and the whole local stack above consumes its repositories directly:

# Ollama pulls GGUF repos straight from the Hub (pick a quantisation suffix where offered)
ollama pull hf.co/Qwen/Qwen2.5-Coder-7B-Instruct-GGUF

# llama.cpp loads Hub GGUFs at launch with -hf
podman run --rm -p 8080:8080 -v ~/.cache/huggingface:/root/.cache/huggingface \
  ghcr.io/ggml-org/llama.cpp:server \
  -hf Qwen/Qwen2.5-Coder-7B-Instruct-GGUF:Q4_K_M --host 0.0.0.0

# vLLM serves any HF repo id (needs a substantial GPU — see GPU Containers)

Weights land in ~/.cache/huggingface — under @home, so multi-gigabyte downloads survive updates and rollbacks without re-downloading.

For scripted access and dataset handling, the hf CLI installs cleanly outside the immutable root:

nix-env -iA nixpkgs.python3Packages.huggingface-hub   # or: pipx install huggingface_hub
hf login                                               # store token in ~/.cache/huggingface/token

Their Inference Providers route requests to hosted open models through an OpenAI-compatible endpoint with a rate-limited free tier — another key you can stack alongside the others above.

Local Model Quick-Pick Table (July 2026)

Use Case	Model	Size	VRAM (Q4)	Context	Strength
Best overall (MoE)	Kimi K2.6	32B active / 1T total	~19 GB	256K	SWE-Bench Pro 58.6, agentic coding
Best dense 27B	Qwen 3.6 27B	27B dense	~22 GB	256K	SWE-bench 77.2%, consistent reasoning
Best agentic 24B	Devstral Small 24B	24B dense	~14 GB	256K	Multi-file edits, debugging loops
Best FIM autocomplete	Codestral 22B	22B	~14 GB	128K	Fill-in-middle, Continue.dev
Best 8 GB VRAM	Qwen3 8B	8B dense	~5 GB	128K	Best quality/speed for 8 GB
Reasoning-heavy	DeepSeek-R1 14B / 32B	14B/32B	10 GB / 20 GB	128K	Reasoning-heavy code tasks
Smallest usable	Qwen3 4B	4B	~3 GB	128K	Minimum viable coding

Rule of thumb: 8 GB VRAM → Qwen3 8B; 12 GB → Qwen 3.6 27B; 24 GB → Devstral 24B or Qwen 3.6 27B; 24 GB + headroom → Kimi K2.6 quantized for max quality. FIM (fill-in-middle) for autocomplete: Codestral, Qwen3-Coder, Devstral support it.

Model Selection Decision Framework

flowchart TD
    A[Available VRAM] -->|8 GB| B[Qwen3 8B]
    A -->|12 GB| C[Qwen 3.6 27B]
    A -->|24 GB| D[Devstral 24B / Qwen 3.6 27B]
    A -->|24 GB + headroom| E[Kimi K2.6 MoE]
    
    style A fill:#f9f9f9,stroke:#333,stroke-width:1px
    style B fill:#cfe2f3,stroke:#333,stroke-width:1px
    style C fill:#cfe2f3,stroke:#333,stroke-width:1px
    style D fill:#cfe2f3,stroke:#333,stroke-width:1px
    style E fill:#cfe2f3,stroke:#333,stroke-width:1px

If branching to multiple users: Kimi K2.6 MoE offers the best per-token quality but requires ~19 GB VRAM for Q4. For shared or multi-user setups, Qwen 3.6 27B provides the best consistent reasoning at 22 GB. Devstral 24B excels at multi-file editing loops. Qwen3 8B is the practical floor for daily coding assistance.

Quantization quick-reference: Q4_K_M is the sweet spot — preserves ~80% of original quality at roughly half the VRAM of Q5_XXL. Q3_K_S is viable for 6 GB systems. Avoid Q2_K for code tasks (excessive hallucinations).

Sandboxing Quick-Start Commands

Podman + gVisor (single-user, LLM-generated code):

# Create a dedicated networkless workspace
podman create --name shani-agent \
  --security-opt no-new-privileges \
  --cap-drop=ALL \
  --read-only \
  --tmpfs /tmp:exec \
  --network none \
  -v "$PWD/agent_workspace:/workspace:rw" \
  docker.io/parrothelp/gvisor:latest

# Start the agent workspace
podman start -a shani-agent

Full isolation (multi-tenant / high-value): Use Firecracker booted via machinectl or Kata Containers virtctl. Both present a complete VM with minimal device exposure — ideal when the agent must touch repository contents, SSH keys, or MCP configs.

The root filesystem itself needs no manual snapshot at all. / is always one of the two read-only @blue/@green slots — an agent (or anything else) cannot write to it, and there's nothing to protect there because shani-deploy already keeps the inactive slot as an instant rollback target for OS-level changes. What an agent can actually modify is @home — and unlike the root slots, that has no automatic snapshot schedule out of the box. Set one up once via the systemd timer in Backup & Recovery, or take a one-off manual snapshot right before a specific risky run:

# Extra snapshot of @home right before a risky agent session (read-only, non-destructive)
sudo btrfs subvolume snapshot -r /home /data/snapshots/home/pre-agent-$(date +%s)

# To restore a file or directory afterward, just copy it back out of the snapshot —
# no subvolume deletion needed:
cp -a /data/snapshots/home/pre-agent-<timestamp>/path/to/thing /home/path/to/thing

GPU Notes

AMD/Intel — /dev/dri passthrough is all the container needs; Vulkan/ROCm layers ship in the image.
NVIDIA — the NVIDIA Container Toolkit is not pre-installed on the base system. Use Distrobox (handles GPU/CDI automatically) or, for raw Podman, add the toolkit to the host and then add --gpus all (or the CDI equivalent) to the container instead of --device.
Model size guidance: 7B-class coder models fit 8 GB VRAM or system RAM; 14B+ wants 16 GB+. CPU-only inference works but autocompletion latency suffers — keep local use to chat/refactor requests rather than keystroke autocomplete if you lack a GPU.

Full container details: GPU Containers.

Agent Sandboxing

Running untrusted LLM-generated code requires isolation beyond standard containers. The 2026 consensus: shared-kernel containers (Docker/runc) are insufficient for LLM-generated code — use gVisor at minimum, Firecracker microVMs or Kata Containers for multi-tenant or high-value workloads.

Scenario	Recommended Isolation
Single-user, trusted prompts	Podman + `--security-opt no-new-privileges` + seccomp
LLM-generated code, single-user	gVisor (`runsc` runtime) or rootless Podman with gVisor
Multi-tenant / shared infra	Firecracker microVM or Kata Containers (KVM)
GPU workloads	gVisor (GPU support) or full VM; Firecracker lacks GPU passthrough

Capability restrictions to apply inside any sandbox:

Filesystem: read-only root; writable workspace only under scoped directory
Network: default-deny egress; explicit allowlist for required endpoints
Kernel: drop all capabilities except strictly required (--cap-drop=ALL)
Persistence: block writes to ~/.ssh, ~/.config, MCP configs, IDE configs, git hooks

Prompt injection mitigation: never feed untrusted external content (emails, web pages, issue bodies) into the same agent context that has tool access. Run tools that touch sensitive data in isolated contexts without sharing that context with external input sources.

For Shanios desktop users: run agentic workloads inside a Distrobox or Podman container with the restrictions above. The root filesystem needs no extra protection — it's already read-only regardless — but @home has no automatic snapshot schedule out of the box, so either set up the recurring timer in Backup & Recovery or take a manual one (see above) before a specific long autonomous run.

Privacy Posture

Shanios ships with zero telemetry, but AI assistants are your choice to make:

Local-only workflow — code never leaves the machine; verify by blocking the endpoint or simply using no API keys. Agentic harnesses configured against Ollama keep file contents, diffs, and shell output entirely on-device.
Cloud workflow — read your vendor's data-retention terms before enabling; most offer opt-outs for training on your code (Copilot business tiers, API no-training defaults). Harnesses send more context than autocomplete extensions — full files and command output — so this choice matters more for agents.
A practical middle path many teams use: cloud/harness providers for boilerplate and tests, local models (Qwen/DeepSeek distills) for proprietary code paths.

Troubleshooting

Symptom	Fix
Extension can't reach `localhost:11434` from Flatpak editor	Flatpak apps share the host network namespace by default; if you restricted it, allow the port: `flatpak override --user --share=network org.visualstudio.code`
Ollama container ignores GPU	Check `ls -l /dev/dri` exists; NVIDIA requires toolkit flags — see GPU Containers page; verify with `podman exec ollama nvidia-smi` (NVIDIA image variant)
Very slow completions on CPU	Drop to a smaller model (3B class) or reserve local models for chat-style requests
`ollama pull` fails behind proxy	Pass proxy env into the container (`-e HTTPS_PROXY=...`)