Development Environment

The compiled GPU dependencies (PyTorch 2.11+cu130, Flash Attention, DeepEP/NVSHMEM) live only inside the images. Pick the variant by what you are doing:

VariantUse it forHowGPU deps (FA/DeepEP)Runs GPU code
In-imageTraining, tests, any script that executesmake targets / docker run (Docker)YesYes
Host .venv (uv)IDE go-to-definition, hover, find-usagesuv syncNoNo
Dev ContainerIDE inside the image — full env, terminal that runs GPU code.devcontainer/devcontainer.jsonYesYes

In-image is canonical for running; the other two only add IDE intelligence.

Host .venv for the IDE

Bash

uv sync                # creates .venv/ from uv.lock; installs halo editable

Python is pinned to 3.12 (requires-python = ">=3.12,<3.13"), the image's interpreter. uv sync resolves torch 2.11.0+cu130, transformers, trl, accelerate, peft, datasets, ray, and halo editable, enough for Pylance across the tree.

It does not resolve flash_attn or deep_ep, which the Dockerfile installs outside [project.dependencies]: Blackwell takes FA2 from the NGC base and FA4 from a prebuilt wheel, Hopper source-builds FA2 and FA3, and DeepEP is source-built in both. Navigation elsewhere is unaffected because both are imported behind optional-dependency guards. vllm is never imported into the training env; it runs in a separate container reached over HTTP plus the weight-sync clients (halo/weight_sync/).

The .venv torch is the cu130 wheel: it imports without a GPU, but CUDA operations fail. The .venv is git-ignored.

VS Code / IDE setup

Agent skills live at skills/ (.claude/skills and .agents/skills are tracked symlinks to it, so Claude Code and Codex both discover them). .vscode/ is git-ignored — create your own settings.json with the values below.

SettingValueWhy
python.defaultInterpreterPath${workspaceFolder}/.venv/bin/pythonBinds Pylance to the host .venv
editor.defaultFormatter (Python)charliermarsh.ruffFormat + import-sort on save (line length 119)
python.analysis.diagnosticModeworkspaceDiagnostics cover the whole workspace; check mode comes from [tool.pyright]
python.testing.pytestEnabledtrueTest discovery under tests/

Type checking is Pyright in basic mode — full checks need the deps, so run those in the image.

Dev Container

.devcontainer/devcontainer.json runs the editor inside halo:blackwell, so Pylance resolves flash_attn / deep_ep and the integrated terminal runs torchrun / pytest with no docker run wrapper. It carries --gpus all, --ipc=host, --ulimit, --shm-size=128g, --cap-add=SYS_PTRACE (py-spy attaches) and --env-file .env — so cp .env.example .env before first open, or Docker refuses to start the container.

The host scratch volume mounts at /scratch, from $HALO_SCRATCH, which the container refuses to start without: export it in your local shell after confirming with findmnt / df -h that the target really is a large device. make (which resolves it when unset) and docker-compose.vllm.yml read the same knob (HALO_SCRATCH), so one export points them at that volume too.

containerEnv then pins HF_HOME, HF_DATASETS_CACHE, TMPDIR and HALO_DATA_ROOT under it, so HF caches, temp files, the S3 dataset cache and profiler traces all stay off the small in-container root FS.

The Dev Containers extension (ms-vscode-remote.remote-containers) is a UI extension — install it on your local VS Code, not on an SSH remote (VS Code refuses the remote install), which is why it is absent from the Dev Container's extensions list in .devcontainer/devcontainer.json. Then Cmd/Ctrl+Shift+P → "Dev Containers: Reopen in Container"; Pylance binds to /usr/bin/python inside the image. "Reopen Folder Locally" drops back to the host .venv. For Hopper, change "image" to halo:hopper.

Environment variables

The code does not auto-load .env — it is passed with docker run --env-file .env. Only the GPU make targets do this (DOCKER_RUN, and only when ENV_FILE is non-empty); the CPU targets (check-lock, test-cpu, seed-hf-cache, verify-move, verify-artifacts, verify-efficiency, verify-ep and verify-cp at TIER=cpu, and make bench's band step) pass no credentials at all; docs and diagrams run on the host.

The CPU targets other than check-lock mount HF_CACHE (default $HALO_SCRATCH/hf) read-write, because many CPU tests load a real tokenizer. A test calling from_pretrained directly hard-fails when the cache is missing and the Hub is unreachable; one going through tests/common/tokenizers.py skips instead, or fails under HALO_TEST_REQUIRE_HUB_CACHE=1. make seed-hf-cache fills that cache (what the seed holds).

Secrets live in the repo-root .env; the cache and path redirects onto a verified large volume and DOCKER_RUNTIME are on Docker → Standard training launch, and the toolkit and cache variables are cataloged in Environment variables.

VariablePurposeWhere setOwner page
WANDB_API_KEYWeights & Biases logging.env—
HF_TOKENGated HuggingFace model/dataset access.env—
AWS_DEFAULT_REGIONRegion for your S3 bucket / ECR registry.envAWS Auth
OPENROUTER_API_KEY, OPENAI_API_KEY, TAVILY_API_KEY, SERPER_API_KEY, BRAVE_API_KEY, KEENABLE_API_KEYExternal LLM + web-search backends.envNative Tool-Use
VLLM_API_KEYThe key a local OpenAI-compatible server expects (its --api-key), else OPENAI_API_KEY.envEnvironment variables
HALO_SCRATCHThe large host volume the make targets, the dev container and docker-compose.vllm.yml mountshell export (compose also reads .env; make does not)Docker
SGLANG_MODEL_DIRThe host directory docker-compose.sglang.yml mounts (required, no fallback)shell export or .envRollout Servers
VLLM_GROUP_HOST, SGLANG_GROUP_HOSTThe trainer address a rollout server on another host dials back to.envRollout Servers

Paths

PathRole
/workspaceRepo bind-mount and working dir in the image (-v $(pwd):/workspace -w /workspace); also PYTHONPATH
scratch volume (HALO_SCRATCH)Checkpoints, logs, HF cache, temp — verify its capacity before a large write
.env (repo root)Secrets, passed via --env-file .env
/root/.awsAWS credentials, bind-mounted for S3 / ECR
.venv/ (repo root)Host-side uv environment for the IDE (git-ignored)

Next steps

Source on GitHubUpdated 18 November 2026