Reliably pull ModelScope / Hugging Face weights on network-restricted GPU servers, resume after disconnects, verify and store them, then hand them straight to vLLM / Ollama.
The challenge
- LLM weights often run to tens of gigabytes, and downloads on private or restricted networks keep breaking and need retries.
- Several models and versions queue up with no visible progress, and someone has to watch for completions and failures.
- The download layout does not match the inference service, so files need sorting by hand.
The solution
Deploy dlmodel on the GPU servers:
- Multi-source fallback: the auto source tries ModelScope first, then an HF mirror, with aria2 multi-connection downloads that step down to resumable HTTP and the native SDK on failure.
- Resumable: pause, resume and continue from where it stopped; after a restart, tasks are requeued automatically.
- Verified storage: LFS files are checked against the SHA-256 recorded in the repository and saved in the vLLM layout, sharing the model volume with vLLM / Ollama.
- Notifications: start, finish, failure and cancel events go to DingTalk, Feishu or WeCom.
The value
- Reliable access to mainstream open-source weights without a proxy.
- One path from download to inference, with vLLM start commands generated from the model library.
- MIT open source, SQLite storage, deployed with one docker compose command.