GPU hopping just hit a wall. One integration tore it down.

Anyone who has ever tried to train an AI model on a GPU in a different cloud provider than the data knows the drill: either copy your dataset into every vendor's bucket, or pay egress fees, about $0.09/GB on AWS, each time you pull data across the boundary. The costs pile up fast when base models are tens of gigabytes and datasets run into terabytes. The next trillion-dollar bottleneck in AI isn't…

A new integration between Hugging Face and SkyPilot targets that pain point directly. Hugging Face Storage, previously a standalone service, is now a first-class backend in SkyPilot, the open-source multi-cloud job scheduler. The result: any Hugging Face repo, model, dataset, or bucket, can be mounted into a SkyPilot task using a single hf:// URL, and reading from that mount carries zero egress charges no matter which cloud the GPU runs on.

How the integration works

The mechanism is simple. A SkyPilot task declares a file_mounts block with a source like hf://Qwen/Qwen3.5-4B and store type hf. SkyPilot uses Hugging Face's hf-mount FUSE backend to present that repo as a local filesystem path inside the container. The mount can be read-only for models and datasets, or read-write for Hugging Face Buckets used as checkpoint stores. Alibaba's Qwen is building a model for every AI job,…

Authentication uses the HF_TOKEN environment variable that most teams already have configured. That same token works whether the job lands on AWS, GCP, Azure, Nebius, Lambda, or a Kubernetes cluster. No per-cloud bucket keys to manage.

Lazy reads keep GPUs busy

One critical performance detail: the FUSE mount uses lazy fetching. When a process calls read() on a file, hf-mount pulls only the bytes actually requested, not the entire file. A training job can start processing data almost immediately, with the first chunks streaming in while later chunks are still being fetched. In benchmark testing with a Qwen 3.5-4B model fine-tune, the model was ready to train in about 30 seconds at peak throughput up to 500 MB/s.

The lazy approach pays off most on the first epoch, when the local on-disk cache is empty. For subsequent epochs, hf-mount keeps a local cache, so repeat reads stay local. This cache behavior is the same whether users specify MOUNT or MOUNT_CACHED, a deliberate design choice that differs from some cloud object store backends.

Eliminating the data gravity problem

The deeper implication is organizational. GPU capacity today rarely comes from a single provider. Teams reserve blocks on hyperscalers, clusters on neoclouds like Lambda Labs or Nebius, and sometimes on-premise racks. SkyPilot was built to schedule across these heterogeneous pools. But object storage has been the tether, data lives in one provider's regional bucket, and running a job elsewhere means either moving a copy or paying egress. The subtle trap waiting for AI agents in production

Hugging Face Storage at $12, 18/TB/month versus roughly $23/TB for AWS S3 plus egress shifts the economics. The bucket becomes cloud-agnostic in practice, not just in theory. Teams can let SkyPilot pick the cheapest or most available GPU cluster without factoring data location into the decision.

Xet-backed deduplication as a hidden performance lever

The storage backend uses Xet, which splits files into content-defined chunks of roughly 64 KB and stores each unique chunk once. This has concrete benefits for checkpoint-heavy workloads. Incremental checkpoints from adapter training or frozen-layer fine-tunes only upload the chunks that changed. In Hugging Face's testing, appending 10K rows to a 100K-row Parquet table transferred about 10 MB instead of the full ~106 MB.

Re-uploads of identical blobs are nearly instant. An 8.43 GB checkpoint already in the bucket took about 8 seconds to verify versus 24 seconds for the first upload, because only chunk hashes needed to be sent. Server-side copies between repos and buckets also happen by reference rather than byte duplication. The 12,000-line secret behind Bing's speed: how…

What this means for the multi-cloud stack

The integration is jointly developed and fully open source. Nikhil Jha contributed the initial store: hf support; the Hugging Face team upstreamed the hf-mount FUSE fixes needed for unprivileged containers, a common constraint on Kubernetes; and the SkyPilot team wired it into the storage backend. NVIDIA NeMo AutoModel Delivers 3.7x Faster MoE…

For teams already using both Hugging Face Hub and SkyPilot, the delta is minimal. For teams using only one of the two, this integration lowers the barrier to adopting the other. The biggest adoption constraint is the mount requirement: hf-mount needs a base image with glibc 2.34+ and /dev/fuse available. Most modern Linux containers support this, but some minimal images may not.

GPU demand continues to outstrip supply from any single provider. The ability to treat compute as a fungible resource, without storage dragging it back to one vendor, may prove essential. This integration does not fix every multi-cloud headache, but it kills one of the most stubborn ones.