ViQ just gave multimodal AI the one thing it needed: discrete tokens that don't lose detail

Representing images as discrete tokens, like language, has been a dream for multimodal AI since people started building it. The pitch is clean: a unified space for vision and text, simpler models, less compute. The problem is that making an image discrete costs information. You either keep the pixels and miss what they mean, or you capture the semantics and throw away the detail. A new paper from Tencent-Hunyuan, titled "ViQ: Text-Aligned Visual Quantized Representations at Any Resolution," proposes a framework that tries to hold on to both. DiScoFormer found a way to kill the AI bottleneck that…

The core challenge: quantization without sacrifice

The existing approaches to discrete visual representation sort into two camps. Reconstruction-first methods, like VQ-VAE and its offshoots, replicate pixel-level detail but miss the high-level concepts language models need. Contrastive pre-training, like CLIP, gives you semantic structure but compresses or discards visual granularity, making it useless for precise reconstruction. Fifteen articles on AI generation reveal three shifts…

ViQ splits quantization learning into two stages. The first, text-aligned pre-training, exposes the visual encoder to semantic-rich supervision from a pretrained language model. Critically, it handles native-resolution inputs, no cropping, no resizing, no patchifying before encoding. The second stage, feature discretization, uses a novel proximal representation learning strategy to progressively compact the feature space, combined with a position-aware head-wise quantization mechanism that adapts to arbitrary resolutions. Ai2's EMO makes modular AI emerge from data, not human rules

Technical innovation: proximal learning and head-wise quantization

The proximal representation learning strategy is the paper's key contribution. Instead of forcing the whole feature space into a fixed-size codebook in one pass, it applies iterative compression, gradually clustering and quantizing features while preserving neighborhoods from the continuous space. That keeps both local detail and global structure intact. Jet-Long's bifocal attention just killed the…

The position-aware head-wise quantization mechanism adds flexibility. Standard vector quantization treats each feature position independently, which fails to capture spatial relationships in images of varying size. ViQ introduces multiple quantization heads, each attending to a different spatial scale, and weights their outputs based on positional context. The encoder can allocate more bits to semantically important regions without wasting capacity on uniform backgrounds.

Empirical performance: competitive quality with major speed gains

Extensive experiments across multimodal benchmarks, including image captioning, visual question answering, and text-to-image generation, show ViQ's discrete representations performing within 1 to 3 percentage points of state-of-the-art continuous vision encoders (such as SigLIP and DINOv2), while using fewer parameters and producing discrete tokens. On low-level reconstruction tasks, ViQ outperforms other discrete methods by a substantial margin. Why GPT-5.5 dominates a benchmark that tests how agents…

The training acceleration might be the most striking finding. When ViQ's quantized representations are used in multimodal LLM training, the authors report speedups of 20% to 70% depending on the base LLM size and training recipe. The acceleration comes from reduced dimensionality and simpler architecture of discrete tokens, which slash both memory bandwidth and matrix multiplication costs during forward passes.

"ViQ is the first framework to demonstrate that discrete visual representations can simultaneously match continuous encoders on multimodal tasks and offer dramatically faster training," the paper states. The authors attribute the efficiency gains to the compactness of quantized tokens and the removal of expensive continuous-to-discrete projection layers.

Implications for multimodal AI

If ViQ holds up in practice, it could accelerate the development of large-scale multimodal models that process images and text through a single token space. Unified representations are crucial for next-generation assistants, robotics perception, and real-time vision-language systems where latency and compute budget are constrained. Your AI search pipeline is broken. This open-source…

The ability to handle native-resolution inputs is particularly important. Many vision encoders require resizing images to a fixed resolution, discarding detail and introducing distortion. ViQ's resolution-agnostic design means it can process high-resolution documents, microscopy images, or satellite photos without pre-processing, retaining information that would otherwise be lost. Robots that adapt without retraining? This new…

The paper also notes that ViQ's discrete tokens are compatible with existing transformer architectures and can be fed directly into LLMs without architectural modifications, a practical advantage for teams integrating vision without rewriting their model stack.

Limitations and open questions

The study's primary limitation: all experiments ran on curated, clean datasets (COCO, Flickr30k, and a proprietary internal dataset). Performance on noisy, domain-shifted, or low-quality images, common in real-world deployments, remains untested. The codebook size and quantization overhead at inference time are not thoroughly analyzed, if the number of heads or codebook entries grows too large, the efficiency gains could erode. The AI safety framework nobody asked for might be the…

The paper does not release a pre-trained model or code at time of publication, which limits reproducibility. External validation will be essential to confirm that the reported speedups translate to production workflows.

Conclusion

ViQ is a thoughtful step toward a unified representation space for vision and language, tackling the long-standing trade-off between semantic richness and visual detail in discrete tokenization. By combining text-aligned pre-training with progressive quantization and position-aware heads, Tencent-Hunyuan's team achieves competitive multimodal performance while delivering substantial training speedups. As multimodal models grow in scale and ambition, efficient and detail-preserving discrete representations may become a key enabler. Your AI model is a commodity. The pipeline is where the…