Hi everyone, today we’re releasing SGLang Omni v0.2.0! This release merges 134… · @GenAI_is_real · TwKit

@GenAI_is_real

Hi everyone, today we’re releasing SGLang Omni v0.2.0! This release merges 134 PRs and welcomes 15 first-time contributors, who contributed 17 merged PRs. Highlights include Tencent Hunyuan Hy-SimulMT for real-time Chinese–English speech translation, speculative decoding for Qwen3-Omni, and native full-duplex support for PersonaPlex. We also continue to improve core model inference performance, streaming playback, and interruption handling, while expanding hardware support, runtime resource management, and Voxt voice input capabilities. 1. Interactive intelligence infrastructure: PersonaPlex gains native full-duplex support, and Nemotron VoiceChat adds configurable multi-session capacity. Session Graph improves speech interruption handling, playback progress acknowledgment, conversation history truncation, and word-level subtitle synchronization. Qwen3-ASR adds incremental transcription for cascaded systems. 2. Real-time speech translation: Added Tencent Hunyuan Hy-SimulMT-30B-A3B for bidirectional Chinese ↔ English speech translation, with streaming audio input and incremental translated text output. 3. Speculative decoding: Qwen3-Omni Thinker adds DFlash and EAGLE3 support, reusing the shared draft worker and multimodal input interfaces. 4. Performance: Continued optimization across Qwen3-ASR／TTS／Omni, CosyVoice3, MOSS-TTS, PersonaPlex, and Nemotron, covering prefill–decode overlap, CUDA Graphs, PackedDiT, reference encoding, sampling, and kernel fusion. 5. Hardware: MiniCPM-o expands to Ascend NPU, covering image, video, speech, and full-duplex capabilities. Intel XPU gains ARK-ASR Encoder Graph support and automatic attention backend selection for Qwen3-TTS. 6. Runtime: GC freezing after warmup and a fixed compute thread pool reduce heap-scanning overhead and thread-related GPU memory growth. Improvements cover stage failure reporting, communication failure tracing, large-chunk fragmentation, session timeouts, and resource cleanup on cancellation. 7. Native MLX and Voxt: The native C++ runtime adds Voxtral TTS and MiniMax Music3, with optional serial stage offload for Music3. Voxt unifies local speech recognition, VAD, and speaker diarization under the Native HTTP／WebSocket runtime, while improving model file lifecycle management, in-app automatic updates, and the Homebrew release workflow.

曝光 49 · 评论 1 · 点赞 1 · 书签 0