Can't this be used in Comfyui and what about GGUF or int8 model versions?
Can't this be used in Comfyui and what about GGUF or int8 model versions?
Thanks for the interest!
ComfyUI — no official integration right now. JoyAI-Video-Edit is a streaming editor: the server keeps rolling KV state across chunks and edits frames as they arrive over a WebSocket, which doesn't map cleanly onto ComfyUI's stateless graph execution. A fixed-length clip→clip node on top of our pipeline is technically doable, but the cleaner path is the 🤗 Diffusers pipeline on our README TODO — once that lands, building ComfyUI nodes on top becomes much easier. Community ports are very welcome.
GGUF / int8 — no GGUF release planned: it's a custom 16B MMDiT with a custom streaming runtime, so a GGUF file alone wouldn't be runnable anywhere today. The quantization you'd actually want is already in the serving stack, applied at load time:
- all DiT Linears run FP8 by default (per-channel weight + per-token activation quantization, cutlass kernels in
deploy/joyomni_ops); - INT8 attention on GeForce via SageAttention (
JOYOMNI_SAGE_ATTN=1); JOYOMNI_LOW_VRAM=1stages the model to the GPU block-by-block as FP8, so it fits a single RTX 5090 (32 GB) at 840×480 @ 24 FPS.
So there's no separate quantized checkpoint to download — grab the bf16 weights and the server quantizes at startup. Full setup: DEPLOYMENT.md. The local web UI also accepts uploaded clips, not just a webcam. And for a zero-setup try-out there's the live demo: https://huggingface.co/spaces/wxDai/joyai-video-edit