fix: mask all stop_token_ids in sampling_ids + multi-GPU device support

[lonrencn]

Summary

Two bug fixes for CosyVoice3 inference:

1. sampling_ids() only masks speech_token_size (sos) during ignore_eos=True, not eos or other stop tokens — on certain transformers versions, the LLM can predict EOS with high confidence as the first token, and the unmasked EOS leaks through the ignore_eos window, producing garbage output.

2. Hardcoded torch.device('cuda') defaults to cuda:0 — on multi-GPU systems, torch.cuda.set_device() has no effect and models load on the wrong GPU.

Changes

cosyvoice/llm/llm.py — sampling_ids fix

CosyVoice3LM.stop_token_ids = [speech_token_size + i for i in range(200)] covers sos(6561), eos(6562), task_id(6563), fill_token(6564), etc. The original code only masked index speech_token_size (sos=6561), leaving eos(6562) unmasked during the minimum-length window.

This is usually harmless because a well-functioning LLM doesn't predict EOS early. But with slight numerical differences across transformers versions (e.g., SDPA mask handling changed between 4.x and 5.x), the logits can shift enough (~0.012) to flip EOS from #2 to #1, causing the model to output garbage.

The fix uses hasattr to maintain backward compatibility with CosyVoice1/2.

cosyvoice/cli/model.py + cosyvoice/cli/frontend.py — multi-GPU fix

torch.device('cuda') always defaults to cuda:0, ignoring torch.cuda.set_device(). Changed to torch.device(f'cuda:{torch.cuda.current_device()}') in all model classes.

Testing

Verified with Fun-CosyVoice3-0.5B-2512:

• Chinese: "你好世界" → ASR: "你好，世界。" ✅
• English: "Hello, this is a test." → ASR correct ✅
• Environment: Python 3.11, torch 2.11+cu128, transformers 5.3.0

[LauraGPT]

FunAudioLLM-side triage/validation on current head a12ae5e2c9eadce54c97a1b97f9fde91c0e2075c.

What I checked locally:

git merge-base HEAD origin/main
# 074ca6dc9e80a2f424f1f74b48bdd7d3fea531cc, same as current main

git diff --check origin/main...HEAD
# no whitespace errors

python3 -m py_compile cosyvoice/cli/frontend.py cosyvoice/cli/model.py cosyvoice/llm/llm.py
# passed

The sampling_ids() change is consistent with the class definitions I checked: Qwen2LM.stop_token_ids covers speech_token_size + range(3), and the CosyVoice3 LM class covers speech_token_size + range(200), so masking self.stop_token_ids during the ignore_eos window is aligned with the model-specific stop-token range while preserving the old fallback for older classes without that attribute.

I also sanity-checked the CPU branch of the new device expression in this environment; torch.device(torch.device("cpu")) is accepted by the installed PyTorch, so the expression is not an immediate CPU runtime blocker. I did not run full model inference because this environment does not have the required CosyVoice3 weights and usable CUDA driver, so this is a scoped syntax/static review rather than an end-to-end approval.