support dsa fused

[Jintao-Huang]

No description provided.

[gemini-code-assist]

Code Review

This pull request introduces the apply_dsa_kernel_fusion parameter to enable fused DSA sparse-attention kernels, updating both the arguments and documentation. It also adds logic to set the expert_dtype configuration attribute for deepseek_v4 models when saving weights. The feedback suggests deleting the expert_dtype attribute instead of setting it to None to prevent writing null to config.json, and recommends adding dependency validation for apply_dsa_kernel_fusion in __post_init__ to provide clearer error messages.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[gemini-code-assist]

When args.model_type == 'deepseek_v4' and expert_dtype is None, calling HfConfigFactory.set_config_attr(hf_config, 'expert_dtype', expert_dtype) will write "expert_dtype": null into the saved config.json. To keep the configuration clean and avoid potential issues with downstream tools expecting a string or the absence of the key, it is better to delete the attribute when it is None.

                if args.model_type == 'deepseek_v4':
                    if expert_dtype is not None:
                        HfConfigFactory.set_config_attr(hf_config, 'expert_dtype', expert_dtype)
                    else:
                        HfConfigFactory.del_config_attr(hf_config, 'expert_dtype')

[gemini-code-assist]

When apply_dsa_kernel_fusion is enabled, it requires flash_mla and nvidia-cudnn-frontend >= 1.24.0. It is highly recommended to add validation in __post_init__ to check these dependencies and raise a clear error message if they are missing, rather than failing with a cryptic error later during execution.