From aa198e77527f1aa6b7e40739b80e161877203b09 Mon Sep 17 00:00:00 2001 From: sayakpaul Date: Wed, 15 Jul 2026 11:39:45 +0530 Subject: [PATCH] add notes about model implementation in our skills. --- .ai/models.md | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/.ai/models.md b/.ai/models.md index 744c6b3a5234..bdc6464c9aec 100644 --- a/.ai/models.md +++ b/.ai/models.md @@ -6,6 +6,7 @@ Linked from `AGENTS.md`, `skills/model-integration/SKILL.md`, and `review-rules. ## Coding style - All layer calls should be visible directly in `forward` — avoid helper functions that hide `nn.Module` calls. +- Prefer descriptive variable names over short ones. For example, prefer spelling out `query` over `q`. - Avoid graph breaks for `torch.compile` compatibility — do not insert NumPy operations in forward implementations and any other patterns that can break `torch.compile` compatibility with `fullgraph=True`. - No new mandatory dependency without discussion (e.g. `einops`). Optional deps guarded with `is_X_available()` and a dummy in `utils/dummy_*.py`. @@ -182,4 +183,8 @@ Boolean gate. If `False` (default), calling that method raises `ValueError`. All ``` See `transformer_flux.py`, `transformer_flux2.py`, `transformer_wan.py`, `unet_2d_condition.py`, and `pipeline_pixart_alpha.py` for reference usages. Never leave an unconditional `torch.float64` in the model. -6. **Using `torch.empty`.** - Do not use `torch.empty` to initialize parameters. Use `torch.zeros` or `torch.ones`, instead. \ No newline at end of file +6. **Using `torch.empty`.** - Do not use `torch.empty` to initialize parameters. Use `torch.zeros` or `torch.ones`, instead. + +7. **Tensor contiguity.** - Non-contiguous tensors can degrade performance. Therefore, try to maintain contiguity +of the tensors whenever possible. A non-contiguous tensor is usually produced because of the operations. A common +example is a `flatten()` followed by a `transpose()`. This sequence is known to produce non-contiguous layouts. So, prefer calling `contiguous()` on the output tensor to maintain performance. \ No newline at end of file