[FEATURE] Change default normalization ?

[asusdisciple]

Is your feature request related to a problem? Please describe.
At the moment TIMM uses torchvision.transforms.functional.normalize() by default to process the input images.
But the implementation is actually a standardization of z = (x- µ /sigma) if I am not wrong. This can be problematic
because negative values in conv2d layers without bias can produce instability quite frequently. And these layers are often the first after the input.

My question would be why do we normalize around the mean and not from 0-1 which can be easily scaled as well. Are there any advantages of standardization over normalization? I dont think it can be assumed that images in general follow a normal distribution when you look at histograms etc. .

Describe the solution you'd like
Replace with min/max norm.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Context is I got a lot of stability issues until I normalized the input in my forward function.

[rwightman]

This is basically the way it's been done, most models use measured imagenet mean/std as that's been the most common default for imagenet pretrained models. There are some that use [-1, 1] (mean 0.5, std 0.5), some that use [0, 1] (mean 0, std 1), some use the mean/std calculated by OpenAI for their CLIP pretraining. At the end of the day it doesn't actually seem to matter much as long as you are consistent when you use the model. timm model metadata includes this info.

I posted stats on the timm model mean/std counts to a similar question about normalization on twitter https://x.com/wightmanr/status/1798156340579471478

And yeah, it's technically standardization but has always been called normalization. Negative values are not an issue.

You could also roll you own min/max norm, but would have to train/fine-tune model yourself. Can be useful for certain real world imaging applications but should limit/clamp.