Missing Normalization of Input/Output in SAE Implementation

[tatsuropfgt]

Scaling and evaluating sparse autoencoders(Gao et al. 2024) mentions that normalization is applied to the input/output of SAEs in 2.1 Setup.

We subtract the mean over the d_model dimension and normalize all inputs to unit norm, prior to passing to the autoencoder (or computing reconstruction errors)."

However, I could not find such an implementation within this repository.
I would like to understand whether this is a mistake in this implementation, if there is a specific reason for not performing the normalization, or if I might have overlooked something.

Thank you.