Datadog research recently released Toto-2.0, their new time series model. The model features some unique properties compared to its previous version Toto-1.0: Contiguous Patch Masking (CPM) replaces autoregressive decoding with a single parallel forward pass. Arcsinh normalization keeps small fluctuations visible while compressing extreme spikes - perfect for sparse data. NorMuon optimizer handles the sign-valued gradients of pinball loss far better than AdamW. u-µP hyperparameter transfer tunes settings once on a 10M proxy model and reuses them across all 5 target sizes. Full discussion and tutorial about the model here submitted by /u/nkafr
Originally posted by u/nkafr on r/ArtificialInteligence
You must log in or # to comment.