So I guess to whom it may concern, perhaps @shunting314, it does appear that inference needs split online softmax. Versions. Collecting environment information... PyTorch version: 2.7.0+cu128 Is debug ...