I liked the distinction between torch.compile cleaning up the operations you already wrote and FlashAttention changing the math itself. It’s a helpful reminder that a compiler can speed up a path without necessarily finding a better path.