Created in June 20, 2025
2025
Your sliding window size of local attention can be reduced to 512, see preprint.