Natural Language Processing

Study notes on NLP fundamentals, Transformer architectures, and modern attention mechanisms.

Child Notes

• Transformers — Notes on the “Attention Is All You Need” architecture, encoder-decoder structure, and advantages over RNNs.
• Hybrid Attention Mechanism Options — Deep technical comparison of Kimi Delta Attention (KDA) and DeepSeek Sparse Attention (DSA), including mathematical formulations and PyTorch implementations.