Paper Thoughts

Basic summaries of academic papers that have shaped my thinking. Note - I am not an expert in AI - the notions discussed in these Paper Thoughts reflect my personal thoughts and I write them up as a way to improve my own understanding of the content.

Belief Dynamics Reveal the Dual Nature Of In-Context Learning And Activation Steering

Eric Bigelow et al., 2025 | Reviewed November, 2025

This paper presents a unified Bayesian framework for understanding two seemingly different methods of controlling language model behavior.

Emergent Introspective Awareness in Large Language Models

Jack Lindsey, 2025 | Reviewed November, 2025

This paper begins a discussion on the ability of LLMs to introspect on their internal states.

Attention Is All You Need

Vaswani et al., 2017 | Reviewed December 8, 2025

This groundbreaking paper introduced the Transformer architecture, which has become the foundation of modern natural language processing. The paper proposes a novel approach that relies entirely on attention mechanisms, dispensing with recurrence and convolutions entirely.