Jason Xu
Generative AI
From early probabilistic language models to today's transformers.
Generative AI has roots in early probabilistic language models like Markov chains and mid-20th-century theories of formal grammars. It evolved through computational linguistics and foundational neural networks to today's deep-learning transformer models — systems that can now autonomously produce coherent and creative content: text, images, and code.
History
From Markov chains to transformers.
Prompt engineer
Techniques, chains, and iterative refinement.
AIs
Custom GPTs and research assistants I've built.
Agent skills
What happens when a model gets tools.
A very short history
Early probability
Markov chains model sequences of symbols — the seed of statistical language modeling.
Formal grammars
Chomsky's hierarchies gave us a theory of what languages are — and what machines can generate.
Neural foundations
RNNs, LSTMs, and word embeddings made meaning computable.
Transformers
Attention is all you need — and generative AI became a household phrase.
