Neural Networks Book
Every Neural Architecture, Rigorously Drawn
A 119-page LaTeX book covering all major neural network families — each with exact equations, pseudocode training algorithms, and native TikZ figures.
119
pages
13
chapters in 5 parts
256
numbered equations
42
native TikZ figures
26
pseudocode algorithms
36
cited bibliography entries
Overview
Artificial Neural Networks — Methods, Equations and Graphical Representations is a complete, self-contained book spanning the field from Rosenblatt's 1958 perceptron to 2024's Kolmogorov–Arnold networks. It is built on one strict organizing principle: every architecture gets rigorous equations, an estimation or training algorithm in pseudocode, and a faithful graphical representation.
Every one of the 42 figures is drawn natively in TikZ/pgfplots — no imported images — so each diagram is exactly as precise as the equations it illustrates. Notation is unified across all 13 chapters: bold lowercase vectors, bold uppercase matrices, the Hadamard product, and a shared color palette for inputs, hidden units, outputs, gates and memory.
Coverage runs through five parts: foundations and learning (backpropagation, Adam, regularization), core architectures (CNNs, LSTMs, Transformers, ViT, MoE, Mamba), graphs and energy (GCN, GAT, GIN, Hopfield, RBMs), generative models (VAEs, GANs, normalizing flows, DDPM diffusion), and specialized architectures from spiking networks to Neural ODEs and KANs.
Key Features
One principle, applied everywhere
Every architecture is presented as equations plus a training algorithm plus a faithful figure — from the perceptron to Kolmogorov–Arnold networks.
100% native TikZ figures
All 42 diagrams are drawn in TikZ/pgfplots with zero imported images, so figures carry the same precision as the mathematics.
Unified notation throughout
Bold vectors and matrices, Hadamard products and a shared color palette are defined once in main.tex and reused by every chapter.
Full deep learning canon
Backpropagation's three equations, Adam with bias correction, ResNet gradients, the six LSTM equations, and scaled dot-product attention with the variance argument.
Modern architectures included
Vision Transformers, Mixture of Experts with noisy top-k routing, state-space models and Mamba's selective scan, and scaling laws.
Graphs, energy and memory
Message passing, GCN's spectral derivation, GAT, GIN with the 1-WL expressiveness theorem, Hopfield energy descent, and contrastive divergence for RBMs.
Generative models in depth
The VAE ELBO with closed-form Gaussian KL, GAN minimax theory through WGAN, RealNVP flows, and DDPM with both training and sampling algorithms.
Emerging and bio-inspired frontiers
Spiking neurons with STDP and surrogate gradients, self-organizing maps, echo state networks, capsule routing, Neural ODEs and Neural Turing Machines.
Clean by construction
The three-pass pdflatex build finishes with zero errors, zero undefined references and zero unresolved citations.
How It Works
Part I — Foundations and Learning
Perceptron with the Novikoff convergence theorem, backpropagation, the optimizer family up to Adam, initialization, and the full regularization toolbox.
Part II — Core Architectures
CNNs with a worked convolution grid, RNN/LSTM/GRU with BPTT, the complete Transformer encoder–decoder, and modern variants from ViT to Mamba.
Part III — Graphs, Energy and Memory
Graph neural networks (GCN, GraphSAGE, GAT, GIN) and energy-based models from Hopfield networks to RBMs and deep belief networks.
Part IV — Generative Models
Autoencoders, VAEs, GANs, normalizing flows, DDPM/DDIM diffusion with classifier-free guidance, and WaveNet's dilated causal convolutions.
Part V — Specialized and Emerging
Biologically inspired networks — spiking neurons, SOMs, reservoir computing — plus capsules, Neural ODEs, Neural Turing Machines and KANs.
Tech Stack
Typesetting
Build
Highlights
- Spans 66 years of the field in one consistent framework — from the 1958 perceptron to 2024 Kolmogorov–Arnold networks.
- Every figure is code: 42 TikZ/pgfplots diagrams and not a single imported image.
- 26 estimation algorithms in pseudocode, including both the training and sampling procedures for diffusion models.
- All 36 bibliography entries are actually cited in the text — no padding.
- Includes rigorous theorem-level results: universal approximation, Novikoff convergence, Hopfield energy descent, and 1-WL expressiveness for GINs.
- The compiled PDF ships in the repository alongside the full LaTeX source.
Explore Neural Networks Book
ANN methods, equations & figures — the full source is on GitHub.