Relations don't fit in tables. Look at two tables: a user table where each row is a person, and a friendship table where each row is a pair of IDs. Traditional machine learning gets the first one and treats each person as an independent sample—but what you want to predict lives in the second. Who connects to whom matters more than what someone looks like on their own.
A table can hold the attributes of ten thousand people, but not the relations between them. Social networks, molecular structures, citation networks, road networks, knowledge bases—all share one shape: a graph. When data has connections and the connections themselves carry information, you need a graph.
There's an open-source course that turns this idea into 90 minutes. The repo is ai-engineering-from-scratch by Rohit Ghumare: 58,403 stars, top 100 among GitHub's AI repositories. The full course has 523 lessons across 20 phases, roughly 342 hours, from linear algebra to agent swarms. Graph theory sits at Phase 1, Lesson 21—between Fourier transforms and stochastic processes. The last few bricks of the mathematical foundation.
The 90-Minute Route
No libraries allowed. Everything hand-written.
Step one: write a Graph class. Adjacency list for neighbors, plus a function converting the table to an adjacency matrix: A[i][j] = 1 means an edge between i and j. That single step turns relations into a matrix. From that moment, connections can be multiplied, decomposed, computed.
Then BFS and DFS—one uses a queue, the other a stack. The plainest traversals.
The real show starts with the third matrix. The degree matrix D, with each node's connection count on the diagonal. The Laplacian:
L = D − A
One subtraction puts "how connected" and "connected to whom" into a single matrix.
Why that's valuable: the number of zero eigenvalues of L equals the number of connected components—no full traversal needed, the spectrum confesses on its own. The smallest nonzero eigenvalue, the Fiedler value, measures overall connectivity and knows where bottlenecks are. Cut the Fiedler vector by sign and the graph splits in two. That's spectral clustering: one eigendecomposition, no iterative optimization.
The final stop: GNNs. Graph convolution in a neural network is, unpacked, each node collecting neighbor messages, averaging them, passing through a linear transform and an activation. In matrix form:
H' = σ(A_norm · H · W)
where A_norm is the row-normalized adjacency matrix. One matrix multiplication. The GCN formula from the paper—A with a hat—just adds self-loops so nodes don't forget themselves when aggregating; the rest is unchanged.
Run K rounds and every node sees its K-hop neighborhood. Each round, each node reads only its own row of the matrix.
Why It Matters
"Read only your own row" is the entire temperament of the GNN. It doesn't stuff the whole graph into context; whoever needs something looks up their own row. Stuffing and looking up are two different cost structures. Once you internalize this, a lot of current systems become easier to read: knowledge graphs in retrieval-augmented systems, state-machine graphs in agents, ontology graphs in semantic layers—different appearances, same floor: adjacency matrix, degree matrix, Laplacan. The speed of traversing a graph depends not on its size but on whether you fetch only the cell you need.
One more layer: spectral clustering needs no iteration; eigenvalues need no traversal. The graph's structure was always in the matrix—not to be searched, only computed. Many "structure was lost" incidents didn't lose the structure itself; they put it in a container that couldn't hold it.
The Course's Own Methodology
More interesting than the content is the learning protocol. Five steps per lesson: restate the core idea in your own words; type the code yourself (no treating code blocks as decoration); run the commands; keep evidence—commands, working directory, exit codes, outputs, artifacts you modified; finally, you may move on only if you can explain the output and change one thing without guessing.
"Keep evidence" sounds odd in a math course, yet it's exactly right. The acceptance criterion is "can explain, can modify," not "finished watching." Watching doesn't count as learning—the hardest gate of the five. The repo also cleanly separates two implementations: a networkx version for production, the hand-written one for understanding—"use your from-scratch implementation to understand what it does." In an era when content itself is free, the remaining value of a course is these two things: build it yourself, and leave evidence.
Passing Through Customs
The 90 minutes were verified item by item. L = D − A: correct. Zero-eigenvalue counting: correct. Fiedler value and vector: correct. GCN formula: correct (Kipf & Welling, 2017). Karate club graph, 34 nodes / 78 edges: correct—Zachary's real 1977 data. PageRank damping 0.85: correct; the stationary distribution of the random walk proportional to degree holds for undirected graphs: correct.
Two loose spots. One claims GNNs are the fastest-growing area of deep learning—a marketing line with no source. One says the Fiedler vector gives the optimal cut; strictly it's a relaxation-based approximation of the minimum cut problem, not exact—though for k > 2 the course itself reminds you to switch to k-means. In 90 minutes, only these two sentences need a squint.
A final exercise worth stealing: take the karate club graph, compute the degree distribution, Laplacian eigenvalues, and spectral clustering, and compare against the known ground truth. Ninety minutes ago you hadn't written an adjacency list; now you're reproducing a classic sociology result. The next lesson is stochastic processes, with five hundred lessons beyond. For laying foundations, sixty thousand people are in line.
---
- Original: github.com/rohitg00/ai-engineering-from-scratch (MIT, 58,403 stars, 523 lessons / 20 phases / ~342 hours)
- Graph theory lesson:
phases/01-math-foundations/21-graph-theory/docs/en.md(90 minutes, Build type) - Verification boundary: lesson content checked line by line against the original; author background and repo stats from the GitHub page and external search (a third-party top-100 AI repo snapshot showed 54,987 stars in September; now 58,403). The course was not executed; code was verified by reading.