Echoes of Memory: How Case-Based Reasoning Weaves a Network of Knowledge Like a Sage That Never Forgets
*Full English translation of a zhichai.net forum post introducing Case-Based Reasoning (CBR).*
Introduction: From Seeds of Cognition to a Forest of Wisdom
Imagine you are an experienced archaeologist facing an ancient ruin. Instead of drawing a map on a blank canvas, you awaken past excavation experiences from memory—similar clues guide you forward like lighthouses. This is the essence of Case-Based Reasoning (CBR): a computational paradigm originating from cognitive science that uses already-stored knowledge to solve new problems while capturing fresh experiences and making them immediately usable—like a brain building the future from recollection. CBR is not just a problem-solving tool; it is an engine for incremental learning and knowledge discovery, capable of extracting general knowledge (case classes, prototypes, higher-order concepts) from new experience.
CBR was first proposed by the cognitive science community to build computer models that imitate human cognition. It is particularly suited to domains where generalized knowledge is missing, reasoning by retrieving similar cases rather than relying on rules or formulas. For readers unfamiliar with AI, think of CBR as the brain's "experience replay" mechanism: when facing a new problem, it doesn't reinvent the wheel but recalls past memories and refines them—making CBR exceptionally powerful in dynamic environments.
Everything starts with a simple definition: a case F is formalized as a triple (P, E, L), where P is the problem description, E is the explanation of the solution, and L is the problem solution. P is usually summarized as attributes or features—numeric, symbolic, even images or sequences. Through knowledge acquisition, we extract the most predictive attributes to ensure retrieval of the most similar cases. CBR can improve classification by modifying the vocabulary V (attributes, features), the case base CB, or the similarity measure sim—or combinations of these. This flexibility comes from Richter's concept of knowledge containers: vocabulary, similarity measure, solution transformation, and the cases themselves. The first three are compiled knowledge—stable and reliable; cases are experiential knowledge—newly added cases are immediately usable, helping the system cope with dynamic knowledge.
The Symphony of Knowledge Containers
Initially, a CBR system may use simple vocabulary, a crude similarity measure, no solution-transformation knowledge, and a large number of cases. Over time, the vocabulary gets refined, the similarity measure fits the domain well, and the number of cases may shrink—improvements in other containers let the system discriminate better. It's like a novice musician starting from basic scales, refining the score through practice, until fewer notes produce a symphony. Knowledge transfers between containers to keep the system efficient—for example, in medical diagnosis, simple features (symptom lists) may be refined with advanced transformations (probabilistic adjustments), reducing redundant cases while improving accuracy.
CBR also shrinks the case base by abstracting cases into more general forms (concepts, prototypes, case classes) or learning higher-order relations between cases, speeding up retrieval and improving robustness to noise. Abstracting concrete medical cases into prototypes is like extracting the essential pattern of "viral infection" from many cold cases, helping doctors quickly identify variant symptoms. CBR is thus not only a performance tool but a knowledge-acquisition method that helps learn domain theories.
Humorously, the main difficulty in developing CBR systems is like asking a philosopher: "What constitutes a case?" Other challenges include: what is the right similarity measure? How to efficiently organize large numbers of cases? How to acquire and refine new cases? How to generalize specific cases to broader contexts?
The Many Faces of Similarity
Humans love reasoning by similarity but struggle to express it formally—similarity seems incoherent. Cognitively, similarity is multidimensional: a red bicycle and a blue bicycle are similar in the concept "bicycle" but opposite in color. Like viewing an impressionist painting, different angles yield different conclusions. Overall similarity, identity, and partial similarity must be modeled by flexible control strategies—especially in image databases where content can be viewed from multiple perspectives. A system must computerize dialogue strategies to determine the perspective and apply the appropriate similarity: a product recommender must distinguish "similar-style clothing" from "similar-color accessories."
Mathematically, the Minkowski metric is the most common similarity measure for technical problems:
The choice of p depends on how differences are weighted. The metric satisfies symmetry, identity, and inequality properties. For two one-dimensional curves—say, a real robot axis-path signal versus one reconstructed from compressed data—p=2 (Euclidean) averages all data points but emphasizes large differences; p=1 (city-block) weights large and small differences equally; p=∞ (maximum norm) ensures no difference exceeds a predefined value. In the robot example, this means the axis moves smoothly with known deviation rather than sudden ramps. The domain itself provides insight into metric choice—like a chef choosing knives based on ingredients.
Most applications lack prior knowledge, so metrics are selected by trial against quality criteria. For classification problems, error rate is the key criterion; otherwise, one measures clustering of similar objects and separation between groups. Distance metrics are classified by data type: numeric, symbolic, structural, and mixed. Invariances to translation, size, scaling, and rotation matter for technical systems—satellite image analysis relies on rotation invariance to match terrain regardless of shooting angle.
Real applications are complex, with multiple attributes: sensor-value attributes and categorical attributes describing meta-knowledge. These form partial similarities Sim1, Sim2, …, Simn, computable with different measures and each with its own meaning. Overall similarity can be a simple weighted sum: Sim = w1 Sim1 + w2 Sim2 + ... + wn Simn. Learning attribute weights turns global similarity into local similarity, allowing application-specific metrics—methods include linear or stochastic optimization, heuristic search, genetic programming, case ordering, or query ordering in NN classification. Learning distance functions from user feedback is called relevance feedback, popular in database and image retrieval. Bobrowski et al. introduced linear transformation of features for learning distance functions.
The Semantic Scale from 0 to 1
Normalizing similarity values to [0, 1] makes them comparable: 0 means identical, 1 means unequal, 0.5 is neutral, 0.5–0 means more similar, 0.5–1 less similar—like the markings on a thermometer. The common normalization uses upper and lower bounds of feature values. But when the case base is incomplete and cases are collected incrementally, the bounds xmin,i,k and xmax,i,k are based on the finite set at time tk and may not match true bounds—so the scale changes over time, causing inconsistent decisions between times tk and tk+l. Incremental CBR systems need different normalization considerations.
Like a growing child whose judgment of the world's boundaries adjusts with experience, CBR must ensure decision consistency. For example, in a stock-prediction system, initial price bounds are based on limited history; as new trades flow in, normalization updates dynamically, preventing early bias from affecting long-term judgment.
The Architect of Memory
The case base is the heart of a CBR system. Ideally, CBR starts reasoning from empty memory, with capability emerging from incremental learning of processed cases. Unlike database retrieval and nearest-neighbor classification, CBR focuses on conceptual memory structures. k-d trees are spatial partitioning structures for k-dimensional points; conceptual structures are directed graphs where the root covers all input instances and terminal nodes are single instances. Internal nodes represent super-concepts, generalized from associated instances—prototypes, medoids, or user-selected instances. A concept C (a class) is represented by an abstract description (feature names and values) and a list of pointers to sub-concepts M(C) = {C1, ..., Cn}.
Explicit conceptual representation of each node aids human understanding of the domain—in a legal system, a concept hierarchy can subdivide "contract disputes" into "lease contracts" and "sales contracts," each node containing typical cases.
k-d tree construction needs only split and delete; concept learning uses more complex operations: split, merge, add, delete—driven by evaluation functions that are statistical or similarity-based. These operations make the concept hierarchy insensitive to sample order, allowing incremental addition via reorganization. k-d trees lack this flexibility, though adaptive k-d trees exist.
Generalization and abstraction make the case base robust to noise and applicable to broader problems. Conceptual incremental learning places the case base within Schank's dynamic memory view: memory structures must adapt as new information changes memory. Other work bridges implicit and explicit case representations: when data are plentiful, statistical learning (MDL or MML) summarizes them into models; otherwise cases are kept in the case base. The case base controls statistical model learning via hierarchical organization, preventing overfitting and outlier influence. Like human short- and long-term memory, CBR balances implicit models and explicit cases—in weather prediction, common patterns are computed quickly by statistical models, while extreme events rely on recalling specific cases.
Applications: Meta-Learning to Novelty Detection
CBR has been successfully applied to signal interpretation, medicine, geographic information systems, biotechnology, and climate research. Key frontiers: meta-learning for parameter selection, image and signal interpretation, prototype-based classification, and novelty detection.
Meta-Learning for Parameter Selection
Meta-learning applies learning algorithms to machine-learning experiment metadata, aiming to understand how learning can flexibly solve different problems and improve existing algorithms. An important but understudied task is parameter selection for data/signal-processing algorithms—Soares et al. applied it to SVM kernel-width selection; Perner and Frucci et al. to image segmentation. The problem is formalized as: given a signal characterized by signal attributes A and domain attributes B, find processing-algorithm parameters ensuring best output quality: f : A ∪ B → P_i, where Pi is the i-th parameter class for a given domain. CBR's incremental behavior suits many signal-processing problems where signals arrive incrementally rather than all at once.
Image Interpretation
Image interpretation maps a numerical image representation to a logical representation suited to scene description, through preprocessing, segmentation, analysis, and interpretation. Preprocessing and segmentation algorithms typically need multiple parameters tuned per image. Low-level features (lines, edges, ribbons) are mapped to high-level/symbolic features. The interpretation component identifies objects by matching symbolic descriptions against object models/concepts stored in a knowledge base. Most systems use bottom-up control with no feedback; hybrid bottom-up/top-down control allows returning to earlier components to refine outputs. CBR applies at all levels: meta-learning for preprocessing/segmentation parameters, classification for feature mapping, and memory organization for the interpretation unit. Ideal systems start from few samples, learn new cases in use, and update memory incrementally—raising the CBR system lifecycle: over long-term use, case structure may change, obsolete solutions disappear, and methodologies/software architectures must handle this without rebuilding the whole system.
Incremental Prototype-Based Classification
Prototype usage is popular in medical applications, knowledge management, and image classification. Related issues: with a large enough case set, prototypes can be computed automatically as generalizations of similar case sets; in medical and image-catalog applications, humans select or describe them. Since human-selected prototypes may be biased, redundancy must be checked before admission. A minimal prototype-based classification system should implement: classification via a suitable similarity measure, redundancy-reducing prototype selection, feature weighting to determine prototype feature importance and learn the similarity measure, and feature-subset selection. Medical image interpretation systems [48] combine statistical and AI methods; handwriting recognition systems incrementally adapt to users' writing styles; k-NN on data streams uses multi-resolution array structures with case-forgetting strategies for concept drift.
Novelty Detection
Novelty detection—identifying inputs that differ in some way from previous inputs—is valuable when important classes are underrepresented and classifiers cannot be trained reliably; common in information management, medical diagnosis, fault monitoring, and visual perception. The proposed model combines statistical and similarity-based reasoning: a set of statistical models learned offline from observation sets, each representing a case class, with probability density functions implicitly representing data and modeling uncertainty. Bayesian decision rules and mixture models serve as the novelty detector, with models updated based on new observations via MDL or MML learning principles. If the model library cannot classify an event into a case class, it is flagged as novel and passed to the similarity-based reasoning unit, which incorporates it into the case base—using fuzzy similarity measures to model data uncertainty. A case-base maintenance unit advises when to learn new models; the statistical unit verifies via MML whether a new model is warranted. This combination of statistical and similarity-based inference handles both well-represented and rare events—like a space explorer detecting unknown planets, or an autonomous vehicle identifying a sudden roadblock and learning an avoidance strategy.
Conclusion
CBR solves problems with stored knowledge while capturing new knowledge for immediate use—it is a method for incremental learning and knowledge discovery. We covered the CBR process scheme, similarity, memory organization, and open topics: meta-learning for parameter selection, image interpretation, incremental prototype-based classification, and novelty detection, bridging CBR and statistical communities. In this conception, CBR is the cornerstone of intelligent, flexible, robust data-interpretation systems inspired by human case-based processes. Practically, CBR succeeds in medicine, technical diagnosis, image interpretation, GIS, text retrieval, e-commerce, and user support. Like an immortal sage drawing wisdom from past memory toward the future, CBR's potential awaits further exploration.
---
References
1. Perner, P. (2019). Case-Based Reasoning – Methods, Techniques, and Applications. In: Nyström, I. et al. (Eds.), CIARP 2019, LNCS 11896, pp. 16–30. Springer. 2. Richter, M. (1998). Introduction to case-based reasoning. In: Lenz, M. et al. (Eds.), Case-Based Reasoning Technology: From Foundations to Applications. LNAI 1400, pp. 1–16. Springer. 3. Aha, D.W. (1991). Case-based learning algorithms. In: Proceedings of the DARPA Case-Based Reasoning Workshop, pp. 147–158. 4. Perner, P. (2008). Case-Based Reasoning on Images and Signals. Studies in Computational Intelligence, vol. 73. Springer. 5. Schank, R.C. (1982). Dynamic Memory: A Theory of Reminding and Learning in Computers and People. Cambridge University Press.