What Neural Networks Learn

A neural network learns by adjusting numerical parameters so that its outputs better satisfy a training objective. It does not store a neat textbook of rules inside each unit. The learned parameters define a function that transforms inputs through a sequence of layers. During training, optimization changes those parameters in response to error signals. What emerges can support accurate predictions, but the internal organization is distributed across many units and shaped by the task the network was rewarded for performing.

Layers are often described as learning a hierarchy of features. In an image model, early computations may respond to local contrasts or orientations, while later computations combine information across larger regions. In a language model, representations can encode patterns involving words, syntax, topics, and longer contexts. This description is useful but incomplete. A unit may respond to several unrelated patterns, and a meaningful concept may be represented across many units. The same architecture trained on different data or objectives can organize information differently.

The training objective determines which distinctions are useful. If a model is trained to predict an outcome, it may discover any regularity that reduces error, including shortcuts people did not intend. An image classifier can rely on backgrounds, watermarks, or camera artifacts. An educational model can rely on school identifiers, response opportunities, or patterns of missing data instead of the learner process designers hoped to measure. The network has no independent commitment to causal explanation, fairness, or educational meaning. It follows the statistical incentives present in its data and objective.

Interpretability methods provide partial views. Feature visualization can generate inputs that strongly activate a unit or direction. Attribution methods estimate which parts of an input influenced an output. Probes test whether certain information can be recovered from a representation. These tools can reveal patterns and support hypotheses, but they do not automatically explain the complete computation or establish why a model behaves as it does in every case. Human-readable images and labels can also tempt observers to see a clean concept where the representation is more mixed.

For educational use, the practical question is not whether a network thinks like a person. It is whether the learned function remains valid for the intended learners, settings, and decisions. Teams should test plausible shortcuts, compare subgroups, examine failures, and evaluate changes over time. Repeated evaluation with new cohorts can show whether a representation remains useful or has become tied to local conditions that no longer hold. Teachers should receive evidence about capabilities and limits rather than anthropomorphic claims about understanding. Neural networks can learn powerful representations, but those representations are task-oriented products of data, architecture, objectives, and optimization. Their usefulness must be demonstrated through independent evidence and accountable use.