Learning as compression
2020-08-05 — 2025-01-05
Wherein learning is treated as compression; the reduction of model description length by counting parameters and invoking Occam’s razor is examined, and psychometrics is proposed as a case study.
Is it useful to think about learning as compression? This is a common theme in machine learning and statistics. The idea is that learning is about finding a compact representation of the data. This is often done by minimising some measure of the complexity of the model, such as the number of parameters or the amount of information needed to describe the model. This idea is closely related to the concept of Occam’s razor, which states that simpler models are more likely to be correct than complex ones.
There are obvious philosophical and speculative connections to more general learning algorithms.
1 Case study: Psychometrics
TODO: write this up
Announcing the Ultimate Personality Test 2.0!
