Infrastructure · Primitives
JensenShannonDivergence#b3b4
A symmetric, bounded distance between two distributions, built from their mixture
Mechanism
The Kullback-Leibler divergence D_KL(P||Q) = sum_i p_i log(p_i/q_i) measures the information lost when Q approximates P, but it is asymmetric and infinite wherever q_i = 0 and p_i > 0, which empirical histograms with unseen states often produce. The Jensen-Shannon divergence fixes this by comparing both distributions to their mixture M = (P+Q)/2: D_JS = 1/2 D_KL(P||M) + 1/2 D_KL(Q||M). Equivalently D_JS = ShannonEntropy(M) - (H(P)+H(Q))/2. With log base 2 it is symmetric, always finite and bounded in [0, 1], and zero only when P = Q. Typical use: compare an empirical distribution with a null-model distribution over the same state space, so the value reads as distance from the baseline.
Invariants
- D_JS(P,Q) = D_JS(Q,P)
- 0 <= D_JS <= 1 with log base 2
- D_JS = 0 if and only if P = Q
- Both distributions are defined over the same state space
Failure modes
- Comparing distributions built on different state spaces or binnings
- A non-monotonic response: distance from a central reference rises for deviations in either direction (e.g. attraction and repulsion both raise D_JS from a uniform baseline), so the value alone does not say which way the system deviates
- Small samples inflate D_JS between two draws of the same process
Where this came from
- Thesis Kevin Do, PhD thesis on ant collective behaviour (NCSU, 2026), Ch. 1.3.3, Eqs. 1.10-1.11; Ch. 3.1.1, Eqs. 3.2-3.3, Fig. 3.3
Builds on
Used by
Identity
The name is the hash of the definition. Change a word and it becomes a different pattern.
- Sema ID
sema:JensenShannonDivergence#mh:SHA-256:b3b4314fee746b1e97b133372bcdb1d9dd1a3a98f90fd19ca19bb47dfef04a6b- Reference
JensenShannonDivergence#b3b4- Ring
- 0
- Tier
- 0