Expected entropy for samples from a Dirichlet distribution

Samples P = <p1, p2, ..., pk> from Dirichlet distribution with parameter alpha = <alpha1, alpha2, ..., alphak> can be treated as categorical probability distributions with entropy \(H(P) = \sum(-p \log p)\). This function calculates the expected entropy \(E[H(P)]\) given alpha.

Usage

expected_entropy(alpha, k = NULL)

Arguments

alpha: Dirichlet parameter
k: If length(alpha) is 1, number of components in symmetric Dirichlet distribution

Value

Expected entropy \(E[H(P)]\) in bits (log2 scale)

Details

After https://math.stackexchange.com/questions/2266285/expected-entropy-based-on-dirichlet-distribution/3195376#3195376

Examples

alpha = peak_alpha(50, 1)
set.seed(1357)
rdirichlet(500, alpha) |>
  apply(1, entropy) |>
  mean()
#> [1] 0.4568703
expected_entropy(alpha)
#> [1] 0.445642