Hierarchical clustering dendrogram for portfolio construction

The Math Behind Smarter Diversification

Most optimizers fail the moment their assumptions meet real markets. Our team walks through why — and what Hierarchical Risk Parity does differently.
Classical mean-variance optimization looks precise on paper. Feed it a covariance matrix, crank the solver, and out comes a weight vector. The problem is that the solver treats estimation error as if it were signal, and small changes in inputs can produce wildly different portfolios. Hierarchical Risk Parity sidesteps this by using graph theory and hierarchical clustering to read the structure of a correlation matrix directly — no matrix inversion required. Our team takes you through the full three-step algorithm, the mathematics that support it, and the conditions under which it holds up better than classical alternatives.

The Three-Step HRP Algorithm, Explained in Order

The wrong way to teach HRP is to show the dendrogram first and the weights last, as if the visual output were the point. Our team starts where the algorithm starts — with a raw correlation matrix, a problem of structure detection, and a sequence of decisions that a practitioner can follow, verify, and reproduce. Each of the three steps below addresses one specific limitation of classical optimization, in the order the algorithm actually encounters them.
Hierarchical tree clustering of asset correlations

Step One — Tree Clustering

Begin with the correlation matrix. The goal of this step is to detect which assets move together and which move independently — not by assuming a factor structure, but by letting the data reveal its own geometry. Hierarchical agglomerative clustering groups assets iteratively, merging the most similar pair at each step until a single tree is formed. The choice of linkage method — single, complete, or average — determines how similarity between clusters is measured as they grow. Our team works through each linkage variant and explains why the choice matters for the shape of the resulting tree and, downstream, for the weight distribution. The output of this step is a dendrogram: a tree that encodes the full correlation hierarchy of the asset universe, without discarding any structural information.

Quasi-diagonal reordering of asset correlation matrix

Step Two — Quasi-Diagonalization

With the dendrogram in hand, the next step reorders the rows and columns of the covariance matrix so that the most similar assets appear adjacent to one another. This is called quasi-diagonalization, and it produces a matrix whose block structure is visible to the eye — correlated clusters appear as dark squares along the diagonal, and dissimilar groups are pushed toward the corners. The reordering does not discard information or impose a factor model. It simply reorganizes what is already in the data into a form that makes the structure legible. This matters because the next step — recursive bisection — relies on this ordering to split the asset universe into groups that make economic sense, rather than arbitrary subsets. Our team shows the permutation procedure explicitly and connects it to the graph-theoretic concept of minimum spanning tree traversal.

Recursive bisection allocating weights across asset clusters

Step Three — Recursive Bisection

The final step assigns portfolio weights. Starting from the full ordered asset list, the algorithm splits it into two halves — left cluster and right cluster — and allocates capital between them in inverse proportion to their respective variance. It then recurses into each half, splitting again and allocating again, until every individual asset has received a weight. At no point does this procedure require inverting the covariance matrix. That is the structural advantage over mean-variance optimization: the weights are determined by the relative variance of sub-clusters, not by the eigenvectors of a matrix that may be poorly conditioned. The result is a portfolio whose diversification follows the actual correlation structure of the data, rather than the artifacts of an unstable numerical solver. Our team traces the full bisection procedure with a worked numerical example so the logic of each allocation step is explicit.

HRP vs. Classical Allocation Approaches

Key Feature Cenisadteorvyosa Mean-Variance Optimization Equal Risk Contribution
No matrix inversion required
Stable under estimation error
Handles large asset universes naturally
Reflects true correlation hierarchy
Does not require expected return inputs
Produces intuitively diversified weights
Robust out-of-sample Sharpe profile
Why HRP Works

Four Properties That Make HRP Structurally Sound

Our team chose to document HRP not because it is fashionable but because its structural properties hold up where classical methods quietly break down.

No Matrix Inversion at Any Step

Classical optimizers invert the covariance matrix, amplifying estimation error into the weights. HRP never performs this operation — weights emerge from recursive variance comparisons instead, removing the primary source of instability.

Graph Theory Reads the Data's Own Structure

Rather than imposing a factor model, HRP uses hierarchical clustering to let the correlation matrix reveal its own groupings. The resulting tree reflects what is actually in the data, not what the analyst assumed in advance.

Scales Gracefully to Large Universes

Mean-variance optimization degrades rapidly as the number of assets grows, because the covariance matrix becomes harder to estimate reliably. HRP's clustering approach handles large universes without requiring a well-conditioned matrix as a prerequisite.

Diversification Follows Economic Logic

Because weights are assigned along the branches of a correlation tree, assets that are genuinely similar receive allocations that reflect their shared risk. The resulting portfolio is diversified in a way that corresponds to the actual structure of the data, not to a mathematical artifact.

Stronger Out-of-Sample Behavior

Published research by López de Prado and subsequent peer review consistently shows that HRP produces better out-of-sample Sharpe profiles than classical optimizers across a range of asset universes and historical periods — a direct consequence of its lower sensitivity to input estimation error.

Clustering

4
Hierarchical Agglomerative Clustering

A bottom-up clustering procedure that begins with each asset as its own cluster and iteratively merges the two most similar clusters until a single tree structure — the dendrogram — encompasses the entire asset universe. The similarity measure used at each merge step is determined by the chosen linkage method.

Linkage Method
The rule used to compute the distance between two clusters when deciding which pair to merge next in hierarchical clustering. Common choices include single linkage, which uses the minimum pairwise distance; complete linkage, which uses the maximum; and average linkage, which uses the mean. The choice affects the shape of the resulting dendrogram and, consequently, the portfolio weight distribution.

Dendrogram

A tree diagram produced by hierarchical clustering that encodes the full similarity structure of an asset universe. Each leaf represents an individual asset, and each internal node represents a merger event. The height of each node reflects the dissimilarity at which that merger occurred. The dendrogram is the primary output of Step One in the HRP algorithm.
Minimum Spanning Tree

A graph-theoretic construct that connects all assets in a universe using the minimum total edge weight, where edge weights represent pairwise distances derived from correlations. The minimum spanning tree is closely related to the dendrogram and is used in the quasi-diagonalization step to determine the optimal reordering of the covariance matrix.

Matrix Methods

4

Covariance Matrix

A square symmetric matrix in which each entry represents the covariance between two assets in a portfolio. The diagonal entries are the individual asset variances. The covariance matrix is central to all risk-based portfolio construction methods, but its reliable estimation requires a sufficiently long data history relative to the number of assets.

Quasi-Diagonalization

The second step of the HRP algorithm, in which the rows and columns of the covariance matrix are reordered according to the leaf sequence of the dendrogram. The result is a matrix in which similar assets appear adjacent, producing a visible block structure along the diagonal that makes the correlation hierarchy legible and prepares the matrix for recursive bisection.
Estimation Error
The discrepancy between the true population covariance matrix and the sample covariance matrix estimated from historical data. Estimation error is the primary cause of instability in mean-variance optimized portfolios, because small errors in the input matrix can produce large, economically implausible changes in the output weights. HRP's avoidance of matrix inversion substantially reduces sensitivity to this source of error.

Ill-Conditioned Matrix

A matrix whose condition number — the ratio of its largest to smallest eigenvalue — is very large, indicating that it is close to singular. Ill-conditioned covariance matrices produce numerically unstable results when inverted, which is a common failure mode of classical portfolio optimizers applied to large asset universes or short historical samples.

Recursive Bisection

The third and final step of the HRP algorithm, in which portfolio weights are assigned by repeatedly splitting the ordered asset list into two halves and allocating capital between them in inverse proportion to their cluster variance. The procedure recurses until every asset has received a weight, producing a fully allocated portfolio without any matrix inversion.

Inverse Variance Weighting

A weighting scheme in which each asset or cluster receives a weight proportional to the reciprocal of its variance, so that higher-variance assets receive smaller allocations. In the HRP recursive bisection step, inverse variance weighting is applied at each split to determine how capital is divided between the two sub-clusters.

Risk Parity

A family of portfolio construction approaches that allocate capital such that each asset or cluster contributes equally to total portfolio risk, rather than contributing equally in nominal terms. Hierarchical Risk Parity is a member of this family, but it achieves risk balance through a hierarchical tree structure rather than through numerical optimization.

Out-of-Sample Performance

The behavior of a portfolio construction method when applied to data not used in its estimation — the true test of whether a method generalizes beyond the historical window on which it was calibrated. HRP's structural resistance to estimation error tends to produce more stable out-of-sample results than optimizers that are highly sensitive to input assumptions.

Correlation Structure

The pattern of pairwise linear relationships among the returns of assets in a universe, summarized in the correlation matrix. Identifying and interpreting correlation structure is the foundational task of Step One in HRP, because the clustering algorithm uses pairwise correlations — transformed into distances — to build the dendrogram.

Sharpe Ratio

A measure of risk-adjusted return computed as the ratio of mean excess return to the standard deviation of returns over a defined period. The Sharpe ratio is commonly used to compare the efficiency of different portfolio construction methods and is the primary metric in empirical evaluations of HRP against classical alternatives.
Distance Metric
A function that quantifies the dissimilarity between two assets based on their correlation. A common transformation used in HRP is the formula d equals the square root of one half times the quantity one minus the correlation coefficient, which produces a proper metric satisfying the triangle inequality and maps correlation into a distance space suitable for hierarchical clustering.

Common Questions

Does HRP require historical return data to function?

Yes. HRP uses historical return data to estimate the covariance and correlation matrices that feed into the clustering and bisection steps. The quality of those estimates improves with longer data histories, though HRP is considerably more robust to short samples than mean-variance optimization because it never inverts the covariance matrix.

How sensitive is HRP to the choice of linkage method?

The linkage method affects the shape of the dendrogram and therefore the grouping of assets before bisection. Single linkage tends to produce chain-like trees; complete linkage produces more balanced ones. Our team covers each variant in the curriculum and explains how to assess which structure best reflects the economic relationships in a given asset universe.

Can HRP be combined with factor models or return forecasts?

Yes, though the base HRP algorithm does not require return forecasts at all — it is a purely risk-based method. Extensions exist that incorporate factor-adjusted covariance matrices or tilt the weights toward assets with favorable analytical characteristics, but our materials treat the pure form first before introducing those modifications in Module Five.

Is HRP appropriate for retail investors to apply directly?

The mathematical construction of HRP is accessible to any analyst with a solid grounding in linear algebra and basic statistics. Whether the method is appropriate for a specific financial context depends on many factors outside the scope of this platform. Our materials explain the theory and mechanics, not whether the method suits any individual's circumstances.

How does HRP perform during sharp market downturns?

Empirical studies show that HRP tends to produce more stable drawdown profiles than mean-variance optimized portfolios during periods of elevated correlation and volatility clustering, because its weights do not depend on a well-conditioned covariance matrix. However, no portfolio construction method eliminates drawdown risk, and past analytical results do not guarantee future outcomes.

What Python libraries support HRP implementation?

Several open-source libraries implement HRP or provide the building blocks — SciPy for hierarchical clustering and dendrogram construction, NumPy for matrix operations, and Pandas for return series handling. Our companion notebooks in the curriculum demonstrate a complete implementation from raw return data through to final weights using these standard tools.