Skip to content
Plotwise

Level 02

Bivariate.

Two variables, side by side. Do they move together? Linearly, monotonically, or in some stranger way? And is the link real, or is a third variable pulling both strings?

2.1

Bivariate analysis

The study of the relationship between two variables.

It helps reveal dependencies, correlations or patterns that can be critical for feature engineering, data cleaning and model design.

  • Visual inspection: for all the attribute types.
  • Correlation assessment: for numerical variables, it often involves calculating correlation coefficients to quantify the strength and direction of relationships.
    • Linear correlation: Pearson and Spearman
    • Non-linear correlation: Mutual Information

2.2

Visual inspection

The best visualization depends on the involved variable types and the dataset cardinality. All the labs below use the bank data, colored by whether the customer bought a PEP.

Continuous vs continuous

  • Scatter plots when the number of points is manageable (raw data insight).
  • Heatmap when the number of points is large and you need aggregation.
Try it

Age vs income, colored by PEP

Try it

Pair plot of the bank attributes

Diagonal: the density of each attribute per PEP class. Off-diagonal: one scatter plot per pair. Hover a cell to read it.

Categorical vs continuous

  • Box plot: one for each value of the categorical attribute.
  • Density plot: one curve per category.
Try it

Distribution of a number, by category

Categorical vs categorical

  • Stacked bar chart: x-axis v_1, color v_2, height counts or proportions.
Try it

Matrix of stacked bar charts by PEP

2.3

Dataset slicing

The data profile can differ for different portions of the dataset, which should be analyzed by subpopulation:

  • Geographic regions
  • Demographic categories
  • Operating conditions
Try it

Slice the bank customers

2.4

Pearson correlation

The correlation between pairs of objects described by attributes (binary or continuous) is a measure of the existence of a linear relationship between its attributes.

Pearson correlation
Corr(\bx, \by) = \frac{Cov(\bx, \by)}{StDev(\bx)\cdot StDev(\by)}
Cov(\bx, \by) = \frac{1}{n}\sum_{k=1}^{n}(x_k - \xb)(y_k - \yb)
StDev(\bx) = \sqrt{\frac{1}{n}\sum_{k=1}^{n}(x_k - \xb)^2}
  • Pearson correlation normalizes covariance by the product of standard deviations → it is scale-free.
  • The correlation varies in [-1, 1]. A correlation of 1 (−1) means that the attributes can be mutually expressed by a linear relationship of the type x = a\,y + b.
  • There could still be non-linear relationships between the data that would therefore not be captured!
  • Correlation can also be useful for discarding attributes that do not carry additional information. Here \bx and \by are two distinct attributes and their elements the values of the two attributes in the different objects of the data set.
x y Corr(x, y)
(−3, 6, 0, 3, −6) (1, −2, 0, −1, 2) −1
(3, 6, 0, 3, 6) (1, 2, 0, 1, 2) 1
(−3, −2, −1, 0, 1, 2, 3) (9, 4, 1, 0, 1, 4, 9) 0, yet y_k = x_k^2
Play

Correlation sandbox: tap to add, drag to move

Scatter plots

  • Let you determine whether some of the attributes are related. Useful for reducing the number of attributes to be considered in a distance/similarity function.
  • When labels are available, they allow you to determine whether you can classify objects based on the values of two attributes.
Try it

Iris scatter matrix: can two attributes classify?

2.5 · Mini game

Visualizing Pearson correlation

Points are described by continuous attributes x and y. In each chart, attribute values were generated with a different level of correlation. Is x correlated with y?

Look

From −1 to 1

Play

Guess the correlation

2.6

Spearman correlation

Spearman correlation measures the strength and direction of a monotonic relationship between two attributes: when x increases, y always increases (or always decreases), not necessarily linearly. It is the Pearson correlation computed on the ranks of the values instead of on the values themselves.

Spearman correlation
\rho_s(\bx, \by) = Corr(rank(\bx), rank(\by))

The rank of a value is its position in the list of the values sorted in ascending order (1 = smallest); tied values get the average of the positions they occupy: x = (10, 30, 20, 20, 50) \rightarrow rank(x) = (1, 4, 2.5, 2.5, 5).

Without ties
\rho_s(\bx, \by) = 1 - \frac{6\sum_{k=1}^{n} d_k^2}{n(n^2 - 1)} \qquad d_k = rank(x_k) - rank(y_k)

The second formula holds when there are no ties. \rho_s varies in [-1, 1]: 1 (−1) means that y is a monotonically increasing (decreasing) function of x.

Try it

Rank anything

A worked example

Study hours and exam grade of 6 students: the grade grows with the hours, with one exception (students S3 and S4). Edit any value: ranks, d and both coefficients update.

Edit

Hours vs grade

2.7

Spearman vs Pearson

Spearman is preferable when

  • The relationship is monotonic but not linear (exponential, logarithmic, saturation effects).
  • The data contain outliers: with ranks, the largest value always weighs n, whatever its magnitude.
  • Distributions are strongly skewed (e.g., income, prices).
  • Attributes are ordinal (grades, satisfaction levels, rankings): only the order is meaningful, while Pearson requires meaningful differences, i.e., interval or ratio attributes.

Pearson is preferable when

  • We are interested in linear relationships, e.g., to verify whether an attribute can be expressed as x = a\,y + b and then discarded.
  • Attributes are numerical and free of strong outliers: Pearson exploits the actual distances between values, not only their order.
Neither of them captures non-monotonic relationships: for x = (-3, -2, -1, 0, 1, 2, 3) and y = x^2 both Pearson and Spearman are 0 → see Mutual Information.
Try it

Three datasets, two coefficients

2.8

Spurious correlation

Two attributes appear related (high correlation) but there is no direct cause–effect link between them.

  • The association is really present in the data, but it is generated by something else: a hidden third factor, a common trend, chance, or the way data were collected.
  • Correlation does not imply causation: a model can exploit a spurious correlation to predict, but it fails as soon as the hidden factor changes, and decisions based on it are wrong.

Typical sources

  • Confounding variable: a common cause z drives both x and y.
  • Common trend: two time series that both grow (or decrease) over time, e.g., number of smartphones sold and average life expectancy.
  • Chance: with many attributes some pairs are correlated just by chance (100 attributes → 4,950 pairs to test).
  • Selection bias and aggregation: data collected on a non-representative subset, or groups with different behavior merged together.
Try it

Chance: how many pairs are “significant” by luck?

Formal definition

\bx and \by are spuriously correlated with respect to a variable \bz (confounder) when

\rho(\bx, \by) \neq 0 \quad \text{but} \quad \rho(\bx, \by \mid \bz) \approx 0
Partial correlation
\rho(\bx, \by \mid \bz) = \frac{\rho_{xy} - \rho_{xz}\,\rho_{yz}}{\sqrt{(1 - \rho_{xz}^2)(1 - \rho_{yz}^2)}}
  • \rho(\bx, \by \mid \bz) is the partial correlation: the correlation between x and y once the linear effect of z has been removed from both, i.e., the correlation between the residuals of x and y regressed on z.
  • x and y are conditionally independent given z: once z is known, x carries no further information on y.

An example: ice creams and drownings

  • Weekly ice-cream sales and drownings over 3 years (synthetic data) are strongly correlated.
  • Temperature is the confounder: in hot weeks people buy more ice creams and more people go swimming, hence more drownings.
  • Within temperature bands, or after removing the effect of temperature, the correlation (almost) vanishes: banning ice cream would not save anyone. In the course data: \rho_{xy} = 0.80,\ \rho_{xz} = 0.97,\ \rho_{yz} = 0.82, so \rho(x,y|z) = \frac{0.80 - 0.97\cdot0.82}{\sqrt{(1 - 0.97^2)(1 - 0.82^2)}} \approx 0.03.
Try it

Raw, sliced, residuals

Detecting spurious correlation

  • Domain knowledge: is there a plausible mechanism linking x and y? Verify the hypothesis with the domain experts (see Business Understanding).
  • Look for confounders: list candidate common causes (time, season, location, size of the object, …) and control for them.
    • Dataset slicing / stratification: compute the correlation within groups where z is (almost) constant; if it vanishes, the correlation is spurious.
    • Partial correlation: regress x and y on z and correlate the residuals.
  • Time series: remove trends and seasonality (e.g., by differencing) before computing the correlation.
  • Chance: with many attributes, check the statistical significance of the correlation (correcting for multiple comparisons) and confirm it on new data.
Correlation is a hint, not a proof These checks can reveal that a correlation is spurious, but only controlled experiments (randomization, A/B tests) can establish a causal relationship.
Try it

Common trend: smartphones and life expectancy

2.9

Mutual information

How much knowing one variable helps us predict the other.

For two discrete random variables \bx and \by, with joint probability function p(x,y) and marginal probability functions p(x) and p(y), the mutual information is defined as:

Mutual information
MI(\bx, \by) = \sum_{x \in \bx}\sum_{y \in \by} p(x, y)\,\log\frac{p(x, y)}{p(x)\cdot p(y)}
  • MI(\bx, \by) \geq 0
  • MI(\bx, \by) = 0 iff x and y are independent (i.e., knowing the value of x does not provide any information about the y value)
  • MI(\bx, \by) = MI(\by, \bx) symmetry
  • Mutual information captures all the types of correlation (not only the linear one).
  • The definition for continuous variables is technically different, but the concepts are the same.

Joint probability p(x, y)

  • Acts as a weight in the sum.
  • Events that are more likely (higher p(x, y)) contribute more to MI.
  • Rare events (small p(x, y)) contribute less.
  • Intuition: “focus on what actually happens often.”

Product of marginals p(x)·p(y)

  • Represents what would happen if x and y were independent.
  • If x and y are independent, p(x, y) = p(x)·p(y), and the log term becomes 0.
  • Intuition: this is your baseline expectation for each combination; deviation from it is information.

Comparing p(x)·p(y) to p(x, y)

  • If p(x, y) > p(x)·p(y): the pair occurs more often than expected under independence; seeing x increases the likelihood of y (or vice versa).
  • If p(x, y) < p(x)·p(y): the pair occurs less often than expected; seeing x makes y less likely than random chance.
  • Both cases indicate statistical dependence: the probability of one variable depends on the value of the other.
Why the log? The log in mutual information does more than just “make MI zero when independent” (p(x,y) = p(x)p(y) \Rightarrow \log 1 = 0). It transforms the multiplicative ratio into additive information. A ratio of 2 means twice as likely; a ratio of 0.5 means half as likely. The ratio is a factor of change, not an amount of information: averaging 2 and 0.5 does not return 1, while their base-2 logarithms, +1 and −1, express them as equal-sized contributions in opposite directions.
Try it

Rain and umbrellas

Suppose x = whether it's raining, y = whether people carry umbrellas, with p(x = Yes) = 0.3 and p(y = Yes) = 0.2. If independent, p(x = Yes, y = Yes) = 0.3 · 0.2 = 0.06. In reality p(x = Yes, y = Yes) = 0.15 > 0.06: seeing rain makes carrying an umbrella more likely → dependency → positive MI contribution.
Try it

Pearson vs Spearman vs MI on any shape

2.10 · Boss level

The Titanic dataset

The sinking of the Titanic is one of the most infamous shipwrecks in history. On April 15, 1912, during her maiden voyage, the widely considered “unsinkable” RMS Titanic sank after colliding with an iceberg. Unfortunately, there weren't enough lifeboats for everyone on board, resulting in the death of 1502 out of 2224 passengers and crew. While there was some element of luck involved in surviving, it seems some groups of people were more likely to survive than others.

Which attribute tells you most about who survived?

Investigate

Survival by attribute

Survival rate by group

MI with Survived bits, all attributes