R² Score in Machine Learning

The Problem with Absolute Error Metrics

Imagine you've built two regression models. Model A predicts house prices with an MAE of ₹5 lakh ($6,000). Model B predicts student test scores with an MAE of 5 points. Which model is better?

You can't tell. The MAE of ₹5 lakh might be excellent if you're predicting luxury apartments in Mumbai (where prices range from ₹2 crore to ₹50 crore), but terrible if you're predicting studio apartments in tier-2 cities (₹20 lakh to ₹60 lakh). The 5-point MAE could be great for a 100-point test or awful for a 20-point quiz.

Absolute error metrics lack context. They don't tell you how much of the variation in your data you've successfully modeled. That's the fundamental problem R² was designed to solve.

From Statistics to Machine Learning

The coefficient of determination has its roots in early 20th-century statistics, where it emerged as a way to assess the goodness of fit in linear regression. Statisticians needed a scale-invariant metric that could compare models across different domains and units of measurement.

The breakthrough insight was this: instead of measuring absolute error, measure error relative to a naive baseline — a model that always predicts the mean. If your model's predictions have less error than that baseline, you've explained some of the variance. If they have more error, you've actually made things worse.

The Modern ML Context

As machine learning moved from academic statistics departments to production systems at companies like Google, Netflix, and Indian unicorns like Razorpay and PhonePe, R² became a standard diagnostic tool. It answers a question that product managers and stakeholders can understand: "What percentage of the variation in the outcome can your model explain?"

For a financial fraud detection system predicting transaction amounts, an R² of 0.92 means you can explain 92% of the variance in fraudulent transaction sizes based on features like account history, transaction patterns, and merchant categories. The remaining 8% is either genuine randomness or signal you haven't captured with your current feature set.

Historical Note: The notation R² comes from the fact that in simple linear regression (one predictor), it equals the square of the Pearson correlation coefficient (r) between predictions and actual values. For multiple regression, that equivalence breaks down, but the notation stuck.

The Mental Model: Variance Explained

Here's the simplest way to think about R²: it measures how much better your model is than the dumbest possible baseline.

The dumbest baseline is a horizontal line at the mean of your target variable. If you're predicting house prices and the average house costs ₹50 lakh, the baseline "model" just says "₹50 lakh" for every prediction, ignoring all features.

Your actual model presumably does better — it uses square footage, location, number of bedrooms, etc. R² quantifies that improvement:

• R² = 1.0: Your model perfectly explains all variance. Every prediction is exactly correct. (In practice, this usually means you've leaked the target into your features.)
• R² = 0.75: Your model explains 75% of the variance. It's 75% of the way from the baseline to perfection.
• R² = 0.0: Your model is no better than predicting the mean every time. You've learned nothing.
• R² = –0.5: Your model is actually worse than predicting the mean. You've actively destroyed information. This is a red flag that something is deeply wrong.

Why Negative R² Happens (And What It Means)

Negative R² is jarring at first, but it makes mathematical sense. The formula is:

$R^2 = 1 - \frac{SS_{\text{res}}}{SS_{\text{tot}}}$

If your residual sum of squares (SS_res) is larger than the total sum of squares (SS_tot), the fraction exceeds 1, and R² goes negative. This typically happens when:

1. Your model was trained on different data than it's being evaluated on, and it learned patterns that don't generalize.
2. You're using a non-linear model on held-out test data, and it's overfitting.
3. You forced the regression line through the origin (no intercept) when the data doesn't support that constraint.

Negative R² on test data is not necessarily a bug — it's a feature. It's telling you: "This model is worse than useless. It would be better to ignore all your features and just predict the mean."

The Intuition Behind the Formula

Let me unpack the formula piece by piece:

• SS_tot = Σ(y_i – ȳ)²: Total variance in the data. How spread out are the actual values from their mean?
• SS_res = Σ(y_i – ŷ_i)²: Residual variance after your model's predictions. How spread out are the errors?
• R² = 1 – (residual variance / total variance): The proportion of variance not left as residuals.

If your model is perfect (SS_res = 0), then R² = 1 – 0 = 1. If your model is no better than the mean (SS_res = SS_tot), then R² = 1 – 1 = 0.

Key Insight: R² is fundamentally a variance decomposition metric. It doesn't directly measure prediction accuracy — it measures how much of the variability you've accounted for. Those are related but distinct concepts.

Mathematical Foundation

Given $n$ observations with true values $y_i$ and predicted values $\hat{y}_i$ for $i = 1, 2, \ldots, n$, the coefficient of determination is defined as:

$R^2 = 1 - \frac{SS_{\text{res}}}{SS_{\text{tot}}}$

where:

$SS_{\text{res}} = \sum_{i=1}^{n} (y_i - \hat{y}_i)^2 \quad \text{(residual sum of squares)}$

$SS_{\text{tot}} = \sum_{i=1}^{n} (y_i - \bar{y})^2 \quad \text{(total sum of squares)}$

$\bar{y} = \frac{1}{n} \sum_{i=1}^{n} y_i \quad \text{(mean of observed data)}$

Alternative Formulation: Explained Variance

The coefficient can also be expressed in terms of explained sum of squares:

$R^2 = \frac{SS_{\text{reg}}}{SS_{\text{tot}}}$

where $SS_{\text{reg}} = \sum_{i=1}^{n} (\hat{y}_i - \bar{y})^2$ is the explained sum of squares.

For linear regression with an intercept term, we have the identity:

$SS_{\text{tot}} = SS_{\text{reg}} + SS_{\text{res}}$

This decomposition is what makes R² interpretable as "variance explained." However, this identity does NOT hold for non-linear models or linear models without intercepts, which is why R² can behave unexpectedly in those cases.

Range and Properties

Theoretical Range: $R^2 \in (-\infty, 1]$

Practical Interpretation:

• $R^2 = 1$: Perfect fit, all predictions exactly match observations
• $0 < R^2 < 1$: Model explains $R^2 \times 100\%$ of variance
• $R^2 = 0$: Model equivalent to predicting mean
• $R^2 < 0$: Model worse than predicting mean (common on test sets for overfit models)

Adjusted R² for Multiple Regression

The standard R² has a critical flaw: it never decreases when you add more features, even if those features are pure noise. Adjusted R² penalizes model complexity:

$R_{\text{adj}}^2 = 1 - \frac{(1 - R^2)(n - 1)}{n - p - 1}$

where $n$ is the number of observations and $p$ is the number of predictors.

Adjusted R² can decrease when you add unhelpful features, making it a better metric for feature selection and model comparison. The penalty term $(n-1)/(n-p-1)$ increases as you add more predictors relative to your sample size.

Relationship to Correlation Coefficient

For simple linear regression (one predictor) with an intercept, R² equals the square of the Pearson correlation coefficient:

$R^2 = r^2 \quad \text{where} \quad r = \text{corr}(y, \hat{y})$

However, this equality does not hold for:

• Multiple regression (more than one predictor)
• Non-linear models
• Linear models without intercepts

This is a common source of confusion. Correlation measures linear association between two variables; R² measures how much variance a potentially complex model explains.

Computational Complexity

Calculating R² is $O(n)$ in the number of samples — you compute two sums of squares in a single pass through the data. This makes it extremely cheap to compute, which is why it's ubiquitous in ML pipelines.

Technical Note: Some implementations (including scikit-learn) return force_finite=True by default, which replaces NaN (when y is constant and predictions are perfect) with 1.0, and replaces -Inf (when y is constant and predictions are imperfect) with 0.0. This prevents downstream errors in hyperparameter tuning pipelines.

Let's bring it all together.

R² score (coefficient of determination) is a regression evaluation metric that quantifies the proportion of variance in the target variable that your model can explain. It's calculated as $R^2 = 1 - \frac{SS_{\text{res}}}{SS_{\text{tot}}}$, where SS_res is the residual sum of squares (your model's errors) and SS_tot is the total variance (errors if you just predicted the mean).

The metric ranges from negative infinity to 1.0. A score of 1.0 is perfect, 0.0 means you're no better than predicting the mean, and negative values mean you're worse than that naive baseline — a critical overfitting signal that should trigger immediate investigation.

R² is scale-invariant, making it useful for comparing model quality across different domains and units. An R² of 0.85 for predicting house prices in INR has the same interpretation as 0.85 for predicting delivery times in minutes: the model explains 85% of the variance. But this abstraction comes at a cost — R² doesn't tell you the magnitude of errors in real-world units. You must pair it with MAE or RMSE for actionable insights.

For linear regression, R² has a clean "variance explained" interpretation because the decomposition SS_tot = SS_reg + SS_res holds exactly. For non-linear models, you can still compute R², but it loses this strict interpretation and can behave unexpectedly. In those cases, treat it as a benchmark metric rather than a fundamental measure of fit.

Adjusted R² solves a critical flaw: standard R² never decreases when you add features, even if they're noise. Adjusted R² penalizes complexity, making it essential for feature selection and model comparison when architectures differ in the number of parameters. The penalty grows as the ratio of features to samples increases, preventing overfitting on small datasets.

In production ML systems — from Airbnb's home value predictions to Uber's demand forecasts — R² serves multiple roles: model selection (choose the architecture with highest test R²), monitoring (alert when R² drops below threshold, signaling model degradation), and stakeholder communication ("the model explains 78% of variance" is more intuitive than "MAE is 2.3 units").

The key is knowing its limitations. Don't use R² as your sole metric. Don't compare it across different test sets. Don't ignore negative values — they're red flags, not bugs. And always ask: does this R² translate to acceptable prediction errors for my use case?

Final takeaway: R² measures how much of the variation in your outcome you've successfully modeled. It's a powerful diagnostic and communication tool, but it's fundamentally a variance metric, not an accuracy metric. Use it to understand how well you're capturing signal, but always validate that captured signal translates to real-world performance using absolute error metrics.