Мазмұнға өту

Статистика · Орташа

Interpret correlation and regression without confusing them with causation

Interpret associations and model coefficients without importing unsupported causal claims.

14 мин оқу · 2026 ж. 26 қаз.

Бұл бет әлі аударылмаған, сондықтан ағылшын нұсқасын оқып отырсыз.

Негізгі тұжырымдар

  • Plot the relationship.
  • Explain adjustment and units.
  • Make causal assumptions explicit.
Мазмұны

Interpret associations and model coefficients without importing unsupported causal claims.

A correlation describes how two variables vary together. A regression model describes an outcome in relation to one or more predictors. A causal claim asks a different question: what would change if an intervention changed an exposure? Keeping those questions distinct helps you choose an analysis and prevents an interesting pattern from becoming an unsupported explanation.

This tutorial builds a regression example from eight invented observations, then shows how to interpret coefficients, evaluate a model and write a bounded conclusion. All numerical observations and hypothetical study scenarios are for teaching.

Decide whether the purpose is description, prediction or causal inference

A descriptive analysis might ask whether mentoring attendance and confidence scores vary together in a sample. A predictive analysis might ask how accurately available information predicts confidence in new students. A causal analysis might ask how confidence would differ if students were assigned to attend mentoring rather than not attend it. These aims require different design choices and may use different variables.

A predictor can improve forecasting without being a cause. A variable that is useful for prediction can even be inappropriate to adjust for in a causal model. Conversely, a causal analysis may deliberately include variables needed to address confounding even when their contribution to predictive accuracy is modest. State the purpose before choosing a model or discussing “important predictors”. Hernán and Robins develop the distinction between observed associations and intervention questions. S306

In natural science, a correlation between two molecular measurements does not establish which regulates the other. In engineering, temperature may predict failure because it tracks an unmeasured operating load. In social science, attendance may track motivation and access to support. The common lesson is to ask what generated the pattern, not merely whether the coefficient differs from zero.

Begin with a plot and the measurement process

Create a scatter plot with labels and units. Inspect whether the pattern is approximately straight, curved, clustered or dominated by a few points. Ask whether measurements span a useful range and whether values were selected because of the outcome. Check whether observations come from independent units.

A restricted range can alter a correlation. A study of students with nearly identical grades may show a different correlation from a study spanning a broad attainment range. Combining distinct groups can also create or conceal a relationship. Plot groups where justified, while avoiding post hoc subgroup searches for a desired conclusion.

Repeated observations need particular care. A correlation among students’ average study times and scores answers a different question from whether a particular student scores differently during weeks when that student studies more. Combining all weeks as independent rows can mix differences between people with changes within people. Use a model that reflects the level of the question and the repeated structure.

Understand Pearson and Spearman correlation

Pearson’s correlation coefficient, r, standardises linear covariation. It lies between negative one and positive one when defined. Its sign indicates the direction of linear association; its magnitude describes the strength of that linear pattern in the observed data. A zero correlation does not establish independence or the absence of a nonlinear relationship.

For example, take the synthetic x values negative 2, negative 1, 0, 1, 2 and the corresponding squared values 4, 1, 0, 1, 4. Pearson r is exactly zero. Nevertheless, y is completely determined by x through a curved relationship. A scatter plot reveals what the single coefficient misses.

Spearman’s correlation is a correlation of ranks. It describes a monotonic association, meaning that values generally move in one direction as the other variable increases. It can accommodate some patterns that are not straight lines, but it does not remove confounding, repair dependent sampling or establish a causal direction. State which coefficient you used and how ties and missing observations were handled. R documents Pearson, Spearman and Kendall methods and the consequences of different missing value options. S315

A correlation is dimensionless. If metres are converted to centimetres without changing the data structure, Pearson r is unchanged. A regression slope, by contrast, changes with the measurement units. That distinction matters when interpreting results.

Build a linear regression from a complete example

Consider eight fictional students with recorded weekly study hours and a subsequent assessment score. These values were constructed to illustrate calculation; no students were recruited.

Student Study hours Score out of 100
1 1 48
2 2 55
3 3 53
4 4 61
5 5 65
6 6 63
7 7 72
8 8 75

A simple linear model expresses the mean outcome as an intercept plus a slope multiplied by the predictor. Ordinary least squares, or OLS, chooses the fitted coefficients that minimise the sum of squared residuals. A residual is the observed score minus the fitted score. NIST explains this model and its extension to several predictors. S308

For these values, the fitted equation is:

predicted mean score = 45.00 + 3.667 × study hours

The intercept is the fitted score at zero hours. Zero lies outside the observed range of one to eight hours, so the intercept is a component of the equation, not a well supported description of students who never study. The slope is the estimated change in the modelled mean score associated with a one hour difference in study time. Its units are score points per hour.

At five hours, the fitted mean score is 63.33. The actual score in the constructed table is 65, producing a residual of 1.67 points. Different residuals describe the departures from the fitted line. The model does not assert that every student at five hours must score 63.33.

Under the conventional independent normal error model with constant variance, the slope SE is 0.395 points per hour. With six residual degrees of freedom, the 95% CI is 2.70 to 4.63 points per hour. Pearson r is 0.967 and R squared is 0.935. These calculations describe the invented sample and its assumed model; they do not validate an educational theory.

Write what the coefficient actually means

A weak report is: “Studying causes scores to increase by 3.67 points per hour, proving that studying is effective.” The calculation has not separated study time from prior attainment, motivation, course difficulty or measurement error. It also has not established that asking one student to study an additional hour would produce the between student association.

A better teaching report is: “In the eight constructed observations, each additional reported study hour was associated with a 3.67 point higher fitted mean assessment score. The conventional OLS 95% confidence interval was 2.70 to 4.63 points per hour, conditional on the specified model assumptions. The example demonstrates an association and does not estimate the causal effect of additional study time.”

The first sentence defines the scale and association. The second identifies uncertainty and its conditions. The third defines the claim boundary. In a real paper, add the actual sampling, measurement and covariate details, plus the relevant limitations. Avoid replacing them with a generic claim that “correlation does not imply causation” while continuing to use causal verbs elsewhere.

Interpret multiple regression and variable coding

A multiple regression includes several predictors. A coefficient is conditional on the other included terms and the model form. “Adjusted for” names a calculation, not a guarantee that all relevant bias has disappeared.

Suppose a hypothetical model includes attendance, prior score and school. An attendance coefficient compares fitted means at the same specified values of the other covariates. It may differ from an unadjusted comparison because the target comparison and model have changed. Explain why each covariate was included. Avoid a search that includes variables solely because their p values are small.

For a categorical predictor, name the reference category. If control is coded zero and programme attendance one, the coefficient represents attendance relative to control under that coding. Reversing the coding changes the sign. For an interaction, the coefficient for one variable depends on the other variable’s value. A main effect cannot be interpreted as an average effect everywhere when the model contains an interaction that changes it.

Multicollinearity means that predictors contain strongly overlapping information. It can make individual coefficients unstable or imprecise even when overall prediction appears reasonable. It is a reason to examine the question, coding and design, not a reason to delete whichever variable produces an inconvenient result. Report model specification clearly enough that someone can reconstruct the comparison.

Make causal assumptions explicit

A confounder is part of a causal structure that creates an alternative explanation for an exposure and outcome association. In the mentoring example, prior motivation could influence both attendance and later confidence. The resulting association would combine the relationship of interest with this alternative pathway.

Prior motivation can influence both mentoring attendance and confidence. An attendance to confidence arrow is the causal effect being investigated, not established by the diagram.

A directed acyclic graph, or DAG, represents assumptions about causal relationships without directed cycles. Its arrows express proposed directions, not relationships proven by a correlation matrix. Use subject knowledge, timing and the design to construct it. Greenland, Pearl and Robins explain how causal diagrams clarify confounding and variable selection. S319

A mediator lies on a proposed causal pathway. If mentoring improves research skills and skills improve confidence, adjusting for skills can change a total effect question into a different question. A collider is a common effect of two variables. Conditioning on a collider can introduce an association that was not present before selection. These distinctions are why adjusting for every measured variable is not a general solution.

A cautious causal interpretation needs more than temporal order. Ask whether relevant confounding is addressed, whether the comparison groups have adequate overlap, whether the intervention is well defined and whether selection or measurement processes distort the contrast. Unmeasured causes, poorly measured covariates and incorrect model forms can remain. If the available design cannot justify the required assumptions, report an association and explain what further design would address the causal question.

Evaluate fit without worshipping R squared

In a standard OLS model with an intercept, R squared describes the fraction of sample outcome variation accounted for by fitted values relative to an intercept only model. It is not the percentage of an individual’s outcome caused by a predictor. It is also not a measure of accuracy on new cases. NIST emphasises that even a high R squared does not establish an adequate model. S29

Use residual plots to look for curvature, unequal spread, collection order patterns and influential observations. Heteroscedasticity means that error variance changes across predictor values. A suitable robust SE may address some variance misspecification, but does not repair a wrong mean function or dependence that the method ignores. “Robust” always needs a description of what robustness is intended.

For small samples, normal error assumptions can matter for conventional t based intervals and tests. The predictor itself does not have to follow a normal distribution for ordinary linear regression. A strongly curved relationship may require a scientifically justified nonlinear term or another model. Do not fit a complicated curve to a handful of observations solely because it follows every point closely.

Assess leverage, which concerns unusual predictor combinations, and influence, which concerns how much a point changes the fitted analysis. An influential observation may be an error, a valid extreme case or evidence that the model is too simple. Investigate before excluding it and report sensitivity to any debatable decision.

Separate confidence intervals from prediction intervals

A confidence interval for the fitted mean concerns uncertainty in an average response at specified predictor values. A prediction interval for a new observation includes additional variation among individual outcomes under the model. It is generally wider because a new student or device can differ from the average even if the mean is estimated precisely.

Do not extrapolate far beyond the data without a scientific model supporting that extrapolation. A linear fit observed across modest temperatures need not remain linear at much higher temperatures. A narrow calculated interval can be misleading when the physical model is inappropriate outside the observed range. Identify the range in the Results and discuss the deployment range separately.

For predictive claims, reserve evaluation data that were not used for fitting, selecting variables, choosing transformations or tuning model complexity. Fit preprocessing steps using training data only. When rows share participants, devices or time periods, the split should reflect the intended new unit. Scikit learn’s official guidance explains how leakage from test data can inflate performance estimates. S23

Read binary and count model coefficients on their own scales

For logistic regression, a coefficient operates on the log odds scale; exponentiating it gives an odds ratio for the specified predictor contrast, conditional on other terms. An odds ratio is not generally a risk ratio. For a Poisson model with a log link, exponentiating a coefficient gives a multiplicative change in the modelled count, or rate when an appropriate exposure offset is included. S316

Always name the outcome coding and denominator. A model of device failure and a model of device success reverse the interpretation. For readers unfamiliar with odds, estimated probabilities or clearly defined marginal contrasts can help, provided their computation and uncertainty are explained. Do not mix conditional odds ratios with unadjusted percentages as though they are the same estimate.

What to report and what to extract in a review

Your Methods should give the model formula or an unambiguous equivalent, outcome, predictors, coding, transformations, interaction terms, dependence handling and rationale for covariates. State how missing values were handled and how many observations entered each analysis. Report uncertainty and relevant diagnostics, not only coefficients that met a threshold.

In a review, extract whether estimates were adjusted, which variables were adjusted for, whether the outcome was measured on the same scale and whether the estimand is comparable. Two studies using the word “regression” may estimate different quantities. A narrative synthesis should explain such differences; a statistical synthesis must account for them rather than pooling labels indiscriminately.

Practice and model answers

Exercise 1. The synthetic regression has R squared = 0.935. Can you say that studying caused 93.5% of the scores?

Model answer. No. R squared describes sample variation accounted for by this fitted model. It neither allocates causal responsibility nor measures individual treatment effects.

Exercise 2. You want to predict failure in a new machine. You randomly split sensor readings, with readings from every machine in both training and test sets. What should you reconsider?

Model answer. The evaluation may exploit information specific to machines already represented in training. A split by machine better addresses generalisation to new machines, with preprocessing and tuning confined to the training process.

Exercise 3. Write a sentence explaining why each covariate belongs in your model. Identify any variable measured after the exposure and explain whether adjusting for it changes the question. Record the rationale in Template 4.

Sources

S23: Scikit learn developers. Common pitfalls and recommended practices. Official documentation. Accessed 11 November 2026.

S29: NIST/SEMATECH. e Handbook of Statistical Methods. Section 4.4.4, How can I tell if a model fits my data?

S306: Hernán MA and Robins JM. 2020. Causal Inference: What If. Chapman & Hall/CRC. Author website with freely available book and updates.

S308: NIST/SEMATECH. e Handbook of Statistical Methods. Section 4.1.4.1, Linear Least Squares Regression.

S315: R Core Team. R stats documentation: Correlation, Variance and Covariance. Accessed 11 November 2026.

S316: R Core Team. R stats documentation: Family Objects for Models. Accessed 11 November 2026.

S319: Greenland S, Pearl J and Robins JM. 1999. Causal Diagrams for Epidemiologic Research. Epidemiology 10(1), pages 37 to 48. DOI: 10.1097/00001648-199901000-00008.

Файлдар

Бұл файлдарды жүктеп алу үшін кіріңіз.