Мазмұнға өту

Статистика · Бастапқы

Describe data, effect sizes and uncertainty

Distinguish data variation, estimation uncertainty and practical importance.

12 мин оқу · 2026 ж. 26 қаз.

Бұл бет әлі аударылмаған, сондықтан ағылшын нұсқасын оқып отырсыз.

Негізгі тұжырымдар

  • Choose appropriate descriptive summaries.
  • Distinguish SD, SE and confidence intervals.
  • Report effects in interpretable units.
Мазмұны

Distinguish data variation, estimation uncertainty and practical importance.

Statistics help readers answer three different questions: what was observed, how large a difference or association was estimated, and how uncertain that estimate is. A good paper keeps these questions separate. A mean describes one feature of the observations. An effect size compares quantities. An uncertainty interval describes what the analysis can resolve under stated assumptions. None of these can repair a biased design.

This tutorial develops the distinctions using the synthetic controller dataset supplied with this collection. Every numerical example below is constructed for teaching. The examples are not evidence about real controllers, students or biological systems.

Start with the quantity you want to describe

A population is the set of units to which a question refers. A sample is the set actually observed. A parameter is a numerical feature of the population or model, such as a population mean. A statistic is calculated from the sample, such as its mean. An estimand is the precise quantity the research aims to learn about. Defining the estimand prevents a convenient software output from becoming the research question after the analysis.

Suppose an engineering project compares positioning errors across layouts. The mean error across layouts answers a different question from the mean error across all recorded video frames. The latter gives more weight to layouts with more frames. In biology, the mean response across animals differs from a mean across all cells when animals contribute unequal numbers of cells. In a school survey, a mean across participating students may differ from a population mean requiring sampling weights. State whose average is being estimated and how each unit contributes.

Before calculating a summary, create a data dictionary containing the variable name, meaning, unit, allowed values and missing value code. Verify whether a zero represents a measured absence or an unrecorded observation. Check that centimetres and millimetres have not been mixed. These decisions determine what the resulting number means.

Describe the distribution before choosing an average

A distribution describes which values occur and how frequently. Inspect its centre, spread, shape and unusual observations. Also inspect groups separately when combining them could hide meaningful differences. A histogram can show an asymmetric distribution; a scatter of individual points is often more informative when there are only a few observations.

The arithmetic mean adds all values and divides by their number. It is useful when totals and additive differences matter. The median is the middle ordered value, or the average of the two middle values for an even sample. It describes the middle of a distribution and is less affected by an isolated extreme value. The interquartile range, or IQR, is the difference between the 75th and 25th percentiles. Software can use different sample quantile conventions, which matters in very small datasets.

A strongly skewed distribution does not automatically make the mean scientifically wrong. If your question concerns average energy consumption, rare periods of very high consumption contribute to the quantity of interest. Reporting only the median could hide their practical importance. You might report the mean with an appropriate interval, the median and IQR, and the individual or distributional display. Conversely, if the question concerns a typical waiting time, a median may be the clearer primary summary.

For categorical variables, provide counts and denominators. “Twelve devices failed” is incomplete without the number tested and the test conditions. A proportion is a count divided by a defined total. A rate uses an exposure denominator, such as failures per operating hour. These are not interchangeable: a device operated longer has more opportunity to fail.

Work through a descriptive calculation

Controller A has these twelve constructed error values, in millimetres:

18, 21, 17, 24, 20, 19, 22, 16, 25, 23, 18, 21

Their sum is 244 mm. The mean is therefore 244 / 12 = 20.33 mm. After ordering the values, the middle two are 20 and 21, giving a median of 20.5 mm. The minimum is 16 mm and the maximum is 25 mm. These summaries describe the supplied values without making a population claim.

The sample standard deviation, or SD, describes spread around the sample mean. To calculate it, subtract the mean from each observation, square each difference, add the squared differences, divide by n minus 1, and take the square root. Using n minus 1 gives the usual unbiased sample variance under independent, identically distributed sampling; taking its square root gives the sample SD. The R documentation states this variance convention. S315

For controller A, the sample SD is 2.84 mm. It has the same units as the observations. The variance is in squared millimetres, which is useful in calculations but less intuitive in a results sentence. Keep extra decimal places during calculations and round only the reported result.

Distinguish SD, SE and confidence intervals

The standard error, or SE, describes the sampling variability of an estimator. It does not describe the spread of individual observations. Under an independent sampling model, the estimated SE of a mean is SD / square root of n. For controller A, 2.8391 / square root of 12 = 0.8196 mm. With comparable variability, collecting more independent units tends to improve precision; repeatedly measuring the same unit does not automatically have the same effect.

For a mean from independent normal observations with unknown variance, a conventional 95% confidence interval, or CI, is the sample mean plus or minus a t critical value multiplied by its SE. For twelve observations, the degrees of freedom are eleven, the critical value is about 2.201, and the interval is 20.3333 ± 2.201 × 0.8196, giving 18.53 to 22.14 mm. NIST provides this calculation and its interpretation. S14

The same mean is shown with standard deviation, standard error and a 95% confidence interval. Their widths differ because they represent different quantities.

A 95% confidence procedure is designed to include the fixed population quantity in 95% of repeated applications under its assumptions. It does not mean that 95% of observations lie inside this particular interval. Nor does the usual frequentist interpretation assign a 95% probability to the fixed parameter being within the observed limits. That probability interpretation requires a different framework and its assumptions.

Because these values were invented, their calculated interval is a demonstration of a procedure. It has no empirical coverage claim for real controllers. In a real paper, explain the sampling or model assumptions that connect the observations to the target population.

Report an effect size in units readers understand

An effect size quantifies a comparison or association. It need not imply causation. A mean difference in millimetres, a difference in questionnaire points and a ratio of failure risks are all effect sizes. Choose a scale that answers the question and state the comparison direction.

In the paired controller example, calculate A minus B within each layout. The twelve differences are:

4, 3, 4, 4, 3, 4, 3, 2, 5, 5, 2, 4

Their mean is 3.58 mm, their SD is 1.00 mm after rounding, and the SE of their mean is 0.29 mm. The t based 95% CI is 2.95 to 4.22 mm. Positive values mean B has lower error. The uncertainty must come from the paired differences, not from treating A and B as unrelated groups. S28

Notice the question answered by each number. The SD of A describes differences among A measurements. The SD of paired differences describes how much the A versus B contrast varies among layouts. The CI describes uncertainty in the estimated average contrast under the assumed model. An error bar labelled simply “error” prevents readers from distinguishing these quantities.

A standardised effect size divides a difference by a specified measure of variation. For paired data, one convention, Cohen’s d with subscript z, divides the mean difference by the SD of differences. Here that calculation gives approximately 3.60. It is not the same standardisation as dividing by a pooled SD from independent groups. Name the estimator and its denominator, because different conventions can yield different numbers from the same data. Lakens explains these distinctions and their relevance to evidence synthesis. S302

Do not translate a standardised value into “important” using a universal label. Importance depends on the scientific context, measurement quality and consequences. A raw difference should usually accompany the standardised statistic so readers can recover its practical meaning.

Explain absolute and relative differences together

Consider a separate synthetic example in which an event occurs in 10 of 100 units in group A and 15 of 100 units in group B. The event proportions are 10% and 15%. The risk difference is 5 percentage points. The risk ratio is 1.50, corresponding to a 50% relative increase. These statements describe the same comparison using different denominators.

The odds are the probability of an event divided by the probability of its absence. The odds ratio here is (15 / 85) / (10 / 90) = 1.59, approximately. It is not the risk ratio. Logistic regression estimates associations on the log odds scale unless results are subsequently transformed or standardised. R’s model documentation identifies the binomial family and logit link used for this model. S316

A useful sentence would be: “In the constructed example, event proportions were 15% and 10%, a difference of 5 percentage points and a risk ratio of 1.50.” A real study would also report appropriate uncertainty and identify whether observations were independent, paired or clustered. Do not attach an interval calculated for an independent design to a paired comparison.

Separate precision, bias and practical importance

Precision concerns variability or uncertainty in an estimate. Bias concerns systematic discrepancy from the target quantity. Increasing the sample size can improve precision without removing bias. A survey of volunteers may provide a very precise description of those volunteers while poorly representing all students. A sensor with a calibration error can produce tightly clustered but inaccurate measurements.

Before inspecting results, decide what difference would matter. In engineering this might concern a tolerance linked to the application. In social science it might concern a change with an interpretable consequence on a validated scale. In biology it might concern a response large enough to alter a specified function. Justify this threshold rather than selecting it to make the observed estimate appear impressive.

Compare the interval with both zero and that meaningful threshold. An interval entirely above zero can still be compatible only with effects too small to matter. An interval spanning zero can also include important positive and negative effects, showing that the study is inconclusive at that precision. Greenland and colleagues explain why interpretation must consider the full model and uncertainty rather than a single threshold. S205

Turn the analysis into a results paragraph

A weak sentence is: “Controller B was significantly better, with small error bars.” It leaves the outcome, unit, comparison and uncertainty undefined.

A fuller teaching example is: “Across twelve constructed paired layout cases, mean positioning error was 20.33 mm for A and 16.75 mm for B. The mean within case reduction, defined as A minus B, was 3.58 mm, with a t based 95% confidence interval from 2.95 to 4.22 mm. Individual paired differences ranged from 2 to 5 mm. These synthetic values illustrate statistical reporting and do not validate a physical controller.”

The first sentence describes each condition. The second states the estimand, direction and interval. The third adds distributional context. The final sentence prevents a teaching calculation from being mistaken for an experiment. In a real manuscript, replace it with the actual limits of the sampling, measurement and design.

For a review, extract effect estimates with their uncertainty and definitions. Do not compare an SD from one study with an SE from another as though both describe precision. Record whether a reported difference is adjusted or unadjusted, which group is the reference and what population it concerns before attempting a synthesis.

Practice and check your reasoning

Exercise 1. A figure shows a mean and bars extending one SD on each side. A reader calls the bars a 95% CI. What must the legend say?

Model answer. It must state that the bars represent one sample SD and identify the observations and sample size. The reader’s interpretation is wrong because variability among observations and uncertainty in the mean are different quantities.

Exercise 2. You measure ten cells from each of four animals. Can you calculate the SE for a population mean by dividing a cell SD by the square root of forty?

Model answer. Not without a model justifying the independence structure. Cells from the same animal can be correlated. Define the animal and cell levels and use an analysis that respects that structure. Forty rows do not automatically provide forty independent biological replicates.

Exercise 3. Write one result sentence for your project with the estimate, unit, interval method, independent unit and comparison direction. Use Template 11, then ask a colleague to explain what every reported number represents.

Sources

S14: NIST/SEMATECH. e Handbook of Statistical Methods. Section 1.3.5.2, Confidence Limits for the Mean.

S28: NIST/SEMATECH. e Handbook of Statistical Methods. Section 7.3.1.1, Analysis of paired observations.

S205: Greenland S, Senn SJ, Rothman KJ, Carlin JB, Poole C, Goodman SN and Altman DG. 2016. Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations. European Journal of Epidemiology 31, pages 337 to 350. DOI: 10.1007/s10654-016-0149-3.

S302: Lakens D. 2013. Calculating and reporting effect sizes to facilitate cumulative science: a practical primer for t tests and ANOVAs. Frontiers in Psychology 4, article 863. DOI: 10.3389/fpsyg.2013.00863.

S315: R Core Team. R stats documentation: Correlation, Variance and Covariance. Accessed 11 November 2026.

S316: R Core Team. R stats documentation: Family Objects for Models. Accessed 11 November 2026.

Файлдар

Бұл файлдарды жүктеп алу үшін кіріңіз.