Мазмұнға өту

Қайталанымдылық · Бастапқы

Organise data so your results can be checked

Preserve raw records and create a traceable path to every reported result.

11 мин оқу · 2026 ж. 26 қаз.

Бұл бет әлі аударылмаған, сондықтан ағылшын нұсқасын оқып отырсыз.

Негізгі тұжырымдар

  • Write a data dictionary.
  • Document transformations.
  • Prepare reproducible and permitted sharing.
Мазмұны

Preserve raw records and create a traceable path to every reported result.

A reproducible result has a traceable path from recorded observations to the number, quotation, table or figure in the paper. Organisation is therefore part of the research method. If you cannot identify which input produced a panel, you cannot reliably explain that panel to a reviewer or repair an error discovered later.

In this guide, computational reproducibility means obtaining the reported computational result from the same data and a sufficiently specified workflow. Repeating a study with new observations addresses a different question. Reproducing an analysis also does not establish that the original design, measurement or interpretation was valid.

This tutorial explains how to preserve observations, structure data, document transformations and prepare a useful research package. All numerical examples are synthetic teaching examples.

Preserve the original record

Keep raw data separate from derived analysis files. Raw data are the records as acquired or received, such as instrument output, survey exports, transcripts or downloaded public datasets. Preserve their provenance, meaning where they came from and how they were obtained. Record acquisition dates, instrument settings, software versions and relevant permissions.

Make corrections through a documented process. Preserve the original value, the corrected value, the reason and who made the decision. A transcription error can be corrected when the source record supports the correction. A surprising observation should not be replaced merely because it conflicts with an expected result.

Use stable identifiers such as specimen_014 or layout_012. Do not encode evaluative judgments in identifiers such as bad_sample or good_result. Keep identifiers consistent across notebooks, files and scripts. Participant names should not appear in working filenames where study codes can serve the purpose.

Raw observations lead through documented cleaning and analysis to tables and figures. A dictionary and change log support the whole process.

Build a project structure that expresses the workflow

A simple project can separate original inputs, derived data, analysis scripts, outputs and documentation. The exact folder names matter less than their roles. A useful README explains where each role lives, what can be regenerated and which files require controlled access.

For example, original sensor exports could remain in an input directory, cleaned measurements in a derived directory, scripts in an analysis directory, and manuscript figures in an output directory. Avoid manually editing the derived table after the analysis script has produced it. If a correction is necessary, change the documented transformation or correction record and regenerate the output.

Version control records changes in text files and code. It is not automatically a suitable storage system for large recordings, confidential participant data or access credentials. Use the institution’s approved storage and backup arrangements for those materials. Wilson and colleagues provide practical guidance for research computing, including organisation, documentation and preserving the steps that produce results. S103

Define what one row represents

A data table becomes interpretable when each row and column has a defined meaning. In a tidy representation, each variable occupies a column, each observation occupies a row, and each type of observational unit has a table. Wickham describes this structure and why it supports consistent analysis. S102

A row is not necessarily an independent unit. A table of repeated readings can have many rows for one participant, culture or component. Include identifiers that preserve those relationships. Do not rely on spreadsheet row order to remember which records belong together.

layout_id controller run_id error_mm completed
L01 A 1 18 yes
L01 A 2 20 yes
L01 B 1 15 yes
L01 B 2 16 yes

These four invented rows represent two repeated runs for each controller in one layout. They do not represent four independent layouts. A long format like this makes the repetition explicit. A wide table with separate columns for A and B can be convenient for a paired summary, provided the transformation preserves the layout identity and is documented.

Store numbers as numbers. Put units in variable names or the dictionary, rather than mixing entries such as 18 mm, 19 and 0.020 m in the same numeric column. Keep dates in a consistent, documented format and record time zones when timing matters.

Write a data dictionary before analysis

A data dictionary defines each variable’s meaning, type, units, allowed values, missingness convention and derivation. It should allow someone unfamiliar with the project to interpret a row correctly.

For error_mm, specify the error metric, reference measurement, evaluation period and aggregation rule. Mean absolute error and root mean square error are different metrics; “error” alone is insufficient. For a questionnaire score, record the scale version, item coding, reverse scoring, permitted missing items and total score rule. For a biological variable, record the measurement basis and any normalisation.

Distinguish a real zero from a missing value. Zero failures, an unobserved failure count and an unsuccessful attempt to measure failures are different states. Avoid placing a code such as 999 inside a numeric variable without an explicit conversion rule. Otherwise an analysis may treat it as an extreme real measurement.

Hypothetical dictionary entry. error_mm is numeric, measured in millimetres, and records the mean absolute position error during the specified evaluation interval. Values are nonnegative. Missing values indicate that the metric could not be calculated; the reason is stored in a separate status field. Completion failures follow the protocol’s outcome rule rather than being silently removed.

Use Template 5 to record the dictionary and integrity checks.

Validate structure before interpreting patterns

Check that required identifiers are present and unique where expected. If each layout, controller and run combination should occur once, a duplicate combination needs investigation. It might be an accidental duplicate, a legitimate additional run or an incorrect identifier. Do not automatically delete duplicates without identifying what they represent.

Check permitted ranges and categories. A negative duration or an impossible response category may indicate a conversion or entry error. Investigate against the original record. A value outside an expected scientific range is a flag for review, not automatic proof that the observation is invalid.

Check relationships between variables. An end time should follow its start time under the documented timing convention. A total score should agree with its component items and scoring rule. Counts in a manuscript flow diagram should reconcile with the data processing record.

Check merges especially carefully. A join combines tables using shared keys. If a participant table unexpectedly contains two rows for one participant, joining it to repeated measurements can multiply rows and inflate counts. Record the expected relationship, such as one participant record to many measurement records, and compare row counts before and after joining.

Make cleaning decisions visible

Cleaning can include correcting documented entry errors, converting units, standardising categories and creating derived variables. It should not mean removing observations until a preferred conclusion appears. Preserve a log with the affected identifier, operation, reason and date.

For exclusions, distinguish the reason from the consequence. “The file was unreadable because acquisition failed” is a specific reason. “The value made the group difference nonsignificant” is not an acceptable data quality criterion. If a defensible exclusion rule is introduced after seeing the data, disclose the timing and consider an analysis that shows the effect of the decision.

For qualitative material, the same principle applies differently. Document transcription conventions, anonymisation changes, coding versions and analytic memos. Do not silently rewrite a participant’s words to fit a theme. If quotations are shortened, translated or paraphrased, explain the relevant convention without exposing identities.

Work through a complete transformation

Imagine a synthetic engineering dataset with twelve layouts, two controllers and three runs for every layout and controller. The complete table contains 72 rows because 12 × 2 × 3 = 72. The planned comparison is across layouts, with each layout evaluated using both controllers.

First, validate that all 72 expected combinations are present and that their status fields are consistent. Second, apply the prespecified rule for runs and failures. Third, calculate the mean error across the three runs within each layout and controller. This creates 24 controller summaries. Fourth, pair the A and B summaries by layout_id and calculate A minus B, producing twelve layout differences.

For a single illustrative layout with A values 18, 20 and 19 and B values 15, 16 and 17, the controller means are 19 and 16 mm. The paired difference is 3 mm. The analysis should retain the three levels of information: original runs, controller summaries and layout differences. That record makes it possible to trace the final result back to the inputs.

This workflow is appropriate only if averaging runs answers the intended question and the assumptions about layouts are justified. It is not permission to average away failure, unequal observation times or dependence that matters to the estimand. Record those decisions in the protocol and Methods.

Preserve the computational environment

Record the software, package versions, operating environment where relevant, and parameters needed to rerun the analysis. A script name without its dependencies is incomplete documentation. Where feasible, provide an environment specification or another precise installation record.

A random seed controls a pseudorandom sequence within a specified computational process. It can make resampling or simulation repeatable, but does not guarantee identical output across different software versions, hardware or parallel execution settings. Record both the seed and the procedure it belongs to. Multiple seeds may be relevant when performance varies across random initialisations.

A checksum is a value calculated from file contents that can help detect whether a file has changed. It does not establish that the file is scientifically correct. A code version identifier similarly locates a state of the analysis but does not validate its assumptions.

Run the final workflow from its documented starting point in a clean session. A successful run should not depend on a manually created variable left over from yesterday’s notebook. Identify steps that require restricted inputs so others understand what they can and cannot reproduce.

Plan sharing with sufficient context

FAIR means findable, accessible, interoperable and reusable. These principles concern the usefulness of research objects and their metadata. Accessibility can include authentication and access conditions; FAIR does not require publishing confidential data without restriction. S08

A useful shared package includes the permitted data, dictionary, code, README, relevant environment information and a licence or access statement. Cite the dataset and version in the manuscript when a persistent identifier is available. Preserve the link between the archived version and the results actually reported.

For restricted participant data, explain the permitted access process and provide shareable materials where appropriate, such as analysis code and synthetic demonstration inputs. Mark synthetic values clearly. They demonstrate how code runs but do not allow independent verification of the original empirical result.

For a review article, preserve the search log, exported records where redistribution is allowed, screening decisions, extraction forms and synthesis calculations. For a qualitative study, decide what analytic documentation can be shared without compromising participants. Reproducibility should fit the research approach and ethical permissions.

Audit the manuscript against the research package

Choose one main figure and trace every plotted value to its source. Check the independent unit, missingness, exclusions and aggregation. Then compare the figure, table, abstract and Results text. Differences in rounding may be harmless, but incompatible sample sizes or opposite subtraction directions require correction.

Ask a colleague to use the README without verbal guidance. Can they identify the inputs, understand the dictionary and locate the script producing the figure? Their difficulties reveal documentation gaps that familiarity can hide. A second person does not need to repeat every task to provide a useful check.

Practice. A fictional analysis contains 120 rows after merging a measurement table with a participant table, although the measurement table had 100 rows. What should you do before analysing it?

Model answer. Inspect key uniqueness and the intended join relationship. Identify which records multiplied and why. Resolve duplicate or mismatched identifiers using the source records, document the correction, and regenerate the merged data. Do not accept the larger count as additional evidence or delete arbitrary rows until the original total returns.

Use Template 6 to make the final workflow understandable. Good organisation earns its place in a paper by allowing readers to see how the reported evidence was produced.

Sources

S103: Wilson, G., Bryan, J., Cranston, K., Kitzes, J., Nederbragt, L., and Teal, T. K. (2017). Good enough practices in scientific computing. PLOS Computational Biology, 13(6), e1005510. DOI: 10.1371/journal.pcbi.1005510.

S102: Wickham, H. (2014). Tidy Data. Journal of Statistical Software, 59(10), 1 to 23. DOI: 10.18637/jss.v059.i10.

S08: Wilkinson, M. D., and colleagues. (2016). The FAIR Guiding Principles for scientific data management and stewardship. Scientific Data, 3, 160018. DOI: 10.1038/sdata.2016.18.