Жазу · Бастапқы
Write methods that another researcher can evaluate and repeat
Describe the design and analysis so readers can evaluate and repeat the work.
12 мин оқу · 2026 ж. 26 қаз.
Бұл бет әлі аударылмаған, сондықтан ағылшын нұсқасын оқып отырсыз.
Негізгі тұжырымдар
- Define measurements operationally.
- Expose selection and processing decisions.
- Use the reporting guidance for the design.
Мазмұны
Describe the design and analysis so readers can evaluate and repeat the work.
Methods explain how the evidence came into existence. Readers need to judge whether the design answers the question, whether important sources of error were addressed and how the reported analysis can be reconstructed. A list of equipment and software is not enough. The section must explain the decisions connecting the research question to the final observations.
Write Methods from the protocol, laboratory or field records, analysis scripts and decision log. Memory is unreliable, especially after several revisions. If a detail is unknown, retrieve it or state the limitation. Never replace a missing instrument setting, recruitment date or exclusion rule with a plausible value.
Begin with the design and the independent unit
State whether the study is experimental, observational, qualitative, computational, theoretical or a form of evidence synthesis. Describe the setting and relevant period. Explain the target population or system and how the investigated units were selected.
The experimental unit is the smallest unit independently assigned an intervention. The observational unit is the entity on which a measurement is recorded. The unit of analysis is the entity represented as an independent contribution in a particular analysis. These units may differ. Your Methods should show the relationship rather than using “sample” to mean all three.
For example, three measurements from each of ten independently prepared specimens give ten specimens and thirty readings. The extra readings can help assess measurement variability. They do not automatically give thirty independent specimens. If an intervention is assigned to classrooms, an analysis treating every student as an independent intervention replicate must account for the resulting dependence or be reconsidered.
A cluster is a group of observations sharing a relevant source, such as a class, household, patient, animal or production batch. Repeated measures are observations made more than once on the same unit. Both structures should be visible before a statistical test is named.
Use a workflow that a reader can follow
Organise the section around decisions, not the chronological order in which you wrote the report. A useful starting sequence is design and setting; units and eligibility; materials or data sources; procedures; outcomes; processing and quality checks; analysis; and approvals or access conditions. Adapt the order to your discipline.
| Methods component | Essential question | Example of information to provide |
|---|---|---|
| Selection | How did a unit enter the study? | Sampling frame, recruitment route, inclusion criteria |
| Assignment or comparison | Why are these groups comparable? | Randomisation procedure, matching or selection of controls |
| Measurement | What exactly was observed? | Instrument, scale, units, timing and scoring rule |
| Processing | How were observations transformed? | Calibration, filtering, normalisation and aggregation |
| Analysis | How was the question answered? | Estimand, model, dependence, uncertainty and diagnostics |
| Transparency | What can another reader inspect? | Protocol, code, materials, deviations and access limits |
The sampling frame is the set from which units could be selected. Eligibility criteria determine which units qualify. These are different from criteria for excluding an observation after measurement, which should also be reported and justified.
Define every important outcome operationally
An operational definition turns a concept into a reproducible measurement rule. “Performance improved” is an interpretation. “Completion time was measured in seconds from the start signal to successful placement of the final object” defines an outcome.
State the variable, units, measurement time, instrument or observation procedure, and scoring or aggregation rule. For a biological assay, specify the relevant sample preparation, reagent identifiers, acquisition conditions, controls and processing. Include enough information to evaluate specificity and measurement limits. Do not copy a lengthy manufacturer's description while omitting the settings you actually used.
For a survey, identify the instrument, response options, language, score range and treatment of missing items. If you changed wording or removed items, describe the changes. Reliability concerns consistency of measurement; validity concerns whether the interpretation of a measure is justified for its purpose. A measure can be consistent while measuring something different from the intended construct. A reliability coefficient alone does not establish validity.
For an engineering evaluation, define the reference against which error is calculated. Explain whether an outcome averages over time points, runs, devices or environments. State how failures are represented. Dropping failed trials from an accuracy analysis can make a system appear stronger than it is.
Describe the comparison and measures against bias
If random assignment occurred, explain the procedure and who implemented it. “Randomly selected” should not mean that the researcher chose convenient units without an explicit rule. Distinguish random sampling from random assignment: the first concerns selection into a study; the second concerns allocation within a study.
Describe masking or blinding, meaning that relevant people did not know group identities during a specified stage. Name the stage and people: measurement, outcome assessment or analysis. If masking was impossible or not performed, report that fact and any alternative measures taken to reduce judgement driven differences.
Explain control conditions, order of procedures, batch handling and deviations. A negative control can help reveal background signal or contamination, while a positive control can show whether a system can detect an expected response. The appropriate controls depend on the scientific question. A control does not remove every possible source of bias merely because it exists.
Report data processing as part of the method
Describe calibration, data cleaning, transformation and aggregation in the order applied. Normalisation changes values relative to a specified reference or scale. Standardisation often means subtracting a mean and dividing by a standard deviation, although terminology varies. State the exact operation instead of assuming a name is unambiguous.
Record exclusion thresholds and when they were chosen. Explain whether a rule was applied without knowing the outcome or group. Report numbers removed in Results and preserve an audit trail. A rule developed after seeing a result is not automatically invalid, but the timing matters and should not be disguised as a planned decision.
For computational work, separate model fitting from evaluation. Identify the dataset version, partitioning rule, preprocessing parameters, hyperparameter selection and final test procedure. A hyperparameter is a setting chosen outside the model's ordinary parameter fitting, such as tree depth or a regularisation strength. If evaluation information influenced that choice, report the limitation rather than describing the evaluation as untouched.
Describe the analysis as a model of the question
An estimand is the quantity the study aims to learn about, such as a mean difference in a specified population under specified conditions. Name it before naming the test. Identify the outcome, comparison, covariates, independent units, model or test, uncertainty interval and relevant assumptions.
Specify how paired, repeated or clustered observations were represented. Explain missing data handling, multiple comparison procedures, diagnostic checks and sensitivity analyses. A sensitivity analysis asks whether a conclusion changes under a justified alternative assumption or decision. It should address a meaningful uncertainty, not provide a menu from which to select the preferred result.
For a regression, report the response distribution and link function when relevant, predictor coding, reference categories, interactions and model specification. For an interview study, explain the analytical approach, development of codes or interpretations, researcher involvement and handling of conflicting material. For a review, describe the search, selection, extraction, appraisal and synthesis methods. PRISMA provides reporting resources for systematic reviews, including the search and selection process. S06
Do not list methods you considered but did not use as though they all contributed to the reported result. Distinguish planned analyses from later exploration. Report the software and relevant package versions, but make the analysis understandable without relying solely on the software name. Cite important software using its recommended reference and identify the version used. S207
Annotated example: a natural science Methods passage
The following is a hypothetical design exercise, not a report of an experiment that occurred.
The proposed comparison uses twelve independently prepared growth containers, with six allocated to each light schedule using a recorded random sequence. Four seedlings are measured within each container. The container is the experimental unit because all seedlings within it receive the same assigned schedule. Height is measured from the substrate surface to the defined tip of the main stem at the start and end of the observation period. The primary outcome is the container mean of the individual height changes. Measurement instructions and the handling of damaged seedlings are specified before data collection.
The primary analysis estimates the difference in mean container outcomes between schedules, with an uncertainty interval appropriate to the prespecified analysis. Individual seedling measurements are retained for descriptive plots and checks of within container variation. They are not treated as forty eight independent treatment replicates. Records include container assignment, measurement dates, any failed measurements and departures from the planned procedure.
The passage makes the hierarchy explicit and defines the outcome. It is not a complete protocol: an actual study must supply the real light schedules, species, observation period, environmental conditions, sample size rationale and analysis specification. The example deliberately refuses to invent them. If the actual independent assignment occurs at a chamber rather than a container, the unit and design must change accordingly.
Annotated example: a social science Methods passage
The following is another fictional reporting exercise.
The hypothetical survey examines research confidence among students enrolled in one introductory course. All enrolled students are invited through the same course announcement, and participation is voluntary. The Methods section records the invitation period and defines eligibility before the responses are inspected. Attendance is recorded as the number of optional mentoring sessions attended, while confidence is measured using a named questionnaire with an explicitly stated scoring rule. Missing questionnaire items are handled according to a rule justified for that instrument rather than automatically replaced with the overall mean.
The analysis estimates the association between attendance and confidence, with the relevant prior experience measure included because it was identified as a potential common cause before analysis. The paper explains the coding of both variables and reports whether the model assumptions were examined. Because students choose whether to attend and participation in the survey is voluntary, the analysis is not described as an estimate of a randomly assigned intervention effect. The actual recruitment counts, response rate and analysis denominator appear in Results.
The paragraph separates recruitment, measurement and causal interpretation. It also shows why “we ran a regression in statistical software” is insufficient. An actual paper would supply the instrument reference, model specification, justification for covariates and appropriate consent and review information. APA JARS provides relevant guidance for quantitative and qualitative psychological research; use the sections matching your design. S200
Annotated example: engineering and computational work
The supplied controller example contains twelve constructed pairs rather than a physical evaluation. A truthful Methods passage is:
The teaching dataset contains twelve case identifiers, each with one positioning error value for controller A and controller B. The values were constructed for instruction and were not generated by a robot or implemented simulator. For the calculation, the cases are treated as independent and the two values within each case as paired. We define the difference as A minus B so that a positive value indicates lower error for B. All twelve differences enter the mean and t based confidence interval calculation; no pair is excluded.
A real controller paper would need much more: controller specifications, implementation versions, environment generation, sensor and actuator assumptions, calibration, timing, random seeds where relevant, repeated runs, failure handling and the basis for selecting benchmark cases. Reporting the paired calculation correctly does not establish that the physical or simulated evaluation was adequate.
For a machine learning paper, similarly describe what constitutes a genuinely new case. Splitting images at random is not a convincing test on new patients if images from the same patient appear in both training and test sets. The unit relevant to deployment should guide the split and evaluation design.
Choose reporting guidance by design
EQUATOR's directory helps locate reporting guidance, but reporting checklists are not tools for proving design quality. A clearly reported biased study remains biased. S02
ARRIVE 2.0 addresses animal research. STROBE covers major observational epidemiological designs. CONSORT 2025 addresses randomised trials. These resources provide different sets of reporting expectations because the threats to interpretation differ. Engineering and social science authors should also consult the standards of their field and destination. S03 S04 S05
State the actual ethics review, consent, data access or exemption status where applicable. Do not decide retrospectively that approval was unnecessary solely because the project was small or conducted by students. Article 5 explains how to establish the relevant requirements before work begins.
Check whether the section can survive an independent reading
Ask a reader to reconstruct the pathway from selection to the main result. They should identify what was measured, what counted as an independent unit, what was excluded, what the comparison means and which assumptions are needed. If several reasonable choices remain hidden, revise the Methods.
Practice: A report states, “We measured five samples three times and analysed n = 15.” What is missing? Model answer: The identity and independence of the five samples, the purpose of the repeated readings, treatment allocation and the rule connecting readings to the analysis. If they are five independent specimens with three technical readings each, averaging the readings may produce five analysis values. A model representing the measurement hierarchy may also be suitable, but treating fifteen readings as independent specimens is not justified by the count alone.
Use Template 2, Template 4 and Template 7 together. The protocol records intentions, the decision log records changes, and Methods describes what actually happened.
Sources
S02: EQUATOR Network. Reporting guidelines
S03: ARRIVE. The ARRIVE guidelines 2.0
S04: STROBE. Statement and scope
S05: CONSORT 2025 statement, EQUATOR record
S06: PRISMA. PRISMA 2020 resources
S200: American Psychological Association. Journal Article Reporting Standards (JARS).