Skip to main content
Policy Authority
Category: Internal Audit

Sample Testing

Also known as: Sampling and Testing
Simply put

Sample testing is the practice of examining a small, representative portion drawn from a larger population or batch to draw conclusions about the whole. Rather than checking every item, an examiner selects a subset and evaluates it to judge quality, conformity, or condition. This approach is commonly used where testing every item would be impractical.

Formal definition

Sample testing is a method in which a representative subset is selected from a defined population and evaluated to make inferences about the characteristics of that population. In many contexts, the process combines a sampling stage, in which items are selected and, where relevant, recovered or prepared to preserve their integrity, with a testing or analysis stage, in which the selected items are measured against defined criteria. The reliability of any conclusion depends on how representative the sample is; a subset that is not representative may limit the validity of the inference drawn about the wider population. This entry addresses the general concept and does not cover specific statistical sampling methodologies, sample-size determination, or sector-specific testing protocols, which vary by context.

Why it matters

Sample testing addresses a practical constraint that recurs across compliance, quality assurance, and audit work: examining every item in a large population is often impractical, costly, or destructive to the items themselves. By evaluating a representative subset, an organization can form a reasoned judgment about the quality, conformity, or condition of the whole without exhausting resources on complete inspection. This makes it a foundational technique wherever conclusions about a population must be drawn efficiently.

The value of the approach, however, is only as strong as the representativeness of the sample selected. A subset that does not reflect the wider population may lead an examiner to conclusions that do not hold for the whole, weakening the reliability of any inference. This is why the integrity of the sampling stage matters as much as the analysis itself; contamination, poor handling, or a non-representative selection can undermine otherwise sound testing. In fields such as drinking water analysis, hygienic sampling is treated as a critical component precisely because a compromised sample can invalidate the result.

Because of these dependencies, sample testing should be understood as a method that supports informed judgment rather than one that guarantees certainty about every item in a population. Users relying on sampled conclusions should remain aware of the residual possibility that unsampled items differ from those examined.

Who it's relevant to

Compliance Officers
Those responsible for assessing adherence to policies and external requirements often rely on sample testing where full-population review is impractical, using a representative subset to judge conformity across a larger population.
Quality Assurance Personnel
Staff evaluating the quality or condition of products or substances use sampling and testing to draw conclusions about a batch, where hygienic and careful sampling is important to preserving the integrity of the result.
Laboratory and Analytical Teams
Analysts who recover, prepare, and measure samples against defined criteria depend on sound sample processing, including preparation to preserve integrity and remove potential interferents, to produce valid measurements.
Internal Auditors and Assurance Functions
Assurance professionals commonly examine a selected subset of items to form conclusions about a wider population, and should remain mindful that the reliability of their inference depends on how representative the sample is.

Inside Sample Testing

Sample Selection
The process of choosing a subset of items from a larger population of transactions, records, or events to be examined, rather than testing every item. Selection methods may be statistical (such as random or systematic sampling) or non-statistical (such as judgmental or haphazard selection), depending on the objective and the desired ability to project results.
Population Definition
The complete set of items from which a sample is drawn, defined by the tester to align with the control or assertion being evaluated. Accurate definition of the population, including its completeness and the period covered, is a prerequisite for meaningful conclusions.
Sample Size
The number of items selected for testing, which commonly depends on factors such as the assessed level of risk, the frequency of the control's operation, the desired level of assurance, and any tolerable rate of deviation. Sample size may be determined statistically or by reference to established guidance rather than a fixed universal number.
Testing Attributes and Criteria
The specific characteristics or conditions each sampled item is examined against, such as evidence that a control operated as designed. Clear criteria help ensure that results are consistent and that deviations are identified objectively.
Evaluation of Results
The analysis of exceptions or deviations identified in the sample, including consideration of their nature and cause, and, where a statistical approach is used, the projection of results to the wider population. Non-statistical sampling typically does not support formal projection.

Common questions

Answers to the questions practitioners most commonly ask about Sample Testing.

Does passing sample testing prove that a control operated effectively at all times?
No. Sample testing examines a subset of a population rather than every instance, so it provides evidence about the sampled items and supports an inference about the broader population; it does not verify each individual occurrence. A conclusion that a control operated effectively is typically expressed with a degree of confidence rather than as a guarantee, and the possibility of undetected exceptions in unsampled items remains. Results should be read as reasonable, not absolute, assurance.
Is sample testing the same as continuous or full-population monitoring?
No. Sample testing evaluates a selected portion of a population, usually at a point in time or over a defined period, whereas full-population or continuous approaches examine every transaction or event, often on an ongoing basis. They serve different purposes and involve different cost and coverage trade-offs. Where data and tooling permit, some functions supplement or replace sampling with full-population analysis, but that is a distinct technique and out of scope for this entry.
How is the sample size for control testing typically determined?
Sample size commonly depends on factors such as the frequency of the control's operation, the assessed risk, the level of assurance sought, and the tolerable rate of deviation. Some functions use statistical sampling to derive a size that supports a defined confidence level, while others apply non-statistical or judgmental guidance based on control frequency. The appropriate approach varies by framework, function, and organizational policy, so testers should follow their applicable methodology rather than a single fixed number.
How should items be selected to keep a sample unbiased?
Selection methods commonly include random or systematic selection for statistical samples, and haphazard or judgmental selection for non-statistical samples. To reduce bias, the population should first be defined completely and accurately, and the selection method should give relevant items an appropriate chance of being chosen rather than favoring convenient or familiar items. The chosen method and its rationale are typically documented to support the reliability of the conclusion.
What should a tester do when an exception is found in a sample?
An identified exception is generally investigated to understand its nature, cause, and whether it reflects an isolated instance or a systemic weakness. Depending on the methodology, testers may evaluate the exception against a tolerable deviation rate, consider whether the sample remains representative, and determine whether additional testing is warranted. Findings are typically documented and communicated to appropriate parties. Conclusions about the control's overall effectiveness follow from this evaluation rather than from the presence or absence of a single exception alone.
How does sample testing preserve the independence of assurance functions?
When performed by an assurance function, such as internal audit, sample testing is intended to be conducted independently of the management activities and controls being examined. This means the tester should not be evaluating work they themselves performed or controls they are responsible for operating. Maintaining this separation supports the objectivity of the conclusion. Testing carried out by management as part of its own monitoring is a distinct activity and does not provide the same independent assurance.

Common misconceptions

A clean sample proves that no errors or control failures exist in the population.
Sample testing examines only a subset of items and provides evidence about, not a guarantee of, the condition of the population. A sample with no exceptions reduces but does not eliminate the possibility that deviations exist elsewhere in the population; this is commonly described as sampling risk.
Larger sample sizes are always better and more rigorous.
Appropriate sample size typically depends on the assessed risk, control frequency, desired assurance, and tolerable deviation rate. A sample that is larger than necessary can consume resources without proportionate benefit, while the right approach balances sufficiency of evidence against efficiency.
Sample testing performed by management is equivalent to independent assurance testing.
The same sampling technique can be used by different functions, but its purpose and independence differ. Testing performed by management as part of operating or monitoring controls is a management activity, whereas testing performed by an independent assurance function is intended to provide objective evaluation. Keeping these roles distinct is important to preserve the independence of assurance.

Best practices

Define the population precisely before selecting a sample, confirming its completeness and the period it covers so that conclusions relate to what was intended to be tested.
Choose a selection method suited to the objective, and use statistical sampling where the ability to project results to the population is required, recognizing that non-statistical approaches typically do not support formal projection.
Determine sample size with reference to relevant factors such as assessed risk, control frequency, desired assurance, and tolerable deviation rate, rather than defaulting to an arbitrary fixed number.
Document the testing attributes and criteria in advance so that deviations are identified consistently and objectively.
Investigate the nature and cause of any exceptions identified, rather than treating a deviation solely as a numerical result.
Maintain clear separation between management's own testing and independent assurance testing to preserve the independence and objectivity of assurance functions.
Application Security Isn’t Optional Anymore.