Terms Dictionary
A listing of each performance term identified in the PHES-EF scoping review.
Ability to contin
ID: abilityToContinueTrackingAnEvent
See also: N/A
Synonyms: Exact: stability (reliability) • Broad: N/A
Definition: The ability of the surveillance system to continue tracking an event throughout the event life cycle.
Description: An event-based biosurveillance may be quite successful in detecting or forecasting an outbreak of disease, but it may lose its sensitivity as the event progresses (e.g., due to a high background level of indicators following a media announcement).
Background: This reported concept was only identified by Corley 2012 in the context of biosurveillance models and systems.
Measurement: N/A
Version 0.2
Acceptability
ID: acceptability
See also: engagement; collaboration
Synonyms: Exact: N/A • Broad: participation
Definition: The willingness of persons and organizations to participate in the surveillance system.
Description: Acceptability reflects stakeholders’ perceptions and attitudes toward the surveillance system, including their willingness to participate. It encompasses recognition of the system’s importance, satisfaction with processes, trust in data management, perceived reporting burden, and alignment with ethical and cultural expectations. High acceptability is supported by clear communication, demonstrated value, privacy protections, and responsiveness to user needs.
Background: Acceptability was identified as a foundational concept in the evaluation of public health surveillance, being identified on 34 occassions. In our review, the concept was identified as early as 1988, to reflect "the willingness of individuals and organizations to participate in the surveillance system". Overall, definitions have remained largely congruent with the one introduced in 1988. However, the concept has evolved through the incorporation of new elements such as trust and engagement.
Measurement: TBD
Version 0.2
Acceptability and engagement
ID: acceptabilityAndEngagement
See also: acceptability; engagement; collaboration
Synonyms: Exact: N/A • Broad: participation
Definition: N/A
Description: Acceptability and engagement reflect the willingness to participate in and the degree of involvement with a surveillance system. They are shaped by system design (e.g., simplicity, timeliness), perceived value (e.g., importance, relevance, trust), and social context (e.g., cultural fit, stigma, privacy concerns). High levels of acceptability and engagement improve data quality, sustainability, and stakeholder collaboration but may be limited by political will, resource constraints, and communication barriers.
Background: Acceptability as a surveillance system attribute has evolved since its initial conceptualization. The original CDC 1988 definition focused on willingness of individuals and organizations to participate in surveillance activities. The 2001 CDC guidelines maintained this core concept while expanding factors that influence acceptability. Calba et al. (2015) broadened the concept to include not just willingness but "the degree to which each of these users is involved in the surveillance." Contemporary understanding has further expanded to encompass community engagment, participation, and trust—reflecting a shift from traditional, top-down surveillance approaches to more collaborative models. This evolution acknowledges that modern surveillance systems must be acceptable to a wider range of stakeholders beyond traditional public health departments, including affected communities, healthcare providers, private sector entities, and various levels of government. The concept now implicitly recognizes cultural appropriateness, and ethical considerations as components of acceptability.
Measurement: Measuring acceptability often requires qualitative methods to capture user perspectives and motivations.
Version 0.2
Accessibility (data)
ID: accessibilityData
See also: data management and storage; interoperability
Synonyms: Exact: N/A • Broad: N/A
Definition: The ease with which authorized users can retrieve surveillance data and metadata using standardized protocols.
Description: [To incoporate FAIR in description]. Accessibility in public health surveillance refers to how readily and consistently surveillance outputs (such as alerts, reports, or predictive model results) and input data (such as raw surveillance data or supporting information) are available to end-users, decision-makers, and planners. This includes availability through standardized and sustainable protocols, user-friendly dissemination mechanisms, and manageable data formats. Accessible systems reduce technical, administrative, logistical, and practical barriers, enabling timely decision-making and effective public health responses.
Background: FAIR data concepts, introduced by Yang et al. in 2022, have become increasingly prominent. Definitions of accessibility commonly include providing data “when and how users need them,” although these elements overlap with system availability and communication and dissemination. Accessibility involves both output accessibility (ease of disseminating surveillance results to users) and input accessibility (consistent availability and usability of source data). Effective accessibility also requires sufficient human and technological resources, clear authorization balancing openness with privacy and security, and robust infrastructure for reliable data flow.
Measurement: TBD
Version 0.2
Accuracy
ID: accuracy
See also: precision; bias
Synonyms: Exact: validity, data correctness, bias (Peyre 2019, 2022), comparability • Broad: data quality?
Definition: The proportion of data entries that correctly reflect the true value of the data collected.
Description: [Comparison to the reference standard, ISO,varying definitions]. Accuracy is influenced by several factors, including the sensitivity and specificity of case definitions, the clairty of surveillance forms (whether hardcopy or electronic), the technical competence and training of staff, and the care exercised in data management. It is positively correlated with several other data quality attributes/terms, including representativeness, precision, completeness, and consistency, and is negatively correlated with bias. System acceptability can influence how accurately data are captured and recorded.
Background:Accuracy was identified as an independent concept on eight occassions, distinct from the definitions of "data quality" or "data completeness and correctness". However, this concept has been defined inconsistently across the literature, with terminology varying by discipline - often referred to as validity, accuracy, or data correctness. The definition of accuracy that we have adopted aligns with the DQO and ISO/IEC 25012 definition.
The definition aligns with established international standards, including ISO 8000-2:2020, which defines accuracy as the correctness of data relative to real-world conditions. It also corresponds to definitions in the Data Quality Ontology (DQO), which emphasizes accuracy as the correctness and precision with which data represent reality. Accuracy assessment often involves validation procedures, comparisons against reference standards, or independent verification processes.
Measurement: The accuracy of a surveillance system can be accessed using historical data or simulations. To assess its effectiveness in detecting outbreaks, key metrcis should include the number of outbreaks detected, false alarms, and outbreaks missed or detected late. These assessments should be integrated into the system’s routine workflow and have the ability to be conducted with minimal effort or complexity. Routine reporting should be automated where possible. Essential data include: the number of statistical aberrations detected at a set threshold in a defined period of time (e.g., frequency per month at a given p-value); actions taken in response to signals (e.g., review for data errors, in-depth follow-up analysis of the specific conditions within the syndrome category, manual epidemiologic analysis to characterize a signal, examining data from other systems, and increasing the frequency of reporting from affected sites); resources allocated for follow-upt; public health response (e.g., an alert to clinicians, timely dissemination of information to other health entities, a vaccination campaign, or no further response); documentation of how every recognized outbreak was detected; an assessment of the value of the follow-up effort (e.g., the effort was an appropriate application of public health resources); a detailed description of the agent, host, and environmental conditions of the outbreak; and the number of outbreaks detected only late in their course or in retrospect.
Version 0.2
Adaptability
ID: adaptability
See also: N/A
Synonyms: Exact: Flexibility • Broad: N/A
Definition: N/A
Description: N/A
Background: –
Measurement: –
Adherence
ID: adherence
See also: compliance
Synonyms: Exact: N/A • Broad: standards use
Definition: The degree to which a surveillance system follows and implements established guidelines, protocols, classification systems, and data exchange standards in its core and supporting functions.
Description: Standardization encompasses adherence to recognized guidelines across multiple dimensions of surveillance systems, including methodological protocols, data structures, classification systems, and exchange formats. This attribute covers both technical standards (data formats, interoperability frameworks) and operational standards (case definitions, reporting procedures). Standards use is closely related to compliance, but differs in focus. While standards use focuses on internal consistency in following best practices and operational protocols, compliance is centred around relevant legislation, regulations and policies, including ethics and security. Beyond compliance, standards use is also tied to other data quality attributes/terms, and is positively correlated with consistency ans technical competence and training.
Background:The concept of adherence was identified six times in our scoping review. However, the overarching concept referred to here was only identified by Yang et al., (2022), with elements of this definition being referenced multiple times throughout the literature (e.g., standards use, transparency).
High adherence is essential for the consistent functioning of the system, as it reduces errors, ensures uniformity, and facilitates the accurate tracking of health-related events or environmental changes. A system with high adherence is more likely to produce accurate and high-quality data, as following established protocols reduces errors and inconsistencies. It also enhances consistency by ensuring uniform application of procedures across time and by different personnel. Furthermore, training and supervision play a critical role in maintaining adherence, as they ensure that individuals are well-equipped to follow protocols correctly and consistently.
Measurement: Evaluating adherence involves assessing key factors such as the presence of an auditing process to ensure protocols are consistently followed, and whether data collection methods are standardized to maintain high-quality data. It also requires examining the use of harmonized indicators and metrics across sectors to ensure consistent data interpretation. Additionally, adherence can be evaluated by reviewing the use of standardized data elements (e.g., LOINC for lab tests) and electronic data interchange protocols (e.g., Health Level 7). Regular documentation of system protocols, including surveillance strategies and methodologies, is important. Additionally, verifying the enforcement of mandatory requirements for system implementation and data reporting is critical for ensureing full adherence
Version 0.1
Availability (system)
ID: availablity
See also: Accessibility (data); N/A
Synonyms: Exact: N/A • Broad: N/A
Definition: The ability to be operational when needed (Groseclose 2017; Peyre 2019, Peyre 2022; Muellner 2018; Hoinville 2013; Yang 2022; Albali 2023; Amato 2023; Anonymous 2001; Azofeifa 2018; Babaie 2015; Baker 2006; Buehler 2004, 2008; Calba 2015; Drewe 2012, 2015; Harper 2011; Innes 2022; Lucero-Obusan 2022; Marbus 2020; Meynard 2008; Ng’etich 2021; Sosin 2003; Bordier 2019; Velasova 2015).
Description: Availability reflects a system’s readiness to reliably collect, process, and disseminate surveillance data when required. It encompasses the continuous functioning of essential components, including infrastructure, skilled personnel, and communication channels, ensuring timely public health responses.
Background: –
Measurement: –
Benefit
ID: benefit
Synonyms: Exact: N/A • Broad: N/A
Definition: The direct and indirect advantages produced by the surveillance system.
Description: [Carol’s + time horizon component]. The benefit term is influenced by several factors, including the scope of surveillance objectives, the effectiveness of data use in decision-making, and the capacity for early detection and response. It encompasses direct and indirect outcomes such as cost savings, improved public and animal health, enhanced trade, and prevention of losses in productivity, morbidity, and mortality. Benefits are positively correlated with terms such as timeliness, usefulness, and stability, and are shaped by stakeholder engagement and collaboration. The distribution of benefits across sectors also informs the overall value of the surveillance system.
Background: Benefit was identified as an independent concept nine times in our scoping review, distinct from related concepts such as costs, impact, and efficiency. The concept was first introduced by Hoinville et al., in 2013 and was consistently included in animal health surveillance system evaluation frameworks thereafter. The definition has remained largely unchanged with the execption of Calba et al., 2015 introducing the idea of assessing whether users are satisfied that their requirements have been met in their definition. Their paper also focused solely on non-monetary benefits. However, despite the alignment of this definition across animal health surveillance evaluation frameworks, benefit was not identified as a standalone concept in the context of public health surveillance.
Measurement: The evaluation methods for the benefits of surveillance activities involve identifying and quantifying both direct and indirect benefits. One approach includes contingent valuation methods (CVM) like proportional piling, where stakeholders are asked what they would be willing to pay for improvements such as sanitary information. The evaluation process should list all potential benefits, categorize them, and quantify market benefits where possible. Non-monetary benefits, such as improved public health or consumer confidence, should also be considered using alternative methods like quality-adjusted life years (QALY). Additionally, the distribution of benefits across various stakeholders, such as producers, consumers, and the livestock industry, should be assessed. The relationship between surveillance, intervention, and the mitigation of losses should be explored, as many benefits arise from integrated disease control. Finally, the evaluation should capture indirect benefits like enhanced trade opportunities and the improved capacity to respond to future threats or outbreaks.
Version 0.2
Bias (ascertainment performance)
ID: biasAscertainmentPerformance
See also: accuracy; precision; sensitivity; specificity; representativeness; data quality; predictive value positive; false detection rate
Synonyms: Exact: N/A • Broad: N/A
Definition: The extent to which a prevalence estimate produced by the surveillance system deviates from the true prevalence value.
Description: Case detection bias occurs when systematic measurement errors cause surveillance data to systematically differ from the true occurrence of the health event in the target population. Bias can arise from multiple sources including diagnostic test characteristics, reporting practices, population coverage, sampling methods, case definitions, and temporal factors. The magnitude and impact of bias depends on surveillance objectives. For general population surveillance, bias can distort prevalence estimates and trend analysis, leading to incorrect public health conclusions.
Background:Bias was identified as a reported concept six times in our scoping review. The concept was first introduced by Hoinville et al., in 2013 and was consistently included in animal health surveillance system evaluation frameworks thereafter. The definition has remained largely unchanged with the execption of Peyre 2019 & 2022 incorporating the idea of "bias=accuracy" in their definition. Despite its frequent application in evaluating animal health surveillance systems, the term "bias" was not identified in the context of public health surveillance.
Evaluating bias in surveillance systems is essential for accurately monitoring endemic diseases.
Measurement: Evaluating case detection bias (also called ascertainment bias) involves assessing systematic errors that affect a surveillance system’s ability to accurately identify and classify cases or events. This includes selection bias (when certain cases or populations are systematically over- or under-represented) and information bias (when data collection methods systematically affect measurement accuracy). Bias represents the consequence of measurement errors that can be characterized through surveillance performance measures such as sensitivity, specificity, and representativeness. Evaluating bias involves assessing potential sources, such as diagnostic sensitivity and specificity, underreporting in passive surveillance, and how the sample population is chosen (e.g., animals from abattoirs might not represent the overall population). Additionally, geographic, demographic, or ecological factors can contribute to bias. To quantify and correct for bias, methods like comparison across multiple data sources, simulation models, statistical methods, or capture-recapture analyses can be used. Assessment of bias (systematic error) should address the following: * Selection bias: systematic differences between sample and target populations * Performance bias: systematic differences between groups in care provided or in exposure to factors other than the interventions of interest * Detection bias: systematic differences in how the outcome is determined (eg, death scene investigation protocols) * Attrition bias: systematic loss to follow up * Reporting bias: systematic differences in how people report symptoms or ideation * Other: biases related to a particular data source
Version 0.2
Case definition
ID: caseDefinition
See also: bias (case detection)
Synonyms: Exact: N/A • Broad: N/A
Definition: The standard set of criteria used to identify and classify cases or events for consistent reporting and analyses within a surveillance system.
Description: The case definition affects surveillance system performance by determining how accurately and consistently cases are identified and classified. A well-designed case definition supports accurate and consistent detection, improving the quality of the data collected. It directly influences terms like sensitivity, specificity, representativeness, and coverage, and plays a role in determining the system’s cost, acceptability, and flexibility. Clear defined definition may also improve consistency across sites and over time.
Background: The concept of case definition was identifed five times throughout our scoping review.
Measurement: Qualitative assessment methods have been used to evaluate this concept, such as the use of the EVARISK tool.
Version 0.2
Collaboration
ID: collaboration
See also: acceptability and engagement; governance
Synonyms: Exact: partnerships • Broad: N/A
Definition: The active engagement and coordination among diverse partners and end-users to facilitate exchange of data, information, and knowledge, sharing of capacities, and collaborative implementation of surveillance activities.
Description: Collaboration in surveillance systems relies on clear roles, strong legal frameworks, and established inter-sectoral mechanisms. It occurs in four main areas: governance across sectors (health, animal, environment), decision-making at all levels (local to international), interdisciplinary efforts (biosciences, social sciences, engineering), and public-private partnerships (e.g., veterinary companies in antimicrobial resistance). Effective data exchange and a shared databases enhance collaboration. [still working on this one somewhat since multiple terms are being folded into this].
Background: TBD
Measurement: TBD
Version 0.1
Communication and dissemination
ID: communicationAndDissemination
See also: collaboration
Synonyms: Exact: N/A • Broad: N/A
Definition: The ability of the system to deliver information and data in a clear and distinct manner to relevant stakeholders inside and outside of the surveillance system.
Description: Effective communication and dissemination in surveillance systems involve sharing clear, relevant, and timely information with all stakeholders, actors, and end-users (both internally and externally). Surveillance data outputs must meet the needs of diverse users, including data providers, analysts, decision-makers, and the public. Outputs should be accessible, frequent, and based on up-to-date, high-quality data with clear explanations of limitations and biases. Communication and disseminationis is also closely linked to acceptability, timeliness, impact, and collaboration.
Background: TBD
Measurement: TBD
Version 0.1
Compatability
ID: compatibility
See also: Coherence; interoperability
Synonyms: Exact: N/A • Broad: Integration
Definition: Compatibility with and ability to integrate data from other sources and surveillance components, e.g., one health surveillance (part of data collection and data management) (Peyre 2019, Peyre 2022).
Description: N/A
Background: –
Measurement: –
Compliance
ID: compliance
See also: adherence
Synonyms: Exact: N/A • Broad: legislative support
Definition: The degree to which the surveillance system complies with all relevant legislation, regulations and policies, including ethics and confidentiality requirements.
Description: Compliance refers to meeting required legal or regulatory requirements, often with external oversight. It is closely related to adherence, but differs in focus. While compliance focuses on relevant legislation, regulations and policies, including ethics and security, adherence is centred around internal consistency in following best practices and operational protocols. Beyond adherence, compliance is also tied to other data quality terms, and is positively correlated with consistency and technical competence and training.
Background: The concept of compliance was identified twice in our scoping review. However, the overarching concept referred to here was only identified by Yang et al., (2022), with components of this definition being referenced multiple times throughout the literature (e.g., legislative support, security, ethical considerations).
Measurement: To evaluate compliance, system security policies and practices should be reviewed to ensure that security levels and procedures for surveillance system data or system access are defined and enforced. Data use and release policies and protocol should be available for review, and access to the surveillance system software applications should be controlled. [Need to add information about legislative support/policies and ethics].
Version 0.2
Consistency
ID: consistency
See also: N/A
Synonyms: Exact: N/A • Broad: repeatability, precision
Definition: Data have the same meanings (e.g. case definition, diagnosis standards) to allow for consistent interpretation. This includes internal consistency (within a dataset) and external consistency (across different data sources), as well as consistency over time.
Description: Consistency refers to the uniformity and reliability of data across time and sources within the surveillance system. It ensures that data remain unchanged when replicated, transferred, or integrated between partners. Consistent data collection methods, coding systems, case definitions, and documentation of transformations are essential for comparability and trend analysis.
Background: Consistency was identified five times in our scoping review.
Measurement: TBD
Version 0.1
Cost-benefit
ID: costBenefit
See also: cost minimization; costs; cost-effectiveness; cost-utility; benefit
Synonyms: Exact: N/A • Broad: N/A
Definition: An analysis where all costs and outcomes are expressed in monetary terms, calculating the net gain or loss by measuring both the costs of an intervention and the monetary value of its outcomes.
Description: Cost-benefit analysis quantifies both the costs and outcomes of a surveillance system in monetary terms to assess its net economic value. This allows comparison of investments against financial benefits such as reduced disease burden, healthcare savings, or productivity gains. It supports decision-making by highlighting the economic return of surveillance activities relative to their expense. Accurate valuation of outcomes and comprehensive cost accounting—including direct, indirect, fixed, and variable costs—are essential. This term complements cost-effectiveness and cost-minimization analyses by providing a broader economic perspective on surveillance system value.
Background: TBD
Measurement: TBD
Version 0.2
Cost-effectiveness
ID: costEffectiveness
See also: cost minimization; cost-benefit; costs; cost-utility; benefit
Synonyms: Exact: N/A • Broad: N/A
Definition: Relationship between the expected outcomes and the costs of surveillance to achieve these outcomes. Surveillance system costs include direct costs (personnel and material resources), indirect costs (resulting from preparedness and response to surveillance findings), and prevention benefits or costs from a societal perspective (e.g., effects of the information generated on decision making and population health). Refers to the relative expenditures (costs) and outcomes (effects) of different surveillance strategies, determining whether the improved health outcomes or surveillance capabilities justify the associated costs to help inform efficient resource allocation decisions (kirsh 2008 adpated for surveillance).
Description: The costs of obtaining surveillance information needs to be balanced against the benefits derived. Assessment of surveillance resources typically focuses on direct costs. Because of the complexity of surveillance and response processes, it is usually difficult to define indirect costs. For some infectious disease surveillance systems, investigators have modeled the expected future costs of strategies for continued vaccination, surveillance, and other public health activities. Efforts to improve sensitivity, positive predictive value, representativeness, timeliness, and stability can increase the cost of a surveillance system.
Background: TBD
Measurement: TBD
Version 0.2
Cost minimization
ID: costMinimization
See also: costs; cost-benefit; cost-effectiveness; cost-utility; benefit
Synonyms: Exact: N/A • Broad: N/A
Definition: The comparison of costs between alternatives with identical outcomes, used to determine the most economical option when no additional benefits are present (often applied to demonstrate the dominance of one strategy over another).
Description: Cost-minimization compares the expenses of alternative surveillance strategies that achieve identical outcomes to identify the most economical option. It helps prioritize resource allocation by demonstrating which approach delivers the same public health impact at the lowest cost. This term is closely linked to cost-effectiveness and efficiency, but unlike those, it assumes equivalent benefits across options. Accurate cost data—including fixed and variable costs—are essential. Considering operational, personnel, and infrastructure expenses ensures a comprehensive evaluation, supporting sustainable and affordable surveillance system design without compromising performance.
Background: TBD
Measurement: TBD
Version 0.2
Cost-utility
ID: costUtility
See also: cost minimization; cost-benefit; cost-effectiveness; costs; benefit
Synonyms: Exact: N/A • Broad: N/A
Definition: An evaluation that compares the system’s interventions or strategies by measuring outcomes in terms of utility gained, using a ratio scale to assess the value of different health states or surveillance outcomes, allowing for comparison across various surveillance approaches and their impact on public health.
Description: Cost-utility analysis compares surveillance strategies by measuring outcomes in terms of utility, often using quality-adjusted life years (QALYs) or disability-adjusted life years (DALYs). This allows valuation of health benefits considering both quantity and quality of life gained from interventions. It facilitates comparison across diverse surveillance approaches with different health impacts, integrating effectiveness and economic costs into a single metric. Accurate utility measurement requires reliable health state valuation and comprehensive cost data. This term complements cost-benefit and cost-effectiveness analyses by focusing on health-related quality of life, supporting informed resource allocation in public health surveillance.
Background: TBD
Measurement: TBD
Version 0.2
Costs
ID: costs
See also: cost minimization; cost-benefit; cost-effectiveness; cost-utility; benefit
Synonyms: Exact: N/A • Broad: N/A
Definition: All expenses associated with the surveillance system, including direct costs (e.g., equipment, personnel) and indirect costs (e.g., training, data management infrastructure).
Description: Cost is a key consideration in evaluating surveillance systems, reflecting their value and efficiency. Costs are both direct (e.g., personnel, data collection, laboratory services, infrastructure) and indirect (e.g., follow-up efforts for false alarms, impact on agency credibility). New or improved systems incur start-up (e.g., software, equipment) and ongoing operational costs (e.g., staff, maintenance). Economic analyses should balance these costs with the public health benefits, such as early outbreak detection and disease prevention, to assess overall value. Direct and indirect costs are usually collected and reported separately.
Background: TBD
Measurement: To evaluate costs, all elements of the surveillance system must be considered, including planning, sample collection, analysis, reporting, and dissemination. It’s important to distinguish between fixed costs (e.g., permanent staff salaries) and variable costs (e.g., sample collection or reagents). Cost-sharing among stakeholders—such as producers, consumers, and public funds—should also be considered, as these costs can vary across the system.
Version 0.2
Coverage
ID: coverage
See also: representativeness
Synonyms: Exact: N/A • Broad: representativeness?
Definition: Proportion of the population of interest (target population) or proportion of areas of interest (e.g. specific habitats or high- risk sites) that is included in the surveillance activity.
Description: Coverage indicates the extent to which the target population or geographic areas are included in surveillance. High coverage enhances representativeness, sensitivity, and early detection—especially for emerging threats. It is related to sampling strategy, reporting completeness, and access to high-risk populations. Mathematically, coverage is the proportion the same represents divided by the total target population size.
Background: Coverage was identified as a reported concept six times in our scoping review. The concept was first introduced by Hoinville et al., in 2013 and has since been consistently included in animal health surveillance system evaluation frameworks. The definition has remained highly consistent within animal health surveillance, aligning with the concept of "population coverage" in public health surveillance.
Measurement: Coverage can be assessed by the proportion of respondents (survey-based) or cases (hospital- or facility-based) included in the surveillance system. Key measurements include population undercoverage, which occurs when parts of the target population are missed, and population overcoverage, where elements outside the target population are included. A demographic analysis can provide benchmarks for assessing completeness of coverage in the existing surveillance data and document changes in coverage from previous periods. Additionally, systems like farmer-based clinical surveillance, syndromic surveillance, or periodic sampling surveys benefit from regular evaluations to identify gaps or misrepresentation in coverage. By reviewing the geographic coverage and the number of areas of interest, it is easier to determine if the surveillance system adequately represents the target population. Furthermore, temporal coverage can be assess by determining the conditional probability that any given unit in the population will be examined or tested within the specified time frame. For example, if the target time frame is 7 days, but testing occurs every 4 weeks, the temporal coverage would be 25%. An assessment of coverage could include: * Characterisation and qualitative comparison of the sampled and target populations. Alternatively, comparison of sampled areas or habitats versus areas or habitats of interest. * Where sufficient data on the target population/areas or habitats of interest exists, simple calculations of the proportional coverage can be made (e.g. 75% of the national herd and 45% of cattle holdings are sampled annually or 30% of the marine ports). * Where sufficient information on the background population, or high-risk areas or habitats respectively, is lacking, more sophisticated sampling designs might be employed (e.g. capture-recapture analysis or drop camera surveillance). * Considering whether the target population, or area of interest, has been adequately defined (i.e. whether the exclusion of certain animals or holdings or sites is merited). * The unit of interest - the level at which coverage is measured - is often the unit of interest of surveillance (e.g. animal, holding, high-risk site or specific marine habitat). If insufficient data exist, alternative perspectives might be desired. Coverage might then be assessed at other aggregate levels (e.g. geographical areas) or relevant intermediate steps in the surveillance pathway (e.g. the proportion of veterinary practices submitting diagnostic samples). * In certain contexts it may be worth establishing a timeframe of reference (e.g. annual coverage). The choice of timeframe should reflect the epidemiology of the disease or life history of a risk organism.
Version 0.1
Credibility (collaboration)
ID: credibilityCollaboration
See also: N/A
Synonyms: Exact: N/A • Broad: N/A
Definition: N/A
Description: TBD
Background: TBD
Measurement: TBD
Version 0.1
Credibility (data)
ID: credibilityData
See also: N/A
Synonyms: Exact: N/A • Broad: N/A
Definition: N/A
Description: TBD
Background: TBD
Measurement: TBD
Version 0.1
Data accessibility
ID: dataAccessibility
See also: N/A
Synonyms: Exact: Accessibility (data) • Broad: –
Definition: –
Description: N/A
Background: –
Measurement: –
Data analysis
ID: dataAnalysis
See also: data collection; data management and storage
Synonyms: Exact: N/A • Broad: N/A
Definition: The use of appropriate methods for the analysis and interpretation of data.
Description: <+ Data analysis applies scientific methods to process surveillance data to generate insights such as detecting unusual patterns (i.e. outbreaks, signal levels beyond expected). Methods range from basic descriptive statistics and trend monitoring to more sophisticated approaches like time series, spatial analysis, and automated aberration detection algorithms. +> assesses how effectively surveillance data are processed and interpreted to generate insights. High-performing systems apply appropriate, context-specific analytical methods and ensure findings are actionable.
Background: Data analysis was cited as a concept in surveillance system evaluation nine times in the literature, spanning public health, animal health, and One Health surveillance system evaluation frameworks. The definition remains largely consistent, <+ relfecting uniformity across disciplines, with refinment over time, to emphasize the use of contextual information (e.g., prior likelihood of disease) in risk-based analysis and the need for analysis/interpretation to be performed timely to support decision-making and action. +>
Measurement: The evaluation of the data analysis attribute/term focuses on assessing the methods used to analyze surveillance data. It includes identifying the techniques applied, such as basic statistics, trend analysis, or more advanced methods like time series and spatial analyses. The evaluation checks if the limitations of the data have been properly considered and addressed in the analysis. Additionally, it evaluates whether the data is being fully utilized or if there are opportunities for further analysis. Reviewing past user needs and whether the analysis methods have met these demands is also important to ensure the analysis is delivering valuable, actionable insights. An evaluation of data analysis should include: The identification of the analysis methods applied to surveillance data: * No analysis * Basic descriptive statistics * Examination of trends * More sophisticated statistical approaches (e.g. time series analyses, spatial analyses); An assessment of whether the limitations of data have been understood and accounted for in statistical analyses?; An indication as to whether the body of data available is being fully exploited or could further use of data be made? It may help to review requirements for information made by users of the surveillance data in the past, to determine whether their needs were met by the methods applied.
Version 0.1
Data collection
ID: dataCollection
See also: data analysis; data management and storage
Synonyms: Exact: N/A • Broad: N/A
Definition: The use of appropriate data sources and collection methods, including a clear a case definition and a data collection protocol.
Description: Effective data collection relies on clearly defined and complete case definitions or risk organism descriptions, including case signalment, clinical and pathological signs, and epidemiological information, and, where relevant, laboratory or taxonomic identification. Diagnostic methods must align with the case definition and be assessed for sensitivity and specificity. A written protocol should guide data and sample collection, with mechanisms to ensure adherence. Data collection methods should be clearly documented, and collection methods should minimize redundancy, address data gaps, and employ appropriate sampling strategies (e.g., risk-based or pooled).
Background: Data collection is a fundamental surveillance activity consistently recognized in evaluation frameworks across public, animal, and One Health systems (five instances within the literature review). Data collection was explicitly mentioned as a concept in the evaluation of surveillance systems in five instances within the literature, including a clear definition and integration into the animal health surveillance field. It is commonly cited as the most costly and difficult component of a surveillance system to implement (Chaudron, 2010; Peyre et al. 2022).(1)
Measurement: Questions to consider when assessing data and information collection include: * Is there (if applicable) a written case definition/organism description for this surveillance system that is clearly defined and complete, with specified inclusion and exclusion criteria? If so Does the case definition/organism description include relevant details of the case signalment, clinical and pathological signs and epidemiological information as appropriate? Does the case definition include laboratory diagnosis? Alternatively does the organism description include taxonomic ID? Are the chosen diagnostic/taxonomic methods appropriate to the case definition/organism description, including in terms of samples being collected? Are syndromes used and - if yes - defined in an appropriate way? In those sectors where symptomatic surveillance is * Is there (if applicable) a written case definition/organism description for this surveillance system that is clearly defined and complete, with specified inclusion and exclusion criteria? If so undertaken, are symptoms clearly defined for when samples need to be taken? * Is there a written sample and/or data collection protocol and are there appropriate assurance mechanisms to ensure the protocols are followed? * Have the sensitivity and specificity of the tests used been assessed (where relevant)? * Are there data collected that are not used in analysis, interpretation or surveillance management (redundancy)? * Are there information needs for which data are not currently collected and feasibly could be? * Are appropriate sampling strategies used, including the use of risk-based approaches and pooled sampling? This could include risk-based requirement calculations or risk- based sampling. The basis of the risks used in the design of the risk-based sampling strategy should be reviewed. Data collection methods should be clearly documented. It may help to review demands for information made by users of the surveillance data in the past, to determine whether their needs were met by the data available.
Version 0.1
Data completeness
ID: dataCompleteness
Synonyms: Exact: data completeness and correctness (completeness) • Broad: N/A
Definition: The proportion of data that were intended to be collected that actually was collected.
Description: Completeness refers to the extent to which all required data are captured, reported, and available for analysis. A system with high completeness allows for more reliable decision-making and enhances the system’s usefulness. Incomplete data may result from issues with data collection, management, or communication, undermining the system’s effectiveness. Regular submission from all contributors and strong data management practices help improve completeness. Ensuring completeness strengthens data quality.
Background: Completeness was identified as an independent concept on 12 occassions, distinct from the definitions of "data quality" or "data completeness and correctness". The concept was first introduced by Thacker et al., in 1988 (A method for evaluating systems of epidemiological surveillance), however it was not indentified as an evaluation concept until 1993 by Cutts et al. The definition of completeness has remained largely unchanged over the years, maintaining consistency across disciplines. It is primarily regarded as a key aspect of data quality and continues to be one of the most frequently used concepts for evaluating surveillance systems.
Measurement: An evaluation of completeness should involve assessing the proportion of cases that were identified and subsequently reported to the system, which can be done through register/database reviews. Additionally, examining the percentage of “unknown” or “blank” responses to items on surveillance forms offers a straightforward and effective measure. The evaluation should also address challenges faced in both manual and automated data management. This includes identifying issues such as coding errors or data loss in manual systems, and programming errors or inappropriate data filtering in automated systems. These evaluations help identify gaps in data reporting and ensure that the surveillance system captures all relevant cases accurately.
Version 0.2
Data correctness
ID: dataCorrectness
See also: N/A
Synonyms: Exact: accuracy (correctness), completeness, data quality • Broad: N/A
Definition: The proportion of data entries that correctly reflect the true value of the data collected.
Description: N/A - data completeness and correctness is an exact synonym for data quality.
Background: TBD
Measurement: TBD
Version 0.1
Data management
ID: dataManagement
See also: N/A
Synonyms: Exact: – • Broad: –
Definition: –
Description: –
Background: –
Measurement: –
Data management and storage
ID: dataManagementAndStorage
See also: historical data
Synonyms: Exact: N/A • Broad: N/A
Definition: Appropriate use and documentation of data management systems for processing information, including data processing protocols, and effective use of data verification procedures and of data storage and back-up procedures.
Description:Appropriate use and documentation of data management systems for processing information, including data processing protocols and effective use of data verification procedures, data storage and back-up protocols. Measures taken to assure authorised computer system access and to maintain confidentiality where needed. Is there a dedicated custodian? Data management is a broad area concerning the collation, storage and maintenance of data, including but not limited to matters of data quality, accessibility, usefulness and security. Assessing this attribute will require an intimate understanding of the data storage and management systems employed by the surveillance activity.
Data management is a broad area concerning the collation, storage and maintenance of data, including but not limited to matters of data quality, accessibility, usefulness and security. Assessing this attribute will require an intimate understanding of the data systems employed by the surveillance activity.
Data management and storage involve the systematic handling, processing, and safeguarding of data throughout its lifecycle. This includes ensuring data quality, accessibility, and security. Effective data management requires robust protocols for processing, verification, storage, and backup, as well as measures to control access and maintain confidentiality where necessary. An essential consideration is the appointment of a dedicated custodian responsible for overseeing these activities.
Background: TBD
Measurement: Evaluating data management and storage involves reviewing the database structure to verify that data fields are well-defined, consistent, and correctly normalized, ensuring efficient storage and retrieval. Evaluators should check that primary keys uniquely identify records and that validation constraints and cross-consistency checks are in place to prevent invalid data input. The documentation supporting the database, such as data dictionaries and entity relationship diagrams, should be reviewed to confirm that it facilitates clear interpretation and understanding of the data. Additionally, the evaluation should assess the existence and sufficiency of data management protocols, including data quality standards (e.g., ISO9000, Good Clinical Practice). This involves reviewing records management policies, retention protocols, and periodic quality control checks. Data security and access control protocols must also be evaluated, ensuring only authorized personnel can access sensitive data.Furthermore, back-up and storage procedures should be tested to confirm data is securely stored and can be recovered if needed. Interviews with data custodians or managers may provide insights into the system’s functionality and reliability. Overall, the evaluation will require a comprehensive review of the system’s design, documentation, and operational practices to ensure proper data handling and storage. An assessment of this attribute should include: 1.Consideration of whether the database structure has been correctly designed: Has each field of data been tightly defined to ensure correctness, conciseness and consistency across records? Have primary keys, uniquely identifying each record, been assigned? Has the database been normalised, to ensure data is stored in the most parsimonious, transparent and useable way? Have validation constraints, preventing the input of invalid data, and internal cross-consistency checks been applied? Is the data stored in a way that allows the required interrogation and analysis?; 2.Consideration of whether documentation of the data is sufficient to facilitate interpretation and understanding of the data: Is there a document providing a summary overview of the data and collection methods and explaining any idiosyncrasies relevant to the analysis and interpretation of the data? Is there a data dictionary that clearly defines each field? Is there an entity relationship diagram that explains how the data relate?; 3.Consideration of whether there are adequate and documented protocols for managing data quality and security: Is the data management system covered by a data quality standard (e.g. ISO9000, Good Clinical Practice or Good Laboratory Practice)? Are periodic data quality control checks implemented? Are records management issues clearly defined, including policy on the retention of data? Are data back-up and storage protocols in place?
Version 0.1
Data quality
ID: dataQuality
See also: N/A
Synonyms: Exact: data completeness and correctness • Broad: N/A
Definition: The completeness and validity of the data recorded in the public health surveillance system.
Description: <+ Data quality measures the inherent correctness and sufficiency of the data gathered by the surveillance system. +> Data quality reflects the completeness and validity of recorded data and is essential for accurate surveillance outputs. <+ Poor data quality can undermine reliability, introduce bias that limits representativeness, and make the system less acceptable to participants. Conversely, high-quality data are essential for accurate analysis and improving the system’s overall usefulness for decision-making and public health action. +> High-quality data improve system acceptability and representativeness, while Poor-quality data—often due to manual entry errors in paper-based systems—can undermine reliability and may indicate problems in data collection or data management and storage. Electronic systems can enhance data quality by improving accuracy, timeliness, and accessibility. Data quality is linked to terms such as timeliness, acceptability, representativeness, and usefulness, and directly affects the reliability and usability of system outputs for decision-making and public health response.
Data quality has historically been defined by the core attributes of completeness and validity. It is routinely recognized as essential across public health, animal health, and emerging One Health surveillance frameworks. In the context of animal health, the term "data completeness and correctness" is used as an exact synonym for data quality.
In more recent years, data quality includes the concept of fit-for-purpose, with sufficeint quality to meet the need of the data’s intended purpose. As surveillance systems evolve, particularly through the use of informatics and electronic systems, data quality is enhanced and expanded. Electronic systems can improve data quality by facilitating accuracy, timeliness, accessibility, and reducing the potential for manual entry errors. As data systems evolve, data quality has been enhanced and expanded with the use of robust data management systems and automated data flow processes. The concept is highly interrelated with other performance metrics; for instance, poor quality data reduces system acceptability and representativeness.
Measurement: Can be measured as the proportion of data intended to be collected that was actually collected (completeness) and the proportion of data entries that correctly reflect the true value of the data collected (validity/accuracy). Includes proportion of unknown, invalid, and missing values for reported data elements. Validity may be estimated by the proportion of errors in surveillance system data compared to analogous data from one of the system’s data sources. Measures for determining data quality include percentages of "unknown," invalid, and missing responses to items on data collection forms. In addition, data quality can be measured by applying edits for consistency in the data. However, a full assessment may require a special study. The acceptability and representativeness of a public health surveillance system are related to data quality. With data of high quality, the system can be accepted by those who participate in it. In addition, the system can accurately represent the health-related event under surveillance. Paper-based systems, such as the E-Book registry, are more prone to human error, and can result in illegible, missing, and incomplete entries. An electronic system could increase data quality. Moreover, conversion to an electronic system would improve the timeliness, accessibility, and usability of captured data, as well as allow greater flexibility in responding to health events in a community. Many data-quality definitions depend on other system performance attributes (eg, timeliness, usefulness, acceptability) (21). Because of reliance on multiple data sources, data quality must be assessed in different ways. For surveillance relying on surveys, concepts of reliability, validity, and comparison with alternative data sources are important. For example, considerations of possible data-quality concerns arise with use of mortality data, particularly underreporting of suicide. It should be assessed using measures of completeness, consistency, and comparison with alternative sources to ensure reliability across diverse data types and collection methods.
Version 0.2
Detection mode
ID: detectionMode
See also: Sensitivity; Specificity; Predictive Value Positive; False-alarm rate; Timeliness; Data analysis
Synonyms: Exact: N/A • Broad: event detection, early detection/warning, surveillance sensitivity (outbreak detection), signal detection, case detection capability
Definition: The operational capability of the surveillance system to identify or forecast an event (e.g., event detection, establish baseline, or event detection expected with no baseline).(Corley_AEC_2012?)
Description: Detection mode characterizes how a surveillance system operationally identifies health events and determines whether detected events represent deviations from an established baseline in terms of frequency and/or magnitude (Corley_AEC_2012?). Systems may be configured to detect events against an established baseline, to first establish baseline patterns before detection can occur, or to detect events when no baseline is available (such as for exotic or novel diseases). Any excess or anomalous occurrence identified through the system’s detection mode should be statistically characterized and include a measure of abnormality or confidence (Corley_AEC_2012?). It should be recognized that baselines fluctuate over time, requiring appropriate statistical methods for pattern recognition and aberration detection (Buehler_ESSEDa_2004?,Groseclose_ESS_2013?). The chosen detection mode directly influences how system attributes such as sensitivity (probability of detection) and timeliness (speed of detection) are defined and measured, and ultimately determines the system’s utility for public health intervention (2,Morbey_OJP_2021?,Corley_AEC_2012?).
Background: Detection mode provides a foundational characterization of the operational capability of a surveillance system, specifically concerning its approach to early warning and event detection (Corley_AEC_2012?). The attribute was identified and defined within the context of evaluating event-based biosurveillance systems, models, and constructs, which frequently rely on analyzing qualitative or quantitative variables to infer population health status (Corley_AEC_2012?). Within biosurveillance evaluation frameworks, detection mode falls under the broader Event Attribute Family and describes the type of operational outcome anticipated from the system (Corley_AEC_2012?). While detection mode represents a distinct operational design characteristic, it overlaps considerably with classic case detection and ascertainment metrics. The performance of a surveillance system’s detection mode is fundamentally measured through traditional epidemiological attributes including sensitivity (the probability and proportion of true events detected), specificity (the ability to correctly identify non-events), predictive value positive (the proportion of detected events that are true events), and timeliness (the speed of detection) (2,Morbey_OJP_2021?,Corley_AEC_2012?,Buehler_ESSEDa_2004?). This attribute is closely connected to syndromic surveillance methods, which heavily rely on statistical tools for pattern recognition and aberration detection applied to data streams to screen for patterns requiring investigation (Buehler_ESSEDa_2004?,Groseclose_ESS_2013?). The chosen detection mode directly determines how these classic ascertainment metrics are operationalized, defined, and measured within the surveillance system context, while balancing detection capability against false-alarm rates to ensure efficient resource utilization (3,Corley_AEC_2012?).
Measurement: TBD
Version 0.2
Effectiveness
ID: effectiveness
See also: usefulness
Synonyms: Exact: efficacy • Broad: Impact, usefulness
Definition: The extent to which a surveillance system is able to achieve its intended objectives. (4,Drewe_EoAaPHS_2012?,WorldHealthOrganization_OtWF_2004?,Peyre_TRVA_2019?).
Description: Effectiveness reflects how well the surveillance system achieves its intended objectives and meets the needs of stakeholders. It represents the capability of producing a desired result and encompasses the system’s technical performance in reaching its objectives, such as disease control or early detection (4,Drewe_EoAaPHS_2012?). The measurement of effectiveness considers multiple underlying attributes including timeliness, sensitivity, specificity, predictive value, and robustness, with the specific attributes of greatest relevance determined by the surveillance system’s objectives (Grosbois_ARtUMoE_2015?,Yang_UOaHS_2022?). Sensitivity (probability of detection) and timeliness are frequently assessed attributes related to effectiveness, particularly for systems focused on early detection. Systems should use collaborative approaches to improve effectiveness across data collection, processing, and dissemination, as integrated surveillance can enhance effectiveness compared to systems operating in isolation (Yang_UOaHS_2022?). Effectiveness is evaluated through indicators that measure processes, outputs, outcomes, and impacts against stated objectives. These indicators may include task completion rates, timeliness metrics, and measurable public health impact, such as whether the system leads to prevention, control, or better understanding of adverse health events (Muellner_SiFAiB_2018?,Peyre_TRVA_2019?). This term is closely related to system usefulness (the tangible benefits and outcomes attributed to the system) and efficiency (the relationship between resources implemented and results obtained) (4,Peyre_TRVA_2019?).
Background: Effectiveness emerged as a core evaluation criterion through the adaptation of general program evaluation principles to public health surveillance. The concept gained prominence in surveillance evaluation frameworks during the late 20th and early 21st centuries, particularly through influential guidance documents from the Centers for Disease Control and Prevention (CDC) and the World Health Organization (WHO), which established effectiveness as a central measure for assessing surveillance system quality (4,WorldHealthOrganization_OtWF_2004?). These frameworks positioned effectiveness as one of three core criteria for assessing public policies and programs, alongside relevance and efficiency, reflecting broader trends in public sector performance evaluation (4). The operationalization of effectiveness in surveillance contexts evolved from general notions of "achieving objectives" to more specific technical performance measures. In surveillance evaluation, effectiveness became understood as the technical performance of the system—the capacity to reach its objective (such as disease control or early detection) through appropriate use of inputs to produce desired outputs (4,Drewe_EoAaPHS_2012?). This conceptualization linked effectiveness to measurable attributes such as sensitivity, timeliness, and specificity, allowing evaluators to quantify technical performance. The development of economic evaluation frameworks, particularly cost-effectiveness analysis (CEA), further formalized effectiveness as a measurable construct, using performance indicators as proxies for benefit when comparing surveillance investments (4,Grosbois_ARtUMoE_2015?). From a policy perspective, this evolution enabled effectiveness to measure the effects (outcomes and impacts) directly attributable to surveillance system implementation, establishing the principle that surveillance systems must be regularly evaluated to ensure they remain both effective and efficient in meeting evolving public health needs.
Measurement: TBD
Version 0.2
Efficiency
ID: efficiency
See also: cost; benefit; value; resource allocation; acceptability (economic)
Synonyms: Exact: economic efficiency, resource efficiency • Broad: cost-effectiveness
Definition: The relationship between the resources consumed by a surveillance system and the results it produces (3).
Description: An efficient surveillance system accomplishes its objectives with minimum expenditure of time, human effort, and cost relative to outputs and benefits generated (Drewe_EaI_2012?). Surveillance incurs both direct costs (personnel, laboratory consumables, travel, equipment) and indirect costs (administrative overhead, opportunity costs), which must be evaluated against outputs such as disease detections, notifications, and actionable intelligence (CDC_UpdGuideEval_2001?). Economic efficiency represents the optimal balance between resources invested and information quality produced, considering performance attributes such as sensitivity, precision, and timeliness (5,6). Economic efficiency can be conceptualized at three hierarchical levels: optimization (maximizing net societal benefit), acceptability (ensuring benefits exceed costs), and cost-minimization (achieving technical targets at lowest cost) (3). Efficiency evaluation requires comparing system performance against alternatives or a counterfactual scenario to determine added value (6). Since surveillance is an intermediate product that informs intervention decisions, optimal evaluation considers the combined costs and benefits of both surveillance and intervention activities (5).
Background: Economic efficiency emerged as a formal evaluation attribute following recognition that public health resources are constrained and require optimal allocation (5,Drewe_EaI_2012?). Early guidelines identified cost as a consideration but did not mandate formal economic analysis (7). The 2001 CDC updated guidelines recommended judging costs relative to benefits, though comprehensive analyses remained uncommon (CDC_UpdGuideEval_2001?). Formalization as a distinct attribute occurred primarily through animal health frameworks between 2004-2015. The OASIS tool defined efficiency as the link between resources and results, establishing it as a value attribute (8). The SERVAL framework refined this by articulating three levels of economic efficiency and integrating economic evaluation as core to surveillance assessment (5). The RISKSUR project and EVA tool synthesized economic theory and practical methods, establishing that surveillance value derives from enabling intervention decisions and requiring evaluation of the surveillance-intervention-mitigation pathway (1,6). Contemporary approaches emphasize integrated evaluation considering monetary and non-monetary benefits, value of information, and the three-variable relationship between surveillance inputs, intervention actions, and loss avoidance (1,6).
Measurement: TBD
Version 0.1
Engagement
ID: engagement
See also: acceptability; collaboration
Synonyms: Exact: participation • Broad: –
Definition: The active involvement of stakeholders, including partners and communities, in the design, operation, interpretation, and use of the surveillance system.
Description: Engagement reflects the depth and quality of participation in surveillance activities. It encompasses contributions to data collection, system design, analysis interpretation, and application of surveillance outputs. Effective engagement is characterized by meaningful interaction, shared responsibility, and mutual benefit among all partners.
Background: The concept of engagement has evolved in surveillance practice, particularly in emphasizing community involvement. Engaged communities participate in various aspects of surveillance, from identifying priorities to collecting data and using results. Evidence of engagement includes consistent participation, valuable feedback, system adaptation based on community input, and local ownership of activities. Modern approaches recognize that engagement should include diverse communities, especially those most affected by health conditions under surveillance. This shift values equitable partnerships where communities are active collaborators rather than simply sources of data.
Measurement: TBD
Version 0.2
Equity data
ID: equityData
See also: representativeness; acceptability; data quality; usefulness; coverage
Synonyms: Exact: equity-relevant data, health equity data, disparities data • Broad: social determinants data, demographic data, stratification variables, population characteristic data, vulnerability indicators
Definition: The surveillance system’s data content necessary to capture population characteristics and contextual factors that enable analysis of differential health impacts across demographic and socioeconomic groups.
Description: Equity-relevant data ensures the system collects information necessary to analyze how disease burden, exposure, or surveillance outcomes vary across populations. For environmental surveillance systems, this requires data that can be spatially or demographically linked to affected communities, stratified by indicators of vulnerability such as socioeconomic status, age distribution, or geographic marginalization. For human health surveillance, this extends to demographic attributes (race/ethnicity, gender, age), and in specialized frameworks, may include measures of structural determinants such as racism or stigma. For animal health surveillance, relevant factors include producer characteristics, economic constraints, and cultural factors (e.g., livestock’s role in local economies) that influence disease reporting and system participation. The attribute assesses whether the system’s data infrastructure enables identification of which populations are disproportionately affected and supports development of targeted, equitable interventions. This directly influences representativeness (ensuring no groups are systematically excluded) and acceptability (ensuring culturally appropriate engagement).
Background: The concept of equity-relevant data in surveillance evaluation has three distinct but converging lineages. Traditional Public Health Surveillance (1990s-2000s): Foundational CDC guidelines from the 1990s required surveillance evaluation to consider disparities or inequities and assess data distribution by population demographics (race, ethnicity, age, socioeconomic status). This was traditionally addressed within the attribute of representativeness, which required collecting demographic characteristics and identifying systematically excluded population subgroups. One Health and Animal Health Frameworks (2013-present): Animal health and One Health evaluation frameworks identified that traditional approaches, focused on technical attributes like sensitivity, failed to capture the social and economic aspects influencing system performance. These frameworks required characterizing socioeconomic context, including local value systems, cultural importance of livestock, and economic constraints—factors that fundamentally shape surveillance effectiveness through their influence on acceptability and data quality. Health Equity and Anti-Racism Frameworks (2021-present): Public Health Critical Race Praxis (PHCRP) evaluation frameworks identified a critical gap: even systems collecting race/ethnicity data often fail to capture structural drivers of health inequities. PHCRP explicitly requires assessing whether systems capture measures of racism (such as residential segregation or discriminatory policies) and validated stigma indicators, moving beyond demographic categories to measure mechanisms producing inequitable outcomes. Convergence: These lineages converge in recognizing that adequate evaluation requires assessing whether systems collect data sufficient to identify who is disproportionately affected (demographics), why disparities exist (structural and contextual factors), and whether the system engages populations equitably (cultural appropriateness). Specific data elements vary by surveillance type, but the principle is consistent: equity-relevant data collection addresses not only demographic completeness but also the social, economic, and structural determinants shaping health outcomes and surveillance performance. This concept intersects with multiple traditional evaluation attributes including representativeness, acceptability, data quality, and usefulness.
Measurement: TBD
Version 0.2
False-alarm rate
ID: falseAlarmRate
See also: specificity
Synonyms: Exact: Inverse of specificity, False positive rate (FPR) • Broad: Predictive value positive (PVP) - related inverse concept; low PVP indicates high false-alarm rate
Definition: The proportion of negative events (e.g. non-outbreak periods) incorrectly classified as events (outbreaks) (3,6).
Description: The false-alarm rate is the proportion of non-events incorrectly flagged as events and represents the inverse of specificity (3,5,6). While some false positives are expected in surveillance systems, a high false-alarm rate can lead to wasted resources through misdirected investigations of non-events (anonymous.2001?,Buehler_EOPHS_2004?,Groseclose_EOSH_2008?). Conversely, efforts to minimize false alarms by raising detection thresholds may compromise sensitivity, increasing the risk of missed outbreaks (Buehler_EOPHS_2004?,Groseclose_SSEDSHD_2013?). This trade-off is particularly critical in early detection and syndromic surveillance systems, where high data volumes and low event prevalence often generate elevated false-positive rates (Groseclose_SSEDSHD_2013?). Excessive false alerts can lead to alert fatigue among users, potentially causing genuine events to be missed, and may erode credibility of public health agencies (Lucero-Obusan_BMC_2022?,Groseclose_SSEDSHD_2013?). System designers should specify desired sensitivity and false-alarm rate thresholds, explicitly balancing the risk of missed outbreaks against the cost of investigating false alarms (Buehler_EOPHS_2004?). The false-alarm rate is closely related to predictive value positive (PVP), as a low PVP directly reflects a high false-alarm rate; both metrics quantify the quality of positive signals in a surveillance system (Buehler_EOPHS_2004?,anonymous.2001?).
Background: The term "false-alarm rate" was proposed and adopted in animal health surveillance evaluation frameworks as an alternative to "specificity" because it was considered more easily understood by non-epidemiologists, including surveillance planners and policy makers (3,6). The concept gained prominence as a key metric within the Evidence Quality or Effectiveness domain of surveillance evaluation frameworks, particularly for systems designed to detect early outbreaks or unexpected events (6,icahs-workshop-report?). In biosurveillance and outbreak detection systems, quantifying the false-alarm rate has been identified as essential for economic evaluations, with stakeholders demonstrating willingness-to-pay for improvements in this attribute (delRioVilas_PfEoOHSTEB_2022?,Amato_EPI_2023?).
Measurement: TBD
Version 0.2
Feasibility
ID: feasibility
See also: sustainability; stability; efficiency; effectiveness
Synonyms: Exact: N/A • Broad: Practicality (ease of operation and administration), Resource adequacy, Implementability
Definition: The extent to which available means (resources, expertise, infrastructure) meet the surveillance system’s needs, ensuring that strategies and methods are suitable and applicable to the surveillance objectives (Yang_UOaHS_2022?,Drewe_EoAaPHS_2012?).
Description: Feasibility evaluates whether a surveillance system is realistic and sustainable given available resources, expertise, and objectives (Yang_UOaHS_2022?). It addresses the fundamental question of whether the means necessary to run the system are available and whether system strategies correspond to public health priorities (Drewe_EoAaPHS_2012?,Yang_UOaHS_2022?). Key components of feasibility assessment include staff capability and multidisciplinary expertise essential for system operation and maintenance, as well as whether personnel have the required knowledge, skills, and experience (Yang_UOaHS_2022?). Feasibility also considers the availability of necessary external resources, such as steering committees, technical committees, and government affiliations that support system implementation. The ability to track surveillance events over time using available data sources represents another critical feasibility consideration (Yang_UOaHS_2022?). Ultimately, feasibility assesses whether available resources are sufficient to support all system components—including data collection, analysis, and dissemination—effectively and efficiently. Limited resources pose significant constraints on achieving ideal surveillance criteria, making feasibility a primary consideration when setting evaluation objectives and determining system design (Yang_UOaHS_2022?).
Background:Governance represents a relatively recent addition to surveillance evaluation frameworks, emerging in response to the complexity of multisectoral and One Health surveillance systems (9,10). Unlike classical epidemiological attributes (sensitivity, timeliness) that emerged from foundational frameworks with consistent definitions, governance developed without a unified evolutionary path, appearing variably across evaluation tools with context-dependent conceptualizations. Early surveillance evaluation recognized managerial and administrative weaknesses as obstacles but did not formalize governance as a distinct assessment domain (11).
The explicit formalization of governance as an evaluation attribute occurred primarily through One Health and integrated surveillance frameworks developed between 2010-2020. The ECoSur tool specifically targeted governance of collaboration in multisectoral systems, while NEOH, ATLASS, and PMP-AMR incorporated governance-related dimensions with varying emphases (9,10,12–14). These tools reflect three distinct but overlapping conceptualizations: structural governance (steering committees, formal agreements), functional governance (decision-making processes, resource allocation), and social governance (stakeholder representation, power dynamics, real versus official practices) (1,1,12).
Governance continues to evolve toward greater recognition of the gap between formal structures and actual practice, with emerging emphasis on adaptive capacity, stakeholder engagement authenticity, and transparent accountability mechanisms suited to complex health security challenges (1,1). Future development likely will focus on standardizing assessment approaches while maintaining flexibility for diverse surveillance contexts.
Governance overlaps substantially with organizational structure, collaboration, and system management concepts that appear in earlier frameworks (1,8). Its distinctiveness lies in three key elements: (1) explicit focus on power dynamics and decision-making authority rather than mere structural arrangements; (2) emphasis on the policy-institutional-operational continuum that links strategic intent to implementation; and (3) recognition of formal versus actual governance practices, distinguishing official rules from how systems truly function (1,12).
Governance serves as an enabling attribute that directly influences collaboration by providing multisectoral structures, supports sustainability through strategic resource planning, and affects acceptability through stakeholder engagement (12,13,15). Poor governance cascades into failures across multiple attributes—inadequate stakeholder engagement leads to poor acceptability and implementation challenges (15). Unlike traditional "government" (hierarchical authority) or "management" (technical operations), governance implies flexible power-sharing among diverse stakeholders (1).
Measurement: TBD
Version 0.2
Flexibility
ID: flexibility
See also: portability; simplicity; sustainability; acceptability
Synonyms: Exact: adaptability • Broad: scalability
Definition: The ability of the system to adapt to changing information needs or operating conditions, including scaling up or down as necessary, with little additional time, personnel, or funds (Drewe_EoAaPHS_2012?,Groseclose_EPHS_2010?,Azofeifa_EBHS_2018?,Hoinville_TRVA_2013?).
Description: Flexibility is the system’s ability to adapt to new information needs, emerging threats, or changing resources with minimal disruption. Flexible systems must accommodate diverse changes including new health-related events or hazards, modifications to case definitions or technologies, and variations in funding or reporting sources (Groseclose_EPHS_2010?,Amato_E_2023?). Examples of such adaptations include responding to changes in data standards (e.g., ICD-9 to ICD-10), incorporating new data sources and technologies (e.g., online data sharing or machine learning techniques), or adjusting case definitions to reflect evolving understanding of health conditions (Groseclose_EPHS_2010?). Standardized formats and modular designs support flexibility by providing robust, adaptable infrastructure. Systems utilizing standard vocabularies or standardized formats (such as PHIN standards) provide flexible code structures that can accommodate changing information needs. Centralized data processing can enhance flexibility by reducing the need for widespread system and operator behavior changes when adaptations are required. Flexibility is generally higher in simpler systems, as fewer components require modification when adapting (Thacker_MEES_1988?). Flexibility is closely related to sustainability, resilience, and system responsiveness. Flexible systems are vital for long-term maintenance, especially when facing external factors such as funding variations or personnel cuts, with infrastructure that can evolve as the system grows. Flexibility also supports acceptability and usefulness, as systems that can be tailored to meet the evolving needs and interests of staff and stakeholders foster greater participation and sustained engagement.
Background: Flexibility emerged as one of the seven original surveillance system attributes identified by Thacker et al. in 1988, establishing it as a foundational evaluation criterion (Thacker_MEES_1988?,Groseclose_EPHS_2010?). The attribute has been consistently included in surveillance evaluation frameworks across public health, animal health, and One Health systems, reflecting its status as a core performance measure. It was retained as one of nine core metrics in the modernized US CDC Evaluation Guidelines and continues to appear in contemporary evaluation tools (Groseclose_EPHS_2010?,Peyre_TRVA_2019?). The definition of flexibility has remained highly consistent since its introduction, emphasizing low-cost adaptation to changing conditions with minimal additional resources (Drewe_EoAaPHS_2012?,Groseclose_EPHS_2010?,Hoinville_TRVA_2013?). The term is often used interchangeably with "adaptability" or referenced as "adaptability to changes," and some evaluation tools formally list the attribute as "Flexibility, adaptability" (Peyre_TRVA_2019?). Recent applications have expanded the concept to explicitly include the system’s ability to scale up, scale down, or expand as necessary, particularly relevant for emerging surveillance contexts such as wastewater surveillance (Amato_E_2023?). Flexibility’s relationship to other evaluation attributes reveals its interconnected role in system performance. The attribute demonstrates a bidirectional relationship with simplicity—simpler systems exhibit greater flexibility because fewer components require modification during adaptation (Thacker_MEES_1988?). Flexibility also supports sustainability by enabling systems to persist despite changing external factors such as funding variations or staffing changes, while simultaneously contributing to acceptability and usefulness by allowing systems to be tailored to stakeholder needs. The relative importance of flexibility may vary across system development stages, with some evidence suggesting it holds particular relevance during initial development and pilot testing, while attributes like reliability and stability assume greater significance during operational maturity (Auer_R_2011?).
Measurement: Flexibility is probably best evaluated retrospectively by observing how a system has responded to a new demand. Unless efforts have been made to adapt the public health surveillance system to another disease (or other health-related event), a revised case definition, additional data sources, new information technology, or changes in funding, assessing the flexibility of that system might be difficult. In the absence of practical experience, the design and workings of a system can be examined. Simpler systems might be more flexible (i.e., fewer components will need to be modified when adapting the system for a change in information needs or operating conditions).
Version 0.2
Governance
ID: governance
See also: collaboration; acceptability; sustainability; organizational structure; engagement; resource allocation; accountability; transparency; multisectoral coordination; intersectoral collaboration
Synonyms: Exact: system governance, surveillance governance • Broad: N/A
Definition: The coordination, leadership, and operational effectiveness of surveillance systems, encompassing the legislative framework, steering mechanisms, and organizational structures that ensure clear roles, shared responsibilities, and inclusive decision-making to achieve surveillance objectives (1,9).
Description:Governance refers to the coordination, leadership, and operational effectiveness of a surveillance system, encompassing how it is organized, steered, and managed (9,16). Strong governance ensures clear roles and responsibilities, shared accountability, inclusive decision-making, and formalized collaboration across sectors and disciplines (12,Ruegg_COH_20222?).
Effective governance operates at three levels: the policy level defines collaborative strategy and goals; the institutional level establishes governance modalities (steering mechanisms, coordination, technical support); and the operational level implements routine collaborative activities (12). These are documented through regulations, charters, or formal agreements with clearly defined roles in structures such as steering committees (8,12). Strong governance requires defining strategic goals, organizing resource allocation, ensuring system adaptivity through evaluation, and managing compliance with legal, ethical, and confidentiality requirements (1,17).
Governance mechanisms define the structure and quality of collaboration in multisectoral systems, making them particularly critical for One Health approaches where strategies must be organized at policy and institutional levels (12,13). Unlike hierarchical "government" authority, governance promotes flexible power-sharing among stakeholders (1). Understanding governance in practice—not just official rules—enables effective system design (1,1). Poor governance, such as inadequate stakeholder engagement, cascades into poor acceptability and implementation failures (15), while good governance enables collaboration, supports sustainability through resource planning, and enhances effectiveness through accountability (12,13).
Background:Governance represents a relatively recent addition to surveillance evaluation frameworks, emerging in response to the complexity of multisectoral and One Health surveillance systems (9,10). Unlike classical epidemiological attributes (sensitivity, timeliness) that emerged from foundational frameworks with consistent definitions, governance developed without a unified evolutionary path, appearing variably across evaluation tools with context-dependent conceptualizations. Early surveillance evaluation recognized managerial and administrative weaknesses as obstacles but did not formalize governance as a distinct assessment domain (11).
The explicit formalization of governance as an evaluation attribute occurred primarily through One Health and integrated surveillance frameworks developed between 2010-2020. The ECoSur tool specifically targeted governance of collaboration in multisectoral systems, while NEOH, ATLASS, and PMP-AMR incorporated governance-related dimensions with varying emphases (9,10,12–14). These tools reflect three distinct but overlapping conceptualizations: structural governance (steering committees, formal agreements), functional governance (decision-making processes, resource allocation), and social governance (stakeholder representation, power dynamics, real versus official practices) (1,1,12).
Governance continues to evolve toward greater recognition of the gap between formal structures and actual practice, with emerging emphasis on adaptive capacity, stakeholder engagement authenticity, and transparent accountability mechanisms suited to complex health security challenges (1,1). Future development likely will focus on standardizing assessment approaches while maintaining flexibility for diverse surveillance contexts.
Governance overlaps substantially with organizational structure, collaboration, and system management concepts that appear in earlier frameworks (1,8). Its distinctiveness lies in three key elements: (1) explicit focus on power dynamics and decision-making authority rather than mere structural arrangements; (2) emphasis on the policy-institutional-operational continuum that links strategic intent to implementation; and (3) recognition of formal versus actual governance practices, distinguishing official rules from how systems truly function (1,12).
Measurement: Governance assessment remains less standardized than classical surveillance attributes, varying substantially by context, institutional complexity, and evaluation purpose. Measurement approaches include: (1) structural assessment through document review of policies, charters, and formal agreements (8,12); (2) functional evaluation using semi-quantitative scoring of governance criteria across multiple dimensions, as implemented in ECoSur’s 22 organizational attributes and 3 governance indexes (12,13); and (3) qualitative assessment through stakeholder interviews and participatory methods to understand real versus official governance (1,1). The challenge in governance measurement lies in capturing both formal structures and actual practices, requiring mixed-methods approaches that combine document analysis, stakeholder perspectives, and observation of decision-making processes.
Version 0.2
Granularity
ID: granularity
See also: completeness
Synonyms: Exact: N/A • Broad: N/A
Definition: The level of detail provided in input sources and output data of the surveillance system.
Description: –
Background: Granularity is considered a crucial attribute that defines the necessary level of detail for both input data sources and the resulting system output. For input data, granularity specifies the required level of detail, potentially tied to a single input field, a full data table, or the entire body of input sources. A critical challenge is preventing a mismatch where the fine granularity required by a system conflicts with the coarser granularity available from source data, as such discrepancies negatively impact surveillance outcomes. In event-based systems, granularity is essential for modeling and normalizing events with multi-granularity across thematic, temporal, and spatial dimensions. Dimensions like spatial location (D Z) or time (D T) can be organized into a hierarchy to consider different granularity levels. For system outputs, granularity determines the specificity of the information delivered, such as whether results forecast a specific location or population of infection, directly influencing the output’s ultimate utility for characterization and decision support.
Measurement: • Defining dimensions (spatial, temporal, thematic) with associated hierarchies. • Normalizing thematic entities using multi-granularity. • Discretizing dimensions (e.g., spatial location D Z and temporal notification date D T) over a set of geographic zones and time intervals to fix the scale of the evaluation. • Assessing the correspondence between required input granularity and available data granularity to identify mismatches.
Version 0.1
Historical data
ID: historicalData
See also: accessibility (data); data quality; data management and storage
Synonyms: Exact: N/A • Broad: N/A
Definition: Refers to the quality and accessibility of archived data.
Description: Effective governance operates at three levels: the policy level defines collaborative strategy and goals; the institutional level establishes governance modalities (steering mechanisms, coordination, technical support); and the operational level implements routine collaborative activities (12). These are documented through regulations, charters, or formal agreements with clearly defined roles in structures such as steering committees (8,12). Strong governance requires defining strategic goals, organizing resource allocation, ensuring system adaptivity through evaluation, and managing compliance with legal, ethical, and confidentiality requirements (1,17).
Background: "Historical data" is an attribute used to evaluate surveillance systems, initially focusing on the "Quality and accessibility of archived data". Its evaluation is essential for improving epidemiological understanding of a disease and serves as a secondary attribute for many other surveillance objectives within frameworks such as SERVAL. The longevity attribute of input data is conceptually related to historical data, assessing how long input data remain valid, often moving from a preliminary status to being archived, where it may be subject to revision. The existence and usability of these records, which encompass information collected over a long period of time, are fundamental for establishing baselines, allowing trend monitoring, and ensuring the continued representativeness of the data population under surveillance. For instance, a low quantity of reliable historical data can negatively impact a system’s ability to describe the population being monitored. Proper data management and storage methods, including rigorous archiving and backup procedures, directly affect the accessibility of historical data. In biosurveillance, the longevity of input data is measured by understanding whether the data expire or require revision over time, helping to determine the duration of the data’s usefulness. The presence of adequate historical data helps link information over time to assess long-term program performance and impact.
Measurement: • Assessment is conducted qualitatively based on evaluating the quality and accessibility of archived data. • Reviewing the longevity of the input data to assess the time frame over which the data are valid. • Determining if information has been maintained over a long period of time in a consistent way, and whether it is linkable to historical records. • Checking if data archiving and disposal procedures are properly managed as a system operation. • Comparing the volume of historical data available between systems (e.g., Praedico vs. CDW) to assess its impact on representativeness Questions to consider include: * How many years of data are stored? * How complete and reliable are the data? * Are the data stored in a way that allows the required interrogation and analysis? * Is there a summary overview of the data and collection methods explaining key idiosyncrasies of the data and changes to the data or collection methods over time? * What use is currently made of historical surveillance data?
Version 0.1
Impact
ID: impact
See also: Effectiveness; Benefit
Synonyms: Exact: usefulness, utility • Broad: benefit, value
Definition: The measure of changes that have been made based on surveillance results in relation to the system’s aims, including the actions taken and outcomes achieved as a result of the information provided (4,icahs-workshop-report?,Drewe_SERVAL_2015?).
Description: –
Background: Impact and usefulness represent closely related concepts that measure the ultimate value of surveillance systems, with their relationship reflecting different evaluation traditions and contexts rather than fundamental conceptual differences. Historically, evaluating surveillance systems included assessing usefulness by asking practitioners about system value, but more rigorous approaches involved assessing the impact of surveillance data on policies and interventions (Thacker_MEES_1988?,Thacker_PPHS_1988?). The term "impact" gained particular prominence in animal health surveillance, economic evaluation, and One Health contexts, where it became the preferred terminology in frameworks like SERVAL and EVA Survtool that assess ultimate value and benefits achieved (4,Hoinville_TRVA_2013?,Drewe_SERVAL_2015?). Impact is often defined specifically as the changes made based on surveillance results or the consequences beyond immediate effectiveness, relating to societal benefits of loss avoidance (Drewe_SERVAL_2015?,Meynard_PFAFM_2008?). This framing connects impact directly to economic evaluation methodologies, linking performance (effectiveness) to monetary benefit (cost-benefit analysis) or non-monetary benefit (cost-effectiveness analysis) (Peyre_TRVA_2019?,Drewe_SERVAL_2015?). Impact evaluation methods seek to answer cause-and-effect questions by comparing outcomes to counterfactuals—what would have happened without the surveillance intervention. In integrated surveillance frameworks like the Integrated Surveillance System Evaluation Project (ISSEP), impact represents a specific evaluation level (Level 5: Contribution to Desirable Outcomes), where ultimate outcomes include improved animal, human, and environmental health, along with associated socioeconomic benefits (Bennani_EISA_2021?,Sandberg_AoEFTfIS_2021?). Modern surveillance evaluation increasingly emphasizes impact assessment, driven by both donor accountability requirements and social accountability demands (4,Peyre_AITAEoPHS_2022?). Workshop participants assessing evaluation attribute importance have generally considered impact alongside costs as highly relevant attributes for system performance, sometimes viewing it as conceptually distinct from organizational benefits (Hoinville_TRVA_2013?). While impact and usefulness are explicitly recognized as synonyms in evaluation literature, impact tends to be emphasized in contexts requiring quantifiable evidence of return on investment and in integrated or One Health systems where demonstrating the value of cross-sectoral collaboration is essential. The attribute’s dependence on virtually all other surveillance qualities (sensitivity, timeliness, representativeness, acceptability, stability) reflects its role as a summary measure of whether the entire surveillance enterprise achieves its intended purpose of improving health outcomes.
Measurement: Evaluating the impact involves examining real-world examples where the system’s data has influenced public health actions, such as policy changes or control measures. This assessment often requires retrospective analysis and gathering input from key stakeholders to understand the system’s effectiveness. Key questions include how well the system’s objectives align with policy and industry needs, how outputs are utilized, and whetherstakeholder information needs have been met. Additionally, the evaluation should examine the system’s influence on the development of disease control policies, the prioritization of health threats, and the early detection or mitigation of outbreaks. It is also important to detail actions taken based on the system’s findings, such as changes in protocols or behaviors. The evaluation should assess the extent to which the system’s objectives have been achieved, providing a clear understanding of its impact on public health and disease management. Retrospective assessments and stakeholder feedback help evaluate whether the system’s outputs have guided effective actions. Regular review of system performance and stakeholder input helps ensure continued relevance and effectiveness in addressing public health threats. Retrospective assessment through stakeholder input helps capture real-world benefits, such as improved disease detection, control, or prioritization.
Version 0.1
Interoperability
ID: interoperability
See also: accessibility (data); data management and storage
Synonyms: Exact: compatibility, integration • Broad: N/A
Definition: Compatibility with and ability to integrate data from other sources and surveillance components.
Description: Interoperability helps ensure timely, efficient data exchange across sectors, disciplines, and administrative entities. Interoperability supports joint analysis, enhances flexibility, and improves sustainability. Effective interoprability requires standardized data formats, unique identifiers (e.g., sample or location IDs), and reliable data-sharing mechanisms. Challenges include fragmented or duplicative systems, lack of digital integration, or a lack of formal communication pathways. The importance of interoperability may depend on how essential inter-system data exchange is to the overall usefulness and effectiveness of the surveillance system.
Background: Interoperability is evaluated as the ease with which one surveillance system can be integrated into another in a suitable format for effective use by stakeholders. This attribute is explicitly defined in animal health evaluation frameworks, noting its necessity for linking different components and potentially enhancing effectiveness or reducing costs. Within public health surveillance, interoperability is recognized as a key concern of public health informatics, which must be addressed alongside system security and the use of standard data formats. Evaluations of surveillance systems, such as Thailand’s Avian Influenza (AI) surveillance, specifically added interoperability as a metric to gauge the system’s ability to cross organizational and sectoral boundaries. For multilateral approaches like One Health surveillance, achieving interoperability is essential for facilitating joint data analysis and overcoming challenges posed by fragmented or duplicative systems. Thus, evaluating interoperability involves verifying the adoption of standards in data formats, coding, and protocols, as these factors are prerequisites for high interoperability.
Measurement: • Assessing the ability of the system to apply evolving data standards and code sets (e.g., Public Health Information Network [PHIN] standards). • Evaluating how well the surveillance system or its components integrate/connect with other surveillance or public health systems,. • Determining if data systems are interoperable between species, geographic units, and pathogens. • Reviewing examples of deployment of similar systems in other settings to assess Portability (Generalizability). • Assessing adherence to standardized protocols and reporting processes. • Evaluating the technical and data standards used to facilitate interoperability and data sharing. • Assessing how integration affects the overall cost of the system. Description of the elements that are integrated in the surveillance system (Focus groups or individual interviews). Evaluation of the relevance and rationale of the integration with regards to surveillance objectives (Focus groups or individual interviews). Evaluation of the governance and multi-sectoral collaboration in the system (Focus groups or individual interviews, complementary tools). Overall evaluation of the level of integration (Semi-quantitative measurement scale, complementary tools).
Version 0.1
Laboratory and sample management
ID: laboratoryAndSampleManagement
See also: technical competence and training
Synonyms: Exact: N/A • Broad: N/A
Definition: Whether testing is carried out using appropriate methods with quality assurance scheme and timely and accurate delivery of results.
Description: Laboratory and sample management evaluates the integration, reliability, and quality of diagnostic laboratory functions within the surveillance system. Key considerations include the implementation of systematic quality management practices, participation in proficiency testing, and adherence to international standards (e.g., ISO 9001, ISO 17025). Effective sample handling, timely analysis, and standardized protocols and harmonization of techniques across laboratories support data comparability. Laboratory and sample management is also closely related to technical competence and training.
Background: TBD
Measurement: TBD
Version 0.1
Likelihood ratio of a positive test
ID: likelihoodRatioOfAPositiveTest
See also: TBD
Synonyms: Exact: N/A • Broad: N/A
Definition: Ratio of the probability of a surveillance system detecting an infected individual to the probability of the system incorrectly identifying them as infected when they are in fact not.
Description: Likelihood ratios do not vary with disease prevalence and so are stable expressions of system performance.
Background: TBD
Measurement: TBD
Version 0.1
Multiple utility
ID: multipleUtility
See also: flexibility; portability
Synonyms: Exact: multiple hazard • Broad: N/A
Definition: The ability of a surveillance system to capture information on several hazards; measure of how generic the system is.
Description: Multiple utility describes a surveillance system’s capacity to collect information on several hazards, reflecting how generic and adaptable it is. Systems with high multiple utility can support various surveillance objectives, improving cost-effectiveness and overall value. This term is positively linked to acceptability, sustainability, flexibility, and cos. Both realised and potential multiple utility should be assessed to identify opportunities for improvement. Systems with broad, representative designs generally offer greater multiple utility than those narrowly focused on specific risks.
Background: TBD
Measurement: An assessment of multiple utility should consider: - What additional information is or could be gathered during sample collection (eg on animal health or husbandry and demographics)? - What other types of samples are or could be collected at the time of sampling? - What other diseases are or could be tested for with the samples collected? - How long are samples stored following testing and could they be used for other purposes (including other research purposes)? For a surveillance system to offer value to other diseases or information needs, the objectives and processes of the system should be aligned to other systems. So it may be expected that more simple systems are likely to have more potential for multiple utility. For example, a simple random survey of holdings, repeated annually and with good coverage and representativeness could be useful for various diseases; whereas a risk-based design aimed at a specific threat may be of limited value for other diseases with differing epidemiology.
Version 0.1
Mutual benefit
ID: mutualBenefit
See also: collaboration
Synonyms: Exact: N/A • Broad: N/A
Definition: The capability of the surveillance system to create opportunities where collaborations can benefit both the system and its partners/collaborators.
Description: Mutual Benefit refers to the extent to which all collaborators gain value from participating in the surveillance system. This includes the availability of shared data infrastructure, clarity in roles and responsibilities, and the fostering of trusted relationships. Mutual understanding of privacy, confidentiality, and security processes supports sustained cooperation. Systems that enable data exchange and define partner benefits are more likely to promote long-term collaboration and stakeholder engagement.
Background: TBD
Measurement: TBD
Version 0.1
Negative predictive value (NPV)
ID: negativePredictiveValue
See also: positive predictive value
Synonyms: Exact: N/A • Broad: N/A
Definition: The probability that no health event is present given that no health event is detected.
Description: Negative predictive value (NPV) reflects the likelihood that a negative surveillance result truly indicates absence of condition, hazard, or event under surveillance. A high NPV suggests few false negatives and supports confidence in the system’s ability to rule out threats when no signal is detected. NPV is influenced by the system’s sensitivity, specificity, and the prevalence of the condition in the population. IIt applies acros various public health surveillance modalities, including lagroatory, syndomic, environmental, and even-based approaches, and should be considered alongside sensitivity and positive predictive value.
Background: TBD
Measurement: TBD
Version 0.1
Performance monitoring and evaluation
ID: performanceMonitoringAndEvaluation
See also: N/A
Synonyms: Exact: N/A • Broad: N/A
Definition: The routine use of performance indicators to monitor surveillance system performance, and the periodic use of internal or external evaluations to assess system outputs in relation to stated objectives.
Description: Performance monitoring and evaluation assesses the extent to which the surveillance system uses performance indicators and scheduled evaluations to track progress, measure effectiveness, and ensure accountability. This includes regular internal monitoring and external assessments of system components, including data quality, timeliness, sensitivity, and stakeholder collaboration. Performance monitoring and evaluation supports system transparency, enables corrective actions, and contributes to long-term improvement. It is closely linked to effectiveness, efficiency, impact, and sustainability.
Background: TBD
Measurement: TBD
Version 0.1
Portability
ID: portability
See also: flexibility; simplicity
Synonyms: Exact: generalizability • Broad: N/A
Definition: The ease with which a surveillance system can be replicated or implemented in a different context or setting.
Description: Portability describes how easily a surveillance system can be adopted or adapted in other settings or jurisdictions. High portability is supported by standardized formats, clear documentation, modular design, and minimal dependence on local infrastructure or specialized personnel. Systems that are portable enable broader uptake, facilitate scalability, and enhance knowledge sharing. Portability is positively associated with attributes such as simplicity, flexibility, and sustainability.
Background: TBD
Measurement: Examples should be provided of the deployment of similar systems in other settings, and the experience of those efforts should be described. In the absence of examples, features of the system that might support or detract from portability should be described. • Assessed retrospectively by observing how the system successfully responded to a new demand or change in operating conditions (e.g., adopting a new disease or revised case definition),,. • Evaluation of the flexibility of the system to meet changing needs, such as the ability to apply evolving data standards and code sets (e.g., PHIN standards). • Assessment should document reliance on person-dependent steps, including criteria related to judgment and action, to improve system replication,. • Measuring the additional costs involved in modifying the system for a new setting,. • Reviewing examples of successful deployment of similar systems in other settings. • Assessing system structure and simplicity,.
Version 0.1
Positive predictive value (PPV)
ID: positivePredicitiveValue
See also: negative predictive value
Synonyms: Exact: N/A • Broad: sensitivity
Definition: The proportion of reported cases that actually have the health-related event under surveillance.
Description: Positive Predictive Value (PPV) is the proportion of cases identified by a surveillance system that are true cases. A high PPV indicates fewer false positives, which helps minimize unnecessary investigtations and conserves resources. PPV is influenced by the specificity and sensitivity of case definitions, as well as the prevalence of the condition in the population. As sensitivity and PPV approach 100%, a system is more likely to be representative of the population with the event under surveillance. However, as sensitivity increases, PPV may decrease; therefore, a balance should be maintained between prioritizing early detection and accuracy. PPV may also influenced by data quality, communication, and reporting practices.
Background: TBD
Measurement: To assess PVP, external data sources, such as medical records, health claims, or registries, are often used as benchmarks to verify the true cases identified by the surveillance system. The assessment of sensitivity and of PVP provide different perspectives regarding how well the system is operating. Depending on the objectives of the public health surveillance system, assessing PVP whenever sensitivity has been assessed might be necessary (47-50,53 ). In this report, PVP is represented by A/(A+B) (Table 3). Evaluating Positive Predictive Value (PVP) involves assessing the accuracy of case detection and outbreak identification based on reported data. This can be done on two levels: individual case detection and epidemic detection. For case detection, the PVP is determined by confirming reported cases through investigation and comparing the number of true cases to the total number of reported cases. This requires keeping records of all interventions triggered by the surveillance system, such as the number of case investigations conducted and the proportion of cases confirmed. For epidemic detection, the PVP is assessed by evaluating the proportion of detected outbreaks that are true epidemics. This can involve reviewing personnel activity reports, travel records, and telephone logbooks to determine if resources are being misdirected due to false-positive case reports. Additionally, external data sources, such as medical records or registries, may be used to validate reported cases and improve the accuracy of PVP calculations. To obtain a comprehensive understanding, it may be necessary to assess PVP for different data sources, system data fields, and specific health-related events. Regular analysis of these factors ensures that the system effectively uses resources and minimizes the impact of false-positive results.
Version 0.1
Precision
ID: precision
See also: accuracy; bias
Synonyms: Exact: N/A • Broad: repeatability, consistency
Definition: The degree to which a numerical estimate is narrowly defined, typically indicated by the tightness of its confidence interval.
Description: Precision reflects the level of statistical uncertainty around an estimate, with narrower confidence intervals indicating higher precision. It is influenced by sample size, confidence level, variability in the data, and overall data quality. Precision is important for interpreting the reliability of surveillance indicators and comparing system performance over time or across regions.
Background: TBD
Measurement: Evaluation of statistical stability (precision) involves calculation of relative standard error of the primary estimate
Version 0.1
Preparedness
ID: preparedness
See also: N/A
Synonyms: Exact: N/A • Broad: N/A
Definition: Preparedness corresponds to the set of actions that are taken as precautionary measures in the face of potential hazard-related issues.
Description: Preparedness refers to the extent to which a surveillance system is equipped to respond effectively to public health emergencies. This includes having written procedures, emergency trained personnel, and readily available resources to support timely action. A well-prepared system can detect, report, and respond to emerging threats quickly and efficiently. The preparedness term is closely linked to timeliness, stability, and sustainability, as it depends on ongoing capacity, planning, and coordination.
Background: TBD
Measurement: TBD
Version 0.1
Redundancy
ID: redundancy
See also: N/A
Synonyms: Exact: N/A • Broad: N/A
Definition: The ability of the surveillance system to operate in the face of component loss or degradation.
Description: Redundancy refers to the presence of backup systems or components—such as staff, IT infrastructure, data sources, or interagency agreements—that ensure continued surveillance operations if one part fails. It supports system stability and resilience, allowing for uninterrupted performance during disruptions. Without redundancy, the failure of a single component can compromise data collection, analysis, or reporting. Redundancy is closely related to stability, robustness, and sustainability, as it enhances the system’s ability to maintain functionality and deliver reliable outputs under various conditions.
Background: Redundancy is an attribute explicitly defined as "system redundancy" in evaluation frameworks for systems like event-based biosurveillance. It measures the ability of a system to operate despite the loss or degradation of components, which may include staff, information technology infrastructure, input data sources, or interagency agreements,. The function of redundancy is to ensure continued operations, thereby supporting attributes such as stability and resilience, and preventing component failure from negatively affecting system performance,. For example, a system may use multiple extractors with failover and redundancy capabilities or run on clusters of hardware for distributed processing to increase reliability,. Assessing redundancy is key to understanding whether a system can be modified while remaining operational. In operational evaluations, planners must determine if sufficient funding, personnel, and material redundancy exists to support the system’s flexibility and ability to cope with operational deviations.
Measurement: • Assessment involves measuring the degree of component reliability, including whether the system can recover from component degradation or loss,. • Evaluating if there is funding, personnel, and material redundancy in place to support system flexibility. • Reviewing whether the system implements technical architectures such as distributed processing, load balancing, high availability, and failover support. • Determining if multiple extractors and redundancy capabilities are monitored to ensure they are operating correctly. • Assessing the presence of backup systems or components, including staff, IT infrastructure, data inputs, or interagency agreements, designed to maintain continued surveillance operations,.
Version 0.1
Relevance
ID: relevance
See also: effectivness; usefulness
Synonyms: Exact: pertienence • Broad: N/A
Definition: The degree to which functions and activities in the surveillance system are relevant to its objectives and priorities.
Description: Relevance assesses how well a surveillance system’s functions, activities, and data align with its objectives, priorities, and user needs. It examines whether the system addresses key data gaps, supports decision-making, and reflects epidemiological, socio-political, and economic contexts. Relevance also involves ensuring collaborative activities (e.g., sampling, lab testing, data sharing) match the system’s goals and stakeholder expectations. High relevance enhances system usefulness, acceptability, and sustainability by focusing resources on priority surveillance needs.
Background: Relevance, often referred to as pertinence,,, is explicitly evaluated as the degree to which a surveillance system’s outputs meet its stated objectives. In the context of the policy cycle, relevance pertains to the consistency between documented social needs and the stated policy goal,. Within Occupational Safety and Health (OSH) surveillance frameworks, relevance specifically assesses the degree to which functions and activities align with OSH objectives and priorities. Historically, the concept of public health importance is a precondition for surveillance, determining if the health event warrants surveillance based on factors such as frequency, severity, preventability, costs, and public interest,,,,. Relevance is considered one of five criteria in prospective evaluation methods, alongside timeliness, efficacy, evaluation, and sustainability.
Measurement: • Qualitative assessment based on measuring the consistency between defined social needs and the stated policy goal,. • Assessing the degree to which surveillance functions and activities align with predetermined objectives and priorities,. • Evaluating relevance as one of five criteria (alongside timeliness, efficacy, evaluation, and sustainability) in prospective evaluations. • Measuring factors related to public health importance, including indices of frequency, severity, preventability, and costs (related to assessing need/justification),,. • Reviewing the scientific basis of a system or indicator. Evaluation considers if unnecessary data collection is minimized and if performance indicators are linked to objectives.
Version 0.1
Repeatability
ID: repeatability
See also: N/A
Synonyms: Exact: – • Broad: consistency, precision
Definition: How consistently the study results can be reproduced over time.
Description: Repeatability is a key concept in validating diagnostic tests and is closely tied to precision. In the context of a surveillance system, it also relates to the terms/attributes of historical data, stability, and sustainability. A surveillance system that excels in repeatability generates data that can be consistently compared across years, with clear documentation of any changes in data collection methods or variables over time. This ensures that variations in the data are well-understood and attributable to actual trends rather than inconsistencies in data collection, facilitating accurate longitudinal analysis and comparison.
Background: TBD
Measurement: One might consider changes to legislation; changes to diagnostic methods, including improvements of adoption of new technology; changes to surveillance design; or influences on disease reporting behaviour in passive surveillance activities. - How have these impacted on the comparability of surveillance data over the time period of interest? - Have these influences been identified and examined and can they be accommodated in interpretation of the surveillance data?
Version 0.1
Representativeness
ID: representativeness
See also: Bias (ascertainment/performance bias); sensitivity; data quality; usefulness
Synonyms: Exact: N/A • Broad: coverage
Definition: The extent to which data adequately represent the population under surveillance and relevant sub-populations by time, place, population demographics and socio-demographics.
Description: Representativeness refers to how well surveillance data reflect the characteristics of the population of interest, allowing findings to be generalized without systematic bias. It considers factors like geography, demographics, and clinical presentation. While sensitivity focuses on detecting cases, representativeness ensures that the distribution of cases mirrors reality. High coverage across regions and groups supports inclusivity, while limited or uneven data can lead to selection or ascertainment bias. The representativeness of a surveillance system is related to both coverage and bias. Representativeness is also closely linked to terms such as sensitivity, precision, and data quality, and is essential for drawing accurate conclusions and informing effective public health planning.
Background: Representativeness was identified as a core quality attribute for evaluating surveillance systems as early as 1988. The attribute was recognized as assessing how accurately a system observes the occurrence of a health event over time and its distribution by person and place in the population. This principle has been consistently featured in subsequent guidelines, including the 2001 updated Centers for Disease Control and Prevention (CDC) guidelines. The concept’s definition generally focuses on ensuring collected data accurately reflect the true characteristics of the population of interest, allowing for the generalization of findings without systematic error. It is a critical aspect of data quality. A key mathematical relationship is that bias is reduced as representativeness is increased. Systems that approach 100% sensitivity and positive predictive value (PVP) are inherently more likely to be representative of the monitored population. However, surveillance systems often face the inherent problem of selective underreporting, where certain subgroups (e.g., those with severe illness, specific socioeconomic status, or seeking care in private facilities) are more likely to be captured, potentially introducing misleading conclusions about the distribution of the health event,. Evaluating representativeness remains challenging, often requiring validation through comparisons against external data sources that offer a complete or highly reliable picture (a "gold standard"), such as detailed medical records, mortality data, or demographic census data. Recent public health surveillance efforts have employed specialized techniques, including demographic analysis, network scale-up, and geospatial analysis, to identify systematically excluded population subgroups (such as the homeless or uninsured). For systems utilizing environmental sampling, such as wastewater surveillance, representativeness is assessed based on geographical area and time, but cannot currently describe distribution by person due to the pooled nature of the data. Maintaining representativeness is also influenced by system stability and the appropriate selection of the denominator for rate calculations.
Measurement: • Comparing the characteristics of reported events to presumed total actual events or cases,,,. • Comparing surveillance data with external sources (e.g., another survey, random population sample, mortality rates, or laboratory reports),,,,,. • Conducting specialized studies to identify a probability sample of all cases,,,. • Examining groups systematically excluded by the data source (e.g., prisoners, homeless, institutionalized persons),,. • Calculating rates and verifying the appropriate choice of denominator data (e.g., ensuring consistency between the numerator and denominator populations across categories such as age and race),,,. • Performing spatio-temporal representativeness assessment by comparing collected spatial data over defined time intervals against a reference standard. • For wastewater surveillance (WWS), assessing how the system describes occurrence over time and its distribution by place compared with clinical indicators,.
Version 0.1
Responsiveness (processes)
ID: responsiveness
See also: N/A
Synonyms: Exact: N/A • Broad: –
Definition: Responsiveness refers to the system’s ability to promptly address user requests and the willingness of support staff to assist with inquiries and provide necessary services.
Description: Responsiveness measures how quickly and effectively a surveillance system addresses user requests and support needs. It reflects the willingness and ability of staff to assist with inquiries, troubleshoot issues, and provide necessary services. High responsiveness supports timely data reporting, improves user satisfaction, and enhances system acceptability. Responsiveness is closely linked to timeliness and acceptability, ensuring the system remains user-friendly and operationally efficient.
Background: TBD
Measurement: Evaluation includes user feedback on response times and support quality, as well as tracking the availability and effectiveness of help resources.
Version 0.1
Robustness
ID: robustness
See also: N/A
Synonyms: Exact: N/A • Broad: N/A
Definition: The ability of the surveillance system to produce acceptable outcomes over a range of assumptions about uncertainty by maximising the reliability of an adequate outcome.
Description: Robustness refers to a surveillance system’s ability to consistently generate reliable results over time, even under varying conditions and uncertainties. It ensures stable, accurate outcomes regardless of environmental or data changes. Robust systems are closely linked to precision, as both require dependable results. Additionally, robustness ties into consistency, flexibility, and portability, allowing the system to adapt to diverse contexts. A robust surveillance system is crucial for providing reliable data that supports long-term public health decisions and interventions, making it vital for effective and sustainable public health responses.
Background: Robustness is defined as the ability of the surveillance system to produce acceptable outcomes over a range of assumptions about uncertainty. This is achieved by maximizing the system’s inherent reliability to deliver an adequate outcome, even when faced with unpredictable internal or external changes or constraints. The concept is measured in event-based biosurveillance evaluation as the fidelity of the system under significant departures from its assumptions. For example, this includes determining if a system can cope with low-quality data or adapt to currently unknown events. The characteristic of robustness is recognized in evaluation frameworks, such as the SERVAL framework for animal health surveillance, as a functional attribute of the system. It is considered important because robust surveillance enhances other system attributes, showing a positive association with precision, consistency, flexibility, and portability. Furthermore, the system’s ability to maintain stable and accurate results over time and its overall stability and resilience to system failures are intertwined with its robustness. Robustness is essential for making sound, long-term public health decisions and implementing effective interventions, especially considering the uncertainties inherent in complex systems.
Measurement: • Assessment can be conducted using info-gap models. • Simulations are recommended to study system performance across a range of scenarios. • Evaluation requires multiple simulation runs to test algorithm performance in different outbreak scenarios. • Proxy outbreak scenarios, such as seasonal events or community outbreaks, can be used to assess the system’s ability to detect threats, instilling confidence in its performance under real-world conditions. • Testing the system’s fidelity under conditions that deviate significantly from its initial assumptions.
Version 0.1
Security
ID: security
See also: N/A
Synonyms: Exact: N/A • Broad: N/A
Definition: The privacy and data confidentiality measures required to prevent surveillance data compromise, based on relevant standards and guidelines.
Description: Security in a surveillance system involves safeguarding the privacy and confidentiality of data to prevent unauthorized access or breaches. Security measures should be reviewed regularly and adapted to emerging threats. Strong security practices ensures data integrity and builds trust between actors/stakeholders of the surveillance system.
Background: Security, defined as protective mechanisms to prevent data compromise, is listed as a separate, novel metric added to the standard Centers for Disease Control and Prevention (CDC) evaluation framework when assessing advanced surveillance systems. Along with privacy and confidentiality, ensuring system security is considered essential for maintaining the credibility of any surveillance system. The lack of security in systems that deal with hardcopy data sources (e.g., manual paper-based forms) means that these data can be easily compromised. Surveillance systems must have policies and procedures in place to assure patient privacy, data confidentiality, and system security. For instance, a surveillance system must describe security procedures employed for data transmission and data management at the central repository. When military syndromic surveillance systems were developed, security was included as a required evaluation parameter due to the need for a high level of security for both medical and military data. Furthermore, the lack of security was noted as a challenge in Thailand’s piloted Avian Influenza (AI) surveillance system due to using multiple applications that were not integrated, exposing the system to data quality problems. The assessment of security should include evaluating compliance with applicable federal and state statutes and regulations. The proliferation of electronic medical records and the push for standardized electronic reporting means that assessment of security is a crucial attribute for modern surveillance systems. The implementation of the Health Insurance Portability and Accountability Act (HIPAA) has mandated the adoption of national standards for security of electronic transactions. Given the complexities of modern systems, the evaluation framework for early detection systems specifies that systemwide characteristics such as security, privacy, and confidentiality must be addressed in the operation description.
Measurement: Surveillance system security policies and practices should be reviewed to ensure that security levels and procedures for surveillance system data or system access are defined and enforced; data use and release policy and protocol is available; and access to the surveillance system software application is controlled.
Version 0.1
Sensitivity
ID: sensitivity
See also: specificity
Synonyms: Exact: recall (Morbey 2021), completeness (Groseclose 2010) • Broad: positive predictive value
Definition: The system’s ability to accurately identify and capture all true cases or events of interest within the target population
Description: Sensitivity can be assessed at three levels: Case detection sensitivity – the proportion of actual cases (individuals or herds) with the condition that are correctly identified by the system; Outbreak detection sensitivity – the likelihood that the system will detect a significant increase in disease incidence, provided a clear definition of an outbreak exists; and Presence detection sensitivity – the probability that the system will detect the disease if it is present at or above a specified prevalence level in the population.
Background: Sensitivity is a foundational attribute used to assess how reliably a surveillance system detects health events, classified into three main contexts: case detection (the proportion of cases detected), outbreak detection (the probability of detecting a significant rise in incidence), and presence detection (the probability of detecting a disease if it exists at a specified prevalence). The measurement of case detection sensitivity involves accounting for factors such as the diagnostic test’s own sensitivity, the extent of population coverage, and the likelihood of reporting by key actors. Historically, the term has often been used synonymously with the completeness of reporting, particularly for monitoring notifiable diseases. In application, increasing sensitivity often conflicts with other desirable system attributes, such as resulting in an increased volume of false alarms, thereby making the assessment of sensitivity against acceptable levels of specificity and Predictive Value Positive (PVP) crucial for effective system design. For early detection surveillance, the surveillance sensitivity is calculated as the product of population coverage, temporal coverage, and detection sensitivity.
Measurement: • Calculation using Gold Standard: Estimating the proportion of total cases detected by comparing reported cases (A) against all true cases found via an external, high-quality data source (A+C), represented by the formula Sensitivity=A/(A+C). • Capture-Recapture Techniques: Used to estimate the total number of unobserved cases in the population (A+C) to calculate system sensitivity, particularly for case detection and outbreak detection. • Scenario Tree Modeling: Used for planning and evaluating surveillance components, notably to estimate surveillance sensitivity (presence) by modeling the probability of detection under specified prevalence and component failure scenarios. • Early Detection Surveillance Sensitivity (EDSSe): Quantified as the product of population coverage, temporal coverage, and detection sensitivity (EDSSe=C p ×C t ×S ed ). • Statistical Assessment of Validity: Using statistical methods and simulations to compare a system’s signal generation (sensitivity) against false alarm rates (specificity). • Likelihood Assessment: Calculating the likelihood ratio of a positive test to evaluate the system’s ability to detect an infected individual versus incorrectly identifying them as infected when they are not.
Version 0.1
Simplicity
ID: simplicity
See also: acceptability; timeliness
Synonyms: Exact: N/A • Broad: N/A
Definition: Is the surveillance system structure, ease of operation, and flow of data through the system.
Description: Simplicity should be inherent in the system as a whole, as well as each component (case definition, reporting procedures, etc.), to make it easy to understand and implement. In general, a surveillance system should be as simple as possible while still meeting its objectives. A simple system typically enhances flexibility, reduces resource requirements, and contributes to better timeliness. Simplicity also positively influences acceptability and engagement, as actors/stakeholders are more likely to adopt and participate in a system that is easy to use and understand. Simplicity also supports sustainability by easing training, maintenance, and management, ultimately leading to more accurate and comprehensive data.
Background: Simplicity was established as a core attribute for evaluating surveillance systems as early as 1988 by the Centers for Disease Control and Prevention (CDC). The original concept focused on assessing the system’s structure and ease of operation, advocating that the system should be as simple as possible while still fulfilling its necessary objectives. The definition of simplicity has remained largely consistent across public health, animal health, and injury surveillance frameworks, universally referring to the straightforwardness of the system’s structure, its day-to-day operations, and the flow of data. Simplicity can be viewed through two corresponding perspectives: the design of the system (e.g., avoiding complex case definitions or multiple administrative reporting levels) and its ease of operation for the end-user. Critically, simplicity is intrinsically linked to several other attributes, generally demonstrating a positive correlation with both acceptability and timeliness, as less complex systems reduce resource burdens and encourage consistent participation. As a core design principle, maintaining simplicity is essential for maximizing operational effectiveness and economic efficiency,. Systems that rely on manageable operations, such as automated data handling, staff training, and clear organizational structure, enhance simplicity. Conversely, increasing system complexity (such as by introducing new technologies or additional reporting layers) risks negatively impacting timeliness, acceptability, and increasing overall operating costs and resource needs.
Measurement: • Creating a chart or drawing a flow diagram to assess the flow of information and lines of response to evaluate the complexity of the system structure. • Evaluating the amount and type of information necessary to establish a diagnosis or meet the case definition criteria. • Measuring the number and type of participating reporting sources. • Assessing the complexity of the data collection process, data analysis, and dissemination procedures. • Determining the level of system integration with other health information systems. • Quantifying staff time spent on core activities, including maintaining the system, collecting information, and preparing reports. • Reviewing staff training requirements. • Examining the clarity of case definitions.
Version 0.1
Specificity
ID: specificity
See also: false-alarm rate; sensitivity
Synonyms: Exact: N/A • Broad: N/A
Definition: The proportion of cases correctly identified by the system as not having the health-related event under surveillance.
Description: Specificity refers to a surveillance system’s ability to correctly identify non-cases, minimizing false positives. It complements sensitivity by ensuring that resources are not wasted on investigating non-events. High specificity is especially important in outbreak detection and in systems monitoring rare or high-impact conditions. It reduces false alarms, conserving time and funding. Specificity is influenced by factors such as case definitions and population denominators. In some contexts, a trade-off with sensitivity may be acceptable to avoid missing critical threats.
Background: Specificity is an attribute used to assess the quality of information provided by a surveillance system, defined as the proportion of true non-events correctly classified as such. The concept has been widely utilized since the 1988 CDC guidelines, where it was established that a surveillance system’s specificity is inversely proportional to the number of false positives it reports. A high rate of false positivity may suggest the case definition being used is too permissive or too sensitive. In the SERVAL framework, specificity is listed as a primary evaluation attribute for all listed surveillance objectives (monitoring endemic disease, case finding, early detection, and demonstrating freedom from infection), confirming its foundational importance across various surveillance purposes. When evaluating animal health surveillance, specificity is recognized as essential when estimating positive predictive value (PPV), as PPV reflects the system’s ability to accurately identify true cases among all reported cases (true positives/all positive reports). A direct measure of specificity is the false-alarm rate (the proportion of negative events incorrectly classified as events), a term favored by planners and policy makers for being easier to understand than specificity. For systems dealing with high-consequence events, such as bioterrorism or animal health threats with major economic impact, low specificity that generates false warnings can severely compromise decision-making and lead to wasted resources. For example, studies on detection systems noted that even with 99% specificity, low initial prevalence meant 88% of positive test results could still be false positives, highlighting the critical need for high specificity when prevalence is low.
Measurement: • Measured as the proportion of true non-events correctly classified as such. • Calculated by determining the proportion of negative events (non-outbreak periods) incorrectly classified as events (outbreaks), which is the definition of the false-alarm rate (the inverse of specificity). • Quantified through the proportion of true non-events compared against the total number of non-events. • Assessed by looking at the proportion of identified true cases out of all cases identified (PPV). • Determined by assessing the number of false positives (erroneous reports) relative to true negatives (non-events) to evaluate the failure of the system to correctly classify health events. • Compared against appropriate reference standards (or "gold standards") to define criteria for the condition of interest based on data source, data type, and response criteria. • Evaluated using Receiver Operating Characteristic (ROC) curves (which plot sensitivity versus 1-specificity) to compare performance across different detection thresholds. • Assessed using simulations that model outbreaks (or non-outbreaks) to evaluate the balance between specificity and sensitivity under varying scenarios.
Version 0.1
Stability
ID: stability
See also: sustainability
Synonyms: Exact: ability to contrinue tracking an event (sub-attribute), reliability, availability • Broad: N/A
Definition: Is the reliability (ie, the ability to collect,manage, and provide data without failure) and availability (the ability to operate when needed) of the public health surveillancesystem.
Description: Stability refers to the system’s ability to consistently operate and deliver results over time, even under stress or changing conditions. It includes reliability, availability, and resilience to disruptions. Stability depends on factors such as workforce capacity, secure funding, legal frameworks, and continuity plans. It is positively correlated with sustainability and acceptability.
Background: TBD
Measurement: Measures for determining stability can include the number or duration of unscheduled outages of the information system(s) supporting the surveillance system; the comparison of the desired and actual amount of time or resources required for the system to collect, manage, analyze, interpret, and release data from the system; or the presence or absence of continuity of operations procedures intended to maintain system performance. This attribute can be measured retrospectively by 1. Looking at the incidence of minor and major faults over a defined period of time or 2. Giving a measure of the proportion of time that the system is fully functional Assessment of this attribute will benefit from consultation with those involved in the generation, management and analysis of surveillance data. If performance indicators have been implemented in the surveillance process, historical data from these will give a good insight into the ongoing functioning of the system.
Version 0.1
Sustainability
ID: sustainability
See also: stability; feasability; N/A
Synonyms: Exact: N/A • Broad: N/A
Definition: The ability of the surveillance system to be ongoing in the long term.
Description: Sustainability refers to the system’s ability to operate effectively over time with consistent support. This includes having skilled personnel, sufficient laboratory capacity for timely sample processing, and clearly defined responsibilities for resource provision. Resources should align with current operational needs. Sustainability is closely linked to usefulness and timeliness, emphasizing practical procedures, adequate budgeting, and long-term system maintenance. It is dependednt on the continued engagement from stakeholders/actors involved in core surveillance functions, as well as the acceptability of end-users of the surveillance system.
Background: Sustainability is explicitly defined as the ability of the surveillance system to be ongoing in the long term. It is frequently linked with the attribute of stability, where the combined term "Stability and sustainability" is used to assess the system’s reliability (ability to function without failure) and availability (operational when needed), in addition to its long-term persistence. Core to achieving sustainability is the provision of adequate resources. Evaluation criteria directly assess whether funding is secure for both the short-term (e.g., 3-5 years) and the long-term (e.g., more than 5 years) future. Similarly, the long-term availability of necessary human resources (staffing) and material resources (e.g., technical infrastructure) is evaluated to determine sustainability. The issue of sustainability is fundamental in policy and decision-making, particularly given the constraints of dwindling government budgets. The value and utility of surveillance systems, often determined through economic evaluation, are recognized as essential for maintaining system sustainability. For instance, highly valued systems, often characterized by high multiple utility and acceptability, tend to be more sustainable. However, sustaining initiatives can be challenging, particularly when external or international funding is temporary or runs out, necessitating local financial commitment and institutional reforms. The continuous nature of surveillance itself often implicitly suggests sustainability, as surveillance is defined by ongoing systematic activities, contrasting with temporary "one-off" surveys. Effective collaboration and stakeholder engagement (acceptability) are also seen as vital to enhancing a system’s sustainability and impact, especially in complex surveillance and One Health initiatives.
Measurement: • Financial and Resource Assessment: Qualitative evaluation addressing whether funding is secure for the short-term (e.g., 3-5 years) and long-term (e.g., more than 5 years). • Human Capital Assessment: Evaluating if the necessary human resources (e.g., staffing) and technical resources (e.g., material infrastructure) are maintainable for both the short- and long-term future. • Operational Continuity Assessment: Measuring the percentage of time the system is operating fully and analyzing the time or resources required to collect, manage, and release data to gauge operational costs and resilience. • Stakeholder Commitment Analysis: Assessing the long-term willingness of individuals and organizations to continue contributing to the surveillance system, often based on internal resources and capacity. • Policy and Cultural Review: Conducting qualitative evaluations or internal review to assess if the surveillance activities are supported by an established legal or regulatory framework, enhancing long-term continuity. • Benchmarking and Gap Analysis: Using evaluation tools to monitor progress over time and identify areas of improvement that, if addressed, will increase the system’s durability and sustainability.
Version 0.1
System description
ID: systemDescription
See also: N/A
Synonyms: Exact: N/A • Broad: N/A
Definition: TBD
Description: N/A
Background: TBD
Measurement: TBD
Version 0.1
Technical competence and training
ID: technicalCompetenceAndTraining
See also: laboratory and sample management
Synonyms: Exact: N/A • Broad: N/A
Definition: Refers to the technical skills of the personnel involved in the surveillance system, including access to relevant training.
Description: The combination of the technical skills, knowledge, and capacities required by personnel to effectively implement and sustain a surveillance system, supported by access to appropriate initial and ongoing training programs. It encompasses the ability to perform system functions accurately, ensure data quality, and engage in collaborative activities, thereby enhancing the overall performance and reliability of the surveillance system
Background: TBD
Measurement: TBD
Version 0.1
Timeliness
ID: timeliness
See also: N/A
Synonyms: Exact: N/A • Broad: N/A
Definition: The time between any two steps in the surveillance process.
Description: Timeliness refers to the speed at which data are collected, reported, analyzed, and used to guide public health action. The required level of timeliness depends on the health event—rapid reporting is critical for acute, highly transmissible diseases, while less urgent conditions may allow for slower reporting. Timeliness is influenced by surveillance system design, resource availability, and event complexity. Delays in lab results or data flow can hinder timely response, especially in resource-limited settings. It must be balanced with accuracy and completeness and aligned with surveillance goals such as outbreak detection, trend monitoring, or long-term planning.
Background: Timeliness is a key performance measure of public health surveillance systems. It has been a standard attribute since the original Centers for Disease Control and Prevention (CDC) guidelines in 1988 and continues to be assessed in military, public health, and animal health systems. The definition of timeliness is generally the speed between steps in a surveillance system. The specific steps measured depend critically on the purpose of the surveillance, such as assessing the time required for immediate outbreak control versus long-term program planning. For outbreak detection, timeliness is measured from exposure to the infectious agent to the initiation of an intervention. This period covers intermediate milestones such as onset of symptoms, capture of data by the public health system, application of pattern recognition tools, and generation of an automated alert. For chronic conditions, which develop more slowly, rapid reporting may be less important than ensuring the accurate and timely availability of exposure data. Factors that can influence timeliness often involve delays across the system, including: • Data form: Transcription from paper to electronic form. • Processing time: Time needed for processing, editing, storing, and transferring data. • Case definitions: Those requiring specialized procedures (e.g., laboratory tests) can introduce delays. • Data availability: The source data latency, or the time between the current date and the most recent available data, directly impacts timeliness. • Technological limitations: The capacity for real-time data pulls can be limited by existing infrastructure. Efforts to improve timeliness often focus on implementing electronic reporting, standardizing processes, and ensuring continuous data flow. However, increasing timeliness may compromise other attributes, such as potentially lowering predictive value positive (PVP) or increasing costs. Conversely, in a study comparing surveillance systems, the overall system was more complete, but the active surveillance approach did not improve timeliness. Timeliness is considered a fundamental metric in many modern evaluation tools, including the SERVAL framework, OASIS, and the framework for early detection of outbreaks.
Measurement: • Measuring the time interval (delay) between any two defined steps in the surveillance process. • Calculating the median reporting delay (lags) in days between a reference date (e.g., date of disease onset, diagnosis date, or date of first report) and the final report date to a public health agency. • Calculating the percentage of cases reported within one or two estimated incubation periods for a specific disease. • Using a timeline with interim milestones (e.g., onset of exposure, onset of symptoms, capture of data, generation of automated alert, initiation of public health intervention) to improve the specificity of timeliness measures. • Comparing the timeliness of statistical alerts generated by a syndromic system against outbreaks detected by an existing surveillance system. • Modeling timeliness with an exponential decay function to obtain a normalized score between 0 and 1. • Using Cox proportional hazards regression analysis to analyze changes in median timeliness over time. • Calculating time-to-detection (or time lag) between wastewater viral load data and clinical indicators. • Qualitative assessment via key informant interviews to determine if timeliness meets surveillance goals or stakeholder needs.
Version 0.1
Traceability (data)
ID: traceabilityData
See also: N/A
Synonyms: Exact: N/A • Broad: N/A
Definition: N/A
Description: TBD
Background: TBD
Measurement: TBD
Version 0.1
Transparency
ID: transparency
See also: N/A
Synonyms: Exact: N/A • Broad: N/A
Definition: The extent to which information can be and is shared across member agencies.
Description: Transparency refers to how openly the surveillance system shares information about its data collection methods, analysis processes, and limitations with stakeholders and end-users. High transparency builds trust, supports data interpretation, and facilitates informed decision-making. Challenges include inconsistent or delayed data sharing, which can reduce stakeholder confidence and limit timely responses. Transparency is linked to terms like acceptability and usefulness, as clear communication of methods and constraints helps users understand system strengths and weaknesses.
Background: TBD
Measurement: Measurement may involve assessing frequency, clarity, and accessibility of data reports and documentation.
Version 0.1
Usability
ID: usability
See also: usefulness; communication dissemination
Synonyms: Exact: N/A • Broad: N/A
Definition: Data and products produced by the surveillance system should be tailored to meet the needs of intended users.
Description: Usability assesses whether surveillance data and products are tailored to meet the needs of intended users. High usability involves clear documentation (data dictionaries, manuals), minimal training requirements, and system workflows that align with user practices. It also considers how well data interpretations (e.g., statistics) are communicated, including accessibility features like language, cultural relevance, and format. The ability to stratify data by relevant factors (age, sex, geography) enhances usefulness. Regular user feedback guides system improvements. Usability links closely with acceptability and usefulness, influencing how effectively data informs public health actions.
Background: TBD
Measurement: TBD
Version 0.1
Usefulness
ID: usefulness
See also: effectiveness; benefit
Synonyms: Exact: impact • Broad: N/A
Definition: The extent to which the surveillance system contributes to prevention and control of health-related events by providing practical value that supports public health actions and policy decisions (4,anonymous.2001?,Azofeifa_EBHS_2018?,WorldBank_PPH_2006?).
Description: Usefulness encompasses two complementary facets: the system’s contribution to preventing or controlling health events (the direct public health outcome) and the practical value of surveillance outputs for decision-makers (the mechanism through which impact occurs). It is demonstrated through real-world outcomes such as policy changes, early outbreak detection, targeted interventions, or improved understanding of health threats that guide evidence-based responses (Groseclose_APHS_2017?,Peyre_TRVA_2019?). A surveillance system is considered useful when it provides value to stakeholders and society by enabling meaningful actions that would not otherwise occur (4,DelRioVilas_HSEitPC_2022?). The assessment of usefulness should begin with a review of system objectives and involves measuring how well surveillance outputs meet stakeholder needs and influence decision-making (anonymous.2001?,Azofeifa_EBHS_2018?). Both qualitative approaches (subjective views of users) and quantitative approaches (measuring impact on policies, interventions, or disease occurrence) can assess usefulness, though measurement is often considered inexact (Thacker_MEES_1988?,Declich_PHS_1994?). Retrospective analysis and stakeholder input help assess whether the system aligns with health priorities and produces information that stakeholders find valuable and actionable. Usefulness is closely tied to—and affected by—virtually all other surveillance attributes including sensitivity, timeliness, predictive value positive, representativeness, simplicity, flexibility, acceptability, and stability. For example, improved timeliness allows control and prevention activities to be initiated earlier, directly enhancing overall usefulness (anonymous.2001?). The quality of data determined by these underlying attributes ultimately shapes the system’s value in guiding effective, evidence-based responses and demonstrating that surveillance efforts justify their investment.
Background: Usefulness represents one of the seven original core attributes established for evaluating public health surveillance systems, with foundational definition and emphasis provided by Thacker et al. beginning in 1988 and subsequently codified in CDC evaluation guidelines (Thacker_MEES_1988?,anonymous.2001?,Groseclose_EPHS_2010?). This provenance establishes usefulness as a cornerstone concept in public health surveillance evaluation, reflecting the fundamental question of whether surveillance systems achieve their intended purpose of improving population health. The attribute has been consistently included in evaluations of diverse surveillance contexts including behavioral health systems, wastewater surveillance, and global public health surveillance under International Health Regulations (IHR 2005) (Azofeifa_EBHS_2018?,Amato_E_2023?). Usefulness encompasses two distinct but complementary facets identified in surveillance evaluation literature. The first represents the direct purpose of surveillance—the public health outcome itself, measured by the system’s contribution to disease reduction, control, or improved understanding (anonymous.2001?,Thacker_MEES_1988?). The second represents the mechanism of impact—how surveillance data translates into policy and action, reflecting the system’s demonstrable utility for stakeholders and decision-makers (WorldBank_PPH_2006?,DelRioVilas_HSEitPC_2022?). Both facets are essential: a system must both achieve public health outcomes and provide practical value to end-users to be considered truly useful. This dual nature is reflected in assessment approaches that combine both qualitative measures (subjective views of users regarding practical value) and quantitative measures (documented impacts on policies, interventions, or disease occurrence) (Thacker_MEES_1988?,Declich_PHS_1994?). Usefulness is explicitly recognized as synonymous with impact in evaluation literature, yet the terms reflect distinct emphasis and measurement traditions (dictionary_documentation?). While usefulness is typically framed around contributions to prevention and control or informing public health policy decisions, impact is often defined more specifically as the changes made based on surveillance results, frequently appearing in animal health, economic evaluation, and One Health frameworks (anonymous.2001?,Azofeifa_EBHS_2018?,Drewe_SERVAL_2015?,Meynard_PFAFM_2008?). The term "usefulness" emphasizes the practical, stakeholder-centered perspective of public health practice—whether the system provides value to those who use it and leads to meaningful action. This contrasts subtly with "impact," which tends to emphasize measurable outcomes and return on investment, particularly in contexts requiring economic evaluation or demonstrating the value of integrated surveillance across sectors. The conceptual alignment between usefulness and impact is substantial: both measure the ultimate value of surveillance by assessing tangible benefits, both focus on how surveillance influences decisions and interventions, both depend on the same underlying technical attributes, and both are highly relevant to stakeholders and decision-makers who need evidence that systems justify investment (4,Groseclose_APHS_2017?,Peyre_TRVA_2019?,DelRioVilas_HSEitPC_2022?). The choice between terms often reflects evaluation context and audience rather than fundamental conceptual difference—public health surveillance evaluations rooted in CDC traditions typically emphasize usefulness, while One Health and economic evaluations frequently emphasize impact. Regardless of terminology, the attribute represents a summary measure of whether the surveillance system achieves its ultimate purpose: improving health outcomes through informed action. Modern surveillance evaluation increasingly demands rigorous assessment of this attribute for both donor accountability and social accountability, requiring demonstration that surveillance efforts produce measurable value (4,Peyre_AITAEoPHS_2022?).
Measurement: TBD
Version 0.2
Markdown Rendering Issues
The following fields had issues rendering markdown:
- dataQuality (term_background): Unclosed change tracking markup found