IA2030 – SP 1.1 Evidence of monitoring, evaluation and action mechanisms

Mean composite score

Indicator selected for global monitoring for Immunization Agenda 2030 Strategic Priority Objective 1.1: Reinforce and sustain strong leadership, management and coordination of immunization programmes at all levels. Data-driven decision-making is an indication of strong leadership and management. This indicator should help in uniting the key stakeholders to drive actions in an accountable manner. Information from ME&A exercises should be reported to higher levels, and recommendations fed back to lower levels. Actions planned/taken should be reported to higher levels and from higher to lower levels.

Mean composite score (0-3) of countries showing evidence of adopted mechanisms for monitoring, evaluation and action at national and subnational levels. Country-level composite scores are calculated by summing the number of the following three criteria met: 1. Presence of a functional NITAG or other equivalent technical advisory group (TAG), as defined by meeting six process criteria: 1) TAG has a formal written terms of reference, 2) There is a legislative or administrative basis for the advisory group; 3) Five areas of expertise are represented in the group as core membership: pediatrics, public health, infectious diseases, epidemiology, immunology, other; 4) Members are required to disclose conflict of interest; 5) Committee meets at least once a year on a regular basis; and 6) Agenda and background documents are distributed to members at least one week ahead of meetings 2. Monitoring, evaluation, and action cycles are in place at national and subnational levels 3. Feedback loop is in place to communicate assessments of progress and recommended actions from the subnational level to the national level and from the national level to the subnational levels • Baseline year: 2020

National Immunization Technical Advisory Group (NITAG) – A committee of recognized experts that provides technical advice and recommendations to the government regarding national immunization policies and programs. NITAG is a tool that enables the government to make evidence-based immunization policies through a transparent, systematic process. NITAGs are not regulatory, implementation or coordinating groups (such as Inter-agency Coordinating Committees or National Regulatory Authorities) and their primary function should focus on offering technical recommendations. As such, Inter-agency Coordinating Committees are NOT to be considered as NITAGs. Terms of References (TORs) – Documentation outlining the group’s purpose, serving also as a charter that outlines the group’s mode of functioning and code of practice for members. Conflict of Interest (COI) – To ensure transparency and avoid conflicts of interests as much as possible, NITAGs should require all members to declare their interests prior to official appointment. A conflict of interest (COI) occurs in the case of the member having a personal investment, activity, or relationship which may affect, or appear to affect, their responsibilities of the NITAG. A COI, whether real or perceived, can compromise the quality of the recommendations made by the group and can compromise the reputation and integrity of the NITAG. It can also compromise the credibility of the group, even if it would not influence the recommendations. Therefore, interests should be declared prior to the individual’s official appointment as a core member. The individual should only be appointed as a member if the person is considered an independent expert so that their interests do not compromise the integrity of the NITAG. Monitoring, Evaluation, and Action (ME&A) cycles –ME&A cycles for continuous quality improvement include the following steps: (1) Monitor: Measure and review key indicators on a routine basis defined by the country. Key indicators are measurable information used to determine if a program is implementing as expected and achieving its outcomes. Key indicators monitor performance of the national immunization program, could include any aspect of the immunization program (not limited to coverage indicators), and should be in alignment with the national immunization strategy or other strategic document. (2) Evaluate: Assess progress and identify areas for improvement (3) Act: For the areas for improvement identified, plan and implement actions to improve programmes.

WHO Member States

National immunization programme

Outcome

Country-level composite scores are calculated by summing the number of criteria (0-3) met by the country as indicated by their responses to the relevant JRF questions. Each criterion is worth one point. Mean composite scores are calculated by taking an unweighted average of all country-level composite scores. Step 1: For each country, determine whether a functional NITAG or other equivalent technical advisory group (TAG) is present. If so, indicate the country meets Criteria 1. Criteria 1 is met if a country reports meeting all six process criteria for a functional NITAG (See definition above). For the third process criteria - Five areas of expertise represented in TAG as core membership - if a country reports less than five areas of expertise on the TAG through the pre-defined JRF questions (pediatrics, public health, infectious diseases, epidemiology, immunology), but reports having an ‘Other’ area of expertise and/or lists areas of expertise in the Comment Field of the eJRF questionnaire, these are counted toward the country’s total number of areas of expertise. JRF Questions used in indicator calculation for Criteria 1: • Did the country have a standing technical advisory group on immunization (NITAG)? (Yes/No/ND/NR) • Has the advisory group formal written Terms of References (TORs)? (Yes/No/ND/NR) • Are there legislative or administrative basis for the advisory group? (Yes/No/ND/NR) • Are the following represented in the group as core membership? o Pediatrics (Yes/No/ND/NR) o Public health experts (Yes/No/ND/NR) o Infectious diseases experts (Yes/No/ND/NR) o Epidemiology experts (Yes/No/ND/NR) o Immunology experts (Yes/No/ND/NR) o Other experts (Yes/No/ND/NR), if yes, specify (free text) • How many times did the advisory group meet in {Year}? (Number/ND/NR) • Were the agenda and background documents distributed (at least 1 week) prior to the meetings in {Year}? (Yes/No/ND/NR) • Are members of the advisory group required to disclose conflict of interest? (Yes/No/ND/NR) Step 2: For each country, determine whether ME&A cycles are in place. If so, indicate the country meets Criteria 2. Criteria 2 is met if a country reported “Yes” ME&A cycles were in place at any subnational level (2nd administrative level or 3rd administrative level) and reported “Yes” to all other relevant questions on the JRF. JRF Questions used in indicator calculation for Criteria 2: • In {Year}, did your country have monitoring, evaluation, and action (ME&A) cycles in place for continuous quality improvement of immunization programs? o In {Year}, did your country measure and review key indicators on a routine basis? (Yes/No/ND/NR) o In {Year}, did your country assess progress and identify areas for improvement? (Yes/No/ND/NR) o In {Year}, did your country recommend, plan and implement actions to improve programmes? (Yes/No/ND/NR) • If yes, what level (national, subnational) were monitoring, evaluation, and action (ME&A) cycles in place? o National level (Yes/No/ND/NR) o 2nd administrative level (Yes/No/ND/NR) o 3rd administrative level (Yes/No/ND/NR) Step 3: For each country, determine whether a feedback loop is in place to communicate assessments of progress and recommended actions from the subnational level to the national level and vice versa. If so, indicate the country meets Criteria 3. Criteria 3 is met if a country reports “Yes” to the relevant question on the JRF. JRF Questions used in indicator calculation for Criteria 3: • In {Year}, were evaluation of immunization indicator results communicated between national to subnational levels? (Yes/No/ND/NR) Step 4: For each country, assign a country-level composite score (0-3) by counting the number of Criteria met. If a country meets all three criteria, they are assigned a score of 3. If the country meets two of the three criteria, they are assigned a score of 2, and so on. Therefore, a country will be assigned a score of 0, 1, 2, or 3. Step 5: Calculate the mean composite score of all countries. To calculate the mean, sum the country-level composite scores among all countries and divide by the total number WHO Member States (denominator equals 194 WHO Member States).

Unweighted mean of country-level composite scores (0-3)

N/A

N/A

Annual

Annual

vpdata@who.int

7776

Reported data is subject to numerous limitations. Country may not have submitted the data through the WHO/UNICEF Joint Reporting Form on Immunization (JRF) yet at the time of reporting. For newer indicators added to the JRF, it may take some time for the country to establish a system to be able to report the data. Additionally, countries with small population sizes may not have subnational systems in place to be able to report on all JRF questions relevant for this indicator, and program delivery in high-income countries may not be organized by the subnational levels and often rely on the private sector.