Methodology

Country researchers are recruited, and invited to complete the full survey. The survey is broken down into three sections:

  • Development
  • Commitments
  • Implementation

The Development and Implementation sections are split into five sub-sections, based on the IAP2 Participation Spectrum:

  • Inform
  • Consult
  • Involve
  • Collaborate
  • Empower

For each question, researchers give their answer on a scale. Scales vary between questions, but all generate scores of between 0 and 3.

For the commitments section, researchers are invited to evaluate five randomly selected commitments from their most recent National Action Plan.

Researchers may provide justificatory notes for their scores.

Following initial research, the results for each country are shared with a series of reviewers who may comment on the scores assigned, but not change the scores.

If there is disagreement over scores, then the survey is assigned back to the researcher to consider updating scores, or to further justify any scores that are unchanged.

Weighting

Each question is assigned a weighting, based on the results of a survey of the OGP community. This affects the relative influence of that question on the overall score for the sub-section and section it is within. The weighted scoring can be found in this Excel document: OGP Review Final Scoring [.xlsx]

Aggregation & reporting

In the commitment section, an average is taken of the responses on each question across the five commitments evaluated. For commitments question 6 (“To what extent does the commitment specify all government, civil society, multilateral or private sector partners?”) and 8 (“If the commitment is related to a commitment from a previous NAP, do you consider it to be a significant improvement?”) the average is based on only those commitments without a ‘Not applicable’ answer.

Sub-section scores are calculated as an average of the weighted scores for all questions in that sub-section.

Section scores are calculated as an average of the weighted scores for all questions in that section.

Sub-section and section scores are reported as a percentage of the total available score for the section.

The overall score is an average of the reported (percentage) section scores. This ensures each section is equally weighted in the final score, regardless of the number of questions it contains.