Statistical methods and tools

Overall objectives

The objective of this activity is to develop and provide tools, guidelines and methods for improving data quality for surveys on FDPs and host communities, as well as build capacity around their use. The proposed tools will be built around free or open-source solutions and designed to operate in capacity-constrained environments. They will address data quality issues in sampling, questionnaire design and fieldwork and data quality assurance.

Activity description

To guarantee the quality of different stages of survey design and implementation, this project proposes the following tools and methods:

1. Sampling and sampling frame: Develop open-source tools to provide data collecting teams with user-friendly, advanced sampling approaches that require minimal programming skills. To cover forcibly displaced and host populations in different settings with the most effective sample design, the tool will allow for standard sampling approaches for sample households in host communities, as well as:

• Sampling from a spatial grid in the absence of any existing area- or list-frame;
• Use of gridded population data like WorldPop
• Sampling based on other satellite-based data.

1. Questionnaire design: The proposed solution will enable users to plan data collection, to easily create questionnaires for populations affected by forced displacement with minimal programming knowledge. Building a repository for the questionnaire templates will help increase access and use of the templates and improve the quality of the survey instruments over time.

• Fieldwork and data quality assurance: This activity will develop guidelines and an application to enable quality control using the paradata produced during electronic data collection on FDPs and host communities. Real time monitoring and the use of paradata is a relatively new and is an underutilized source of data present in all electronic surveys. Processing paradata opens avenues for understanding the behavior of the survey respondents, interviewers and improvement of the questionnaire.

Engagement with partners

The core methodological and software development will be carried out by the World Bank in partnership with the JDC and other partners, such as UNHCR and National Statistical Offices. The World Bank will ensure effective and efficient knowledge exchange as well as wider survey data collection agenda.

Background and Context

Sample surveys are the main way to gather reliable information on socioeconomic conditions. This is also true of forced displacement situations, and even more so in the verification and integration of any alternate data sources, like satellite or big data. The quality of survey data is critical, as it determines how reliable the information is and the quality of future information, such as remote sensing data, which requires validation through primary data. Any errors in the collection process of primary data will probably create issues with the reliability of information in the future.

Sampling frames in some low- and middle-income countries are often not up-to-date, comprehensive or sufficiently informative o the target population and so are often incomplete. This problem becomes even more pronounced when the target population is small or hard to reach, which can be the case with forcibly displaced populations. As a consequence, the sampling frame and design are crucial to produce statistically valid estimates, which are required for evidence-based decision making.

Another component that affects the overall quality is the measurement error, which may arise from fieldworkers, respondents, the environment or questionnaire design.