Data is often described as the new oil, but the analogy is limited with the fact that oil loses value when consumed but the value of data does not. The same data can be stored, sold and used in multiple different ways, completely without the knowledge of its original owner. In general, data cannot be depleted. This brings complex ethical concerns for evaluators in balancing the potential risks to research participants against generalised benefits and cost effectiveness of the research.
Some key concerns of ethical issues that arise as a result of digital data collection include – consent, privacy and confidentiality, ownership and authorship, governance and custodianship, data sharing– social benefits of the research. These are not the only ethical concerns but these are highly relevant to impact evaluation research using digital data.This demands for the evaluators to prioritize these concerns and respond accordingly at different levels.
Any research, evaluation and monitoring activity should:
- Seek to maximise benefit and minimize harm
- Respect people’s right and dignity
- Act with honest, competence and accountability
- Deliver work of integrity and merit
At 4th Wheel, we constantly aim to align with these principles in all our activities at each stage of the project, through the data’s supply chain by asking some difficult questions that help us be proactive on ethical issues in digital data collection.
1. Is the research useful, necessary and feasible?
Given the constraints of time and availability of resources and participants, we want to make sure that there is more value for money for potential beneficiaries.
We do a thorough review of our previous evaluations of similar programmes and other evaluations in the space to make sure that we don’t repeat but use appropriate research methods to draw relevant impact inferences for the programmes. This can help us be more precise in collecting only relevant data for the programme over digital platforms.
We spend sufficient time initially during the inception of the project to make sure that the expectations of the research are reasonable in view of the resources and time frame available.We also want to make sure that there is sufficient budget allocated for disseminating the research findings to different stakeholder groups, including beneficiaries in a manner that is accessible to them, so that there is transparency in the usage of data.
2. Is the research design fit for purpose and appropriate to context?
With participant observation and building rapport with stakeholders (observing local customs and norms) and other sorts of inductive inquiry (including in-situ snowball sampling of interviewees) becoming almost impossible, the struggle is going to be real for evaluators to develop a rich and contextualized perspective of the different programmes.
We at 4th Wheel with our decade long expertise in evaluation research design, make sure that our design evolves and adapts accordingly and are developed collaboratively with the different stakeholders of the programme. This helps us in accounting for the local context (social, cultural, religious values and beliefs) of the programme.
Sampling bias is also a huge concern with weaker sections of the society in the context of several development programmes not having access to quality ICT and there is a high possibility that they may not even show up in our sampling frame. We make sure that there is representative participation from different sections of the society and if not, we explicitly communicate the limitations of the research beforehand to our clients.
3. Is the participation based on informed consent?
We at 4th Wheel make sure that respondents know that their participation in the research is voluntary and they know how their data is going to be used in the future. Very often, terms like digital security and data protection may not be understood by the participants in development programme evaluations and we make sure that our enumerators are trained to explain to them in an accessible language.
We also make sure that any data pertaining to identify individuals specifically, like names are removed by providing the participants with unique IDs. This is our first and foremost step during data analysis for any of our projects.
We make sure that our enumerators are trained to build rapport over CATI (Computer Assisted Telephone Interviewing) with the help of our digital data collection softwares like Survey CTO. The software also enables us to encrypt information like telephone numbers to prevent its misuse but still be able to interview the participants over the phone.
4. Are the roles and responsibilities for data authorship and governance clearly defined?
The management, organization, access and preservation of digital data are all vital to research integrity and represent great challenges of the information age.
We at 4th Wheel ensure that we clearly state the data authorship and governance responsibility at different stages of the project in all our project documents. We clearly define who has access to the data, who is responsible for providing the metadata, who is responsible for long term maintenance of data and who is responsible for the disposal of data at the end of its cycle.
With all that said, we are aware that the field of digital data collection is continuously evolving with its different pros and cons and there is no one size fit all solution to the problem. While our research methods may change and adapt to the demanding situations, we at 4th Wheel are committed to maintaining high ethical standards when it comes to fulfilling our vision and mission in improving the way social programmes are conceptualized, implemented, monitored and evaluated.
- Thorley Lisa and Emma Henrion 2019, “DFID ethical guidance for research, evaluation and monitoring activities”
- Clark Karin et al., March 2015, “Guidelines for the Ethical use of digital data in human research”
- Baalen Sebastian 2018, “Google wants to know your location: The ethical challenges of field work in a digital age” Research Ethics Vol 14(4) 1-17
- Accenture 2016, “Building Digital Trust: The role of data ethics in the digital age”
- Hand David 2018, “Aspects of Data Ethics in a changing world: Where are we now?”