The inaugural South African Technology Salon How Can Big Data Solve South Africa’s Biggest Challenges?, we discussed the potential of Big Data in contributing to the achievement of development objectives in South Africa and the region.
Participating in the discussion was a range of public sector M&E officials, private and not-for-profit organisations, donors and programme implementers across a wide range of sectors.
Our lead discussants were:
- Jerusha Govender, Founder, Data Innovator
- Oscar Baruffo, Data & Strategic Projects, PETCO
- Sara Vaca, Founder and CEO, Visual Brains
Our discussants opened by acknowledging the data revolution and the changing demands from stakeholders.
We All Have Big Data
By recognising that data is now being generated at a speed and volume never before experienced, there is a growing demand for answering increasingly complex questions, but also answering simple questions faster and differently. We need to be better at understanding how we consume Big Data, and be better at using and learning from the data available as opposed to simply continuing to collect more and more data.
From an evaluation community perspective, the community needs to be more effective at advocating for the use of big data where it is relevant, and push back on more traditional M&E practices, that seek to simply ‘re’-collect data rather than recognise existing sources that may need to be ‘unlocked’.
There were some interesting examples of how (big) data exists in the health, education and financial sectors, yet, it is inaccessible to the M&E community. Accessibility to Big Data could really affect the cost of data – and reduce the barrier that is often believed to be regarding the ‘high cost’ of M&E tasks.
We also need to be advocates for open data and the sharing of data and answering of the complex questions, as none of us will be able to answer the challenges on our own.
To date, this shift has been experienced primarily in the private sector, and increasingly in the donor sector, but has not yet taken root in the public sector – exposing a major opportunity for changing the way decisions are made in South Africa and other countries.
The Challenges of Big Data
While these opportunities are presented to us, we need to be aware that Big Data also presents some concerns, such as ethical issues – particularly in a developing country context where legal rights and responsibilities are not necessarily fully understood, thus requiring stakeholders to be increasingly aware of ethical issues.
Additionally, extracting the full benefit from opportunities presented by Big Data, will only be realised with the establishment of infrastructure, systems and corresponding capacity to manage and use the data.
The current skills base in South Africa to leverage for enabling the use of Big Data is limited, as are the systems and processes – particularly in the public-sector context. Further, the value that could be derived if private data sources could be shared for public good.
A useful example was shared regarding vehicle tracking systems that have extensive data on the transport sector in South Africa, however; the data isn’t available to the Department of Transport for their purposes, a second example highlighted the Big Data held by large national retailers on consumer transactions, but again, isn’t available to the government for analysis and information purposes.
Do We Know How to Use Big Data?
There was quite a lot of debate and discussion regarding the nature of questions that are traditionally asked in the development context, and whether these are the right questions. Do we need to be thinking more about what changes need to be made to programmes and interventions, and thus, what data is necessary to inform and support these changes?
Researchers and evaluators need to be guided by specific questions so that the data that is out there can be sourced and applied to respond to the needs (as opposed to the data driving the analysis).
Finally, one of the major issues raised was ethics, the issue of privacy, and consent. Some in the room felt that we need to particularly conscious given the vulnerable populations in the region, however; there is also a viewpoint, that the ‘consent ship’ has sailed, and most people don’t realise that they have opted in when they open Facebook or other digital accounts.
There were discussions regarding the changing structure of organisations and how these need to cater for data and the identification of a requisite gatekeeper or watch dog roles within organisations to ensure data integrity.
Ultimately, the discussion was concluded by highlighting three main themes emerging from the Big Data, promises and pitfalls: 1) Access 2) Ethics and 3) How Big Data will change M&E practices in South Africa.
By Alyna Wyatt, Partner, Evaluation for Development, Genesis Analytics