Big Data for Development. You’ve heard that sentence/question/presentation title before. We’ve talked about it at length and the conversation continues in earnest in a time where more quantitative analysis is needed and increasingly expected.
Thankfully, excellent progress is being made. The World Bank’s 2017 Atlas of Sustainable Development Goals was very well received during this year’s Spring Meetings and the UN’s Action Plan for Sustainable Development Data, adopted last March, has increased provisions for geospatial and citizen-generated data, to name just a couple recent, major successes.
But when the primary providers should also be the biggest consumers, who is the main audience for these resources? How are we doing in using our own big data for organizational decision making? Who is making sure all the effort to is fed back into our operational decisions? What more can we be doing?
Our July 2017 Technology Salon DC, “How Can We Have Big Data-Driven Results for Development?“, explored themes in big data for development results, including thoughts from:
- Luis Crouch, Chief Technical Officer and VP at International Development Group, RTI International,
- Andrew Whitby, Data Scientist, Development Economics Data Group at World Bank Group, and
- Syed Raza, Senior Director of Data, Digital Impact Alliance (DIAL)
Here’s four main insights from our frank discussion from a conversation that focused far more on what is not working than what is, which itself speaks volumes.
1. It might not be time for big data for development.
We don’t need more volume, we need to be more revolutionary in our approach in using all or at least more of the data we currently have. Most organizations use about 20% of the total, if they even know of or have access to the “total,” considering in so many cases there’s not a main repository for data or employees know about it, have access to it, or are confident working with it.
Moreover, consider the capacity of our partners: cash registers in an average Walmart may generate more data in a day than a Ministry of Education with little access to electricity and equipment, still working in analog means. Organizations need to first evaluate and confirm intake from sources and analyze how much is currently — at least — accessed.
2. Consider the role of big data teams and approaches for your organization.
Is having a team specifically focused on big data a solution looking for a problem? We don’t have teams like “cars for development” so what is it about big data that warrants separate teams? Individuals on those teams are finding themselves most commonly in the middle of programs generating (or seemingly so), lots of data and thinking there could be a more data-driven approach is a better course of action.
However, data scientists brought in at this stage are asked to fire up clusters and create algorithms where they aren’t necessary and feeling stuck on telling their colleagues no. Those organizations looking at big data implementations holistically should be addressing org-wide administration around big data vendors and approaches, not inserted into existing projects to come up with a silver bullet.
3. Think about priorities in data collection, especially in public-private partnerships.
When it comes to technology and data-enabled projects, we’re obsessed with privacy and security. For good reason — our base aren’t just customers and a do no harm policy and approach are essential. But as a pitch to private companies, such as mobile network operators who do have access to truly big data, how much of that priority is leading our partnership pitches?
Our conversation needs to focus much more on service delivery, which is the language that private companies are most interested in and how we might help them in gaps in their metrics. It’s not unfair to our work to consider how our focus on “the last mile” is attractive to companies seeking new markets as their existing data is so valuable to our current work.
4. The biggest problems with big data aren’t due to technical expertise but administration restrictions.
Across the board, our biggest sense of agreement was that change management for executive teams and project leads would go further for most institutions in operationalizing big data approaches than adding more data scientists to the mix. When we have situations where the analytics or experts hesitate on big data solutions, or when the analytics don’t always deliver positive results, leadership needs to provide the top cover to make changes and allow for pivoting when needed.
Without first making foundation investments in team autonomy and procedural agility, big data is a hard path to an expensive sinkhole, both financially and in employee morale if this additional effort is ultimately not about allowing for change.