Wrangling Open Data to Create Useful Datasets for Applications

Open data is on the rise and freely available, bringing huge value and new features to applications.

However, the challenge for developers is to efficiently wrangle, store, and deliver open datasets without having to build complex data architectures.

In a recent DBTA webinar Raj Singh, developer advocate at IBM Cloud Data Services, discussed how to optimize location-based apps with open data.

He used crime data to illustrate how people can bring the data into an analytics environment. “Government open data has been used a lot but I think it is slowly getting to the maturity point where people are looking to operationalizing it and using it in a more pervasive and reliable way across application development,” Singh said.

Open data such as crime and governmental data is easy to find, Singh explained.

According to Singh, most of these cities releasing this data are using a tool called Socrata to publish its data. There is a single API that can grab data and make it easier to clean and sort.

After that data is located, it has to be queried and then place appropriately. The reason to collect the crime data in this instance was to pare it with the Pokémon Go application to see where the dangerous areas are, Singh said.

There are a variety of applications that can pull out these data sets including Cloudant and CouchDB.

An archived on-demand replay of this webinar will be available here.