The Open Kimono: Toward a General Framework for Open Data Initiatives in Cities

Source: Sage Journals | Pascual Berrone, Joan E. Ricart, Carlos Carrasco

As the urbanization process accelerates worldwide, many cities have joined the “smart city” trend1 and have employed various strategies that heavily rely on information and communication technology (ICT). These initiatives aim to create efficient energy systems,2 develop new business models for transportation and mobility,3 and improve health care services,4 developments that ultimately are expected to help foster sustainable communities.5

In this context, open data policies are becoming increasingly widespread.6 Open data are “data that can be freely used, re-used and redistributed by anyone—subject only, at most, to the requirement to attribute and share alike.”7 Supporters of this idea suggest that certain data should be freely available to everyone to use as they wish, without restrictions of copyright or other control mechanisms. An open data strategy involves developing a platform that promotes the use of open data. Open data strategies focus on a cultural transformation in the relationship the government establishes with other stakeholders. By revealing data to the outside, governments “open their kimonos” to share information between interested parties.

According to the Center for the Development of Information and Communication Technologies (CTIC) Foundation, the number of open data initiatives increased dramatically worldwide from 26 in 2009 to nearly 300 in 2014—nearly one-third of which are city-level projects. Moreover, the open data movement enjoys distinguished supporters such as Sir Tim Berners-Lee—the man credited with inventing the World Wide Web—who made a compelling call at the influential TED conference for data-gathering bodies to release “Raw Data Now!”8 Amazingly, the direct economic impact of open data on the EU 27 economy was estimated at €32 billion in 2010, with a conservative estimated annual growth rate of 7%.9 Even more optimistically, the consulting firm McKinsey estimated open data’s economic potential at more than $3 trillion in additional value in the global economy. In Spain alone, the infomediary sector (comprising companies that sell services on top of open data) generates €550 million annually.10 Although a recent report by the World Bank recognized variation in published estimates and some methodological difficulties, it concluded that the potential of open data was very large indeed.11

Despite the burgeoning interest in open data, different approaches to open data are still crystallizing across the world,12 and most companies struggle to realize economic benefits from open data.13 This struggle can be explained, at least partially, by the lack of a general framework to guide the process of formulating and implementing an open data strategy. In this context, a strategic approach to open data should provide a purpose and priorities consistent with the city’s goals as well as the course of action and initiatives needed to achieve those goals.

The article addresses this deficit by offering a general open data framework that urban governments can apply and that companies can use to participate in this process to harness the value of open data. We investigate the city of Barcelona as our baseline case because Barcelona represents a “success story in urban development across Europe”14 that can offer lessons for firms and policy makers regarding open data initiatives through the successes and challenges it faced. We combine insights from the Barcelona case with other cases and insights gained from the academic and practitioner literature to propose a model for open data and explore its implications.

Companies and entrepreneurs that intend to leverage the revenue growth potential of open data could be better positioned if they understood the key aspects of each stage of the process. Open data can benefit existing companies that provide urban services by supplementing their existing offering, companies that do not traditionally have cities as clients by offering new products related to open data (e.g., sensors and software), and new firms by developing innovative business models that rely on freely available government information. This article offers a stepwise framework that describes the tasks, challenges, outcomes, and key questions that policy makers responsible for urban policies should consider. Typically, open data have been considered a virtue, and publications seem to have a “pro-open-data” bias.15 Taking account also of the challenges of open data provides more a balanced, and perhaps more realistic, understanding of the phenomenon.

Drivers of Open Data

Traditionally, governments at different levels have used open data to spur transparency and accountability. This “openness” is often motived by the good governance principle of securing the “right to the freedom of information,”16 which has been widely adopted in Western societies. Open data then serve as a tool for citizens to monitor government performance. The United States’s open government strategy provides an example. In January 2009, Barack Obama announced in his memorandum to executive department and agency heads: “My Administration is committed to creating an unprecedented level of openness in Government. We will work together to ensure the public trust and establish a system of transparency.”17 In late May 2009, the U.S. government launched the website www.data.gov, one of the largest repositories of open data collected by governments.18

Open data can go beyond the notions of transparency and accountability and provide a tool for active public engagement, promoting citizen involvement in decision making. Increasingly, municipal governments are including their residents in core city activities and using digital platforms—such as wikis, microblogging, and video sharing—to gather information for projects through online brainstorming, crowdsource solutions to urban problems, coordinate public responses to emergencies and natural disasters, and even create civic juries.19

Moreover, open data can improve government services and management, for instance, by strengthening policing and law enforcement.20 In its government report Putting the Frontline First: Smarter Government, the United Kingdom explicitly mentions open data’s ability to strengthen law enforcement, stating,

The new online crime maps which went live in October 2009 mean that for the first time everyone in the country can search by postcode for facts about crime in their area and what is being done by the police and courts to deal with it.21

Another example is the opening of a data platform and an application programming interface (API)22 for 311 service requests in the city of San Francisco. In 2012, San Francisco opened up access to real-time transit data, resulting in 22% fewer calls to the 311 information line, saving the city $1 million per year.23

Finally, open data can foster economic progress by enhancing entrepreneurial activity and new product and service development. By opening up data, governments can thus stimulate innovative businesses and services that deliver social and economic value. The GovLab (Governance Lab) at New York University (NYU) recently launched a website named www.opendata500.com, a research study that documents the large number of U.S.-based companies that already use open government data. The research shows that private initiatives using open government data span several business areas, such as health, education, energy, real estate, and socially minded purchasing. Moreover, many of these companies would not exist without open government data. In brief, whether motivated by transparency concerns, participatory actions, improved municipal offerings, or commercial interests—or a combination thereof—open data ultimately aim to create social and/or economic value for citizens. However, an important question emerges: how can cities and companies effectively harness the power of open data? We explore the experience of the city of Barcelona in this context.

Barcelona Open Data

Barcelona, the capital of Catalonia and the second largest city of Spain, is situated on the Mediterranean coast and is known worldwide for the 1992 Barcelona Olympics. With more than 1.61 million inhabitants (4.6 million in the metropolitan area), Barcelona is the primary industrial hub of Spain, the most attractive location in the country for foreign direct investment, and the home of leading academic institutions. Barcelona also offers a rich culture, making the city one of the main tourist destinations in the world.24 Moreover, Barcelona is one of the most frequently mentioned cities regarding achievements in different areas,25 including open data. We thus analyzed the open data initiative of Barcelona, following a chronological structure to highlight the evolution of its open data project and the barriers and challenges faced by the process. We also highlight the role of private business in the different stages of the process. For this task, we used an inductive, qualitative approach to study the city of Barcelona, which is suitable for the in-depth exploration of phenomena that are not well understood.26 Data collection involved studying archival documents, interviewing, and observation. Our archival documents were drawn from more than 30 official documents accounting for more than 2,000 pages of information collected between 2012 and 2014, and the analysis included media clips. We also carried out several interviews with key actors and municipal bodies, including the political commissioner for open government and open data, the director of ICT projects of Barcelona, and the technical director of the Open Data BCN project. Finally, we presented and debated the case in front of an international audience of 14 urban managers (including of the city of Barcelona), representatives of multilateral organizations (such as the World Bank), and company executives. This last stage was essential to validate our model.

Background Information and the Vision of Open Data BCN

The Open Data BCN project27 was officially launched in March 2011. This project was not the first municipal program intended to provide open data. Since 1995, when the city’s municipal website, www.bcn.cat, was launched, the Barcelona City Council has offered detailed information to its citizens, such as mapping, geolocation information on municipal facilities, and agendas for major events. This desire to open the city’s information led to the creation of more than 130 municipal websites and, contrary to intentions, this has made it more difficult to find information across the numerous websites. Open Data BCN aimed to provide access to public data and to facilitate the process of information reuse, centralizing data that were offered previously on different websites, helping users search for city information, and increasing the efficiency of the overall process. Moreover, the project was expected to increase transparency levels after government transparency had deteriorated for years. Indeed, from 2008 to 2010, Barcelona fell from fourth to 22nd in the transparency ranking of Spanish city councils according to the report published by Transparency International Spain.

The Open Data BCN website was organized by the Open Data Working Group, technical managers of the city council under the guidance of the information technology (IT) department and the economic development department. This group collaborated with various departments to open their data by encouraging city managers to join the initiative.

The city council elections held in May 2011 saw power move into the hands of another political party, which decided to revamp the open data initiative. As the city was suffering from a severe economic crisis with high unemployment rates (a 16% overall unemployment rate and a 37% youth unemployment rate at the end of 2011), the new government sought to shift the focus toward addressing this problem.

The city council agreed that the new vision for the initiative would be for Open Data BCN to become an international benchmark model in transparent and open data strategy focused on economic promotion with a cluster of globally competitive companies in this sector. The aim was not only to release public data but also to generate a new culture of a data-driven economy, in which data can help to improve policy decisions (internally) and business decisions (externally), to encourage companies to also provide and contribute data to Open Data BCN. With this new vision, the city council established a number of strategic objectives to spell out and concentrate efforts arising from the overview:

Rethink municipal data as a raw intangible asset available to companies to create business models and generate economic activity
Establish a strategy for creating a hub of public and private open data
Define the aggregation and packaging level of data (e.g., raw, aggregated, crossed) and the supply conditions (e.g., free, freemium, size)
Integrate the initiative into the overall strategy to increase business activity, especially with regard to facilitating the smart city strategy.

All these goals required different time frames for deployment and varying degrees of work, so Open Data BCN was divided into phases.

Phase 1: Open Data BCN Development

Although half of the data sets were in the PDF file format, more than 500 data sets were available by mid-2011, and the website experienced a remarkable amount of traffic: 47,000 visits with 102,000 downloads in ten months, more than 335 per day. Regarding website visitors, three-quarters came from servers outside Spain, evidencing an international demand for public data. In mid-2011, the new government took over the initiative and decided to promote and expand it.

As the new strategic objectives were being reconsidered, doubts and questions began to emerge: what data should be published first? How could the government engage local businesses and citizens? How would the city council’s various departments respond to the strategy? How could they create a team to lead the strategy? What was clear for the city managers is that they wanted to be ambitious. As the Open Data BCN project director said, “when you have something new, it is better to look far ahead and then make changes as you go.” Given the need for a strategy, in fall 2011, the city council decided to collaborate with an academic institution, a top-ranked business school with a close and long relationship with the city council, to assist with this strategic definition process. This collaboration marked the beginning of the second phase.

Phase 2: Barcelona City Council and the Definition of the Strategy

The second phase began in early 2012 and aimed to create a system to expand the available data sets. Because the data came from different municipal areas, cooperation was required from the different departments in charge of the data. This was a significant obstacle faced by the promoters of the initiative. The academic partner began by analyzing the available data from the city council, and the results were used to identify which data sets were available in each department. In addition, informal interviews were conducted to understand why there was resistance to sharing data.

To define the work plan and account for both internal criteria and the external needs of economic stakeholders in the city, the open data initiative and the academic institution conducted various activities and meetings during the six months of the joint project (December 2011 to May 2012). These meetings aimed to present the open data project to businesses and business incubators to help develop a sense of community and understand the data needs of potential entrepreneurs. Once the internal and external needs were identified (e.g., information on traffic, building energy consumption, or neighborhoods), the next challenge involved deciding which data to release first.

Prioritization Criteria for Open Data BCN

According to the academic institution’s proposal, the criteria for prioritizing which data to open related to two dimensions: opportunity (internal or external demand, existing data sets in other open data initiatives, and the sector’s importance to the city’s economy) and ease of opening (technical, economic, or political).

The opportunity (demand) criteria

Regarding the demand for data sets, the first criterion was determined through several meetings held with private companies (both individual and group meetings) that helped identify specific demands for data access, for example, the nomenclature of Barcelona streets. Moreover, specific demands also emerged from the internal meetings regarding internal data use, such as noise pollution information or the tree-pruning schedule. The exercise also raised awareness of which data were available from other departments and where to obtain them.

The second criterion was a benchmarking analysis with other open data initiatives. An analysis was conducted of the main international open data initiatives such as those in Paris, London, New York, Washington, San Francisco, Chicago, and Vancouver, as well as other specialized portals (e.g., GovTrack, ePSI, and Euromonitor). This benchmark allowed Barcelona to understand better how the process of opening up data was conducted in other cities and what type of data was prioritized.

The third criterion was the sector’s economic importance to the city’s economy. Here, the academic partner analyzed social security data in Barcelona. The different sectors were classified according to an adapted version of the growth-share matrix methodology of the Boston Consulting Group.28 Sectors were categorized according to their market share (high or low) and participation in the city’s economic activity regarding employment (increasing or decreasing). The analysis identified three groups of star sectors: business services with high added value, education and research/development, and health and social services. The group including the tourism, leisure, and culture sectors was divided between cash cows and stars. The other major sectors were cash cows.

The ease of opening (supply) criteria

Regarding ease of opening, in-depth interviews were conducted with various city council department managers to determine the feasibility of opening up data regarding qualitative and technical ease of opening.

When the meetings addressed the qualitative aspects, challenges began to appear: several managers indicated they were reluctant to provide open access to certain data sets because of confidentiality (e.g., the data concerned individuals), a lack of confidence in the data quality (e.g., data sets on the procedures for and duration of licenses), or the desire to avoid demagogic or other misuse of information (e.g., crime data). These were legitimate barriers to open up data especially in the more “sensible” European context, but in some cases, they were interpreted as excuses not to release data. To overcome these barriers, it became important to have a political leadership committed to transformational change toward an open government culture; this leadership was exercised in Barcelona as we discussed in the section on the implementation phase. In that regard, Josep Ramon Ferrer (former director of Barcelona’s smart city strategy) said that open government involved a change in the way of doing politics, in the way of governing, and in the way of managing. “Technology is the engine that helps in this participation, this transparency, and this opening up of data,” he said.29 Being aware of the difficulties of cultural change, the city council preferred to move slowly to convince everybody.

What we wanted was the opportunity, through opening up the largest number of data sets (although they were in little demand), to achieve critical mass and to show the undecided that it’s OK to open up the data. Even to show them it is beneficial, as we have demonstrated with some cases. This has allowed us to accelerate cultural change,

said the Open Data BCN director during our interview. However, such changes are slow and work is still needed on this front.

Regarding technical difficulty, data sets were defined as being of “low technical difficulty” if they could exploit existing structures, could be collected easily, or could be updated with only minor adjustments. The level of difficulty was defined as “medium” if the information provider needed to perform preparatory work to integrate the data into the archive. Finally, the level of technical difficulty was considered “high” when the information provider needed to analyze the information and develop a specific solution for the data set.

Data Selection

An initial list of data sets was established, with priorities determined by city council managers who had been interviewed. After publication of the list, the academic institution created a scoring system to prioritize data according to the two previously defined dimensions: opportunity (demand) and ease of opening (supply). A synthetic index was produced by adding the scores for each of the two dimensions to create a two-dimensional prioritization matrix. The 24 initially selected data sets, which clustered into three categories (with decreasing priority), are depicted in Figure 1:

Quick wins—data sets with a high level of both opportunity and ease of opening
Strategic short-term data sets—data sets with either a high level of demand (despite being very difficult to open) or a high level of ease of opening (despite having a low level of demand)
Second-phase strategic data sets—data sets that are more difficult to open than other data sets and have a lower level of demand.