1 Background

The first pieces of environmental legislation in the European Union (EU) entered into force more than four decades ago. Since then EU Member States have been regularly collecting and reporting comparable data on a wide range of environmental issues ranging from hourly pollutant concentrations in cities to seasonal measurements of bathing water quality. They also keep track of greenhouse gas emissions, energy consumption, pollutants released from industrial facilities, the size and location of protected areas, etc. These data streams are essential for monitoring progress and ensuring an effective implementation of environmental legislation.

In 2016, the relationship between legislation and environmental monitoring underwent a review. This has also been referred to as a “fitness check” on environmental monitoring and reporting [1]. This fitness check triggered a refocusing on key environmental data and created a legal basis for the coming years to better incorporate new trends and areas for environmental data collection. Meanwhile, the Shared Environmental information system (SEIS) [2] concept, itself a decade old, has been reinforced through ideas such as “active dissemination” and data harvesting. The data harvesting principle is already part of the SEIS principles (manage data as close as possible to the source; avoid duplication). The active dissemination concept is fairly new and has currently only been assessed by a set of pilot studies. The idea reflects the growth of environmental data and information which is too large to be reported in its entirety. The data should therefore be made available at the local or national level. Interested users in the EU Member States and the European Commission (as the owner of the data flows) would harvest national services on demand. This would occur in particular when the state of the specific environmental theme is not developing according to target. National data reporting would then be limited to a sub-set of data only. The EU’s INSPIRE directive [3] follows a similar principle building on a common spatial data infrastructure (SDI) to which data can be flexibly connected or, when legislation demands, a thematic subset can be reported.

The 2016 EU Communication on data, information and knowledge management [4] supports the aforementioned trend. It encourages increased information sharing and collaboration between EU institutions such as through the establishment of knowledge and competence centres. It also calls for capacity to handle “big data” and machine-readable open formats as well as stronger use of particular data and metadata standards.

The paper presents the evolution of European environmental reporting and how it has been used. It connects systemic changes in policy assessments whilst acknowledging that information systems themselves have evolved both from knowledge and technology perspectives.

2 EEA’s Knowledge Management Model

The European Environment Agency (EEA) has, since its establishment in 1993, provided a regular analysis of the state of environment using various methodologies. This analysis has been published in various forms such as reports, factsheets etc. The methodology underpinning this has always been based on the MDIAK concept (monitoring, data, indicators, assessment and knowledge) combined with the DPSIR (driving forces, pressures, state, impact and response) model.

While the five areas in the MDIAK model have not changed, the way they have been used has evolved over time. The technology used in collecting, reporting and analysing environmental data has developed remarkably in this period. We can now collect, store and process larger amounts of data. We can also interlink different data streams to draw increasingly more accurate analyses on what is happening and why it is happening.

Monitoring is coming from more sources and data of different kinds is emerging in high volumes. Our indicator and modelling activities are getting more substantial based on efforts to increase data and indicator quality. Assessments become more cross-cutting and systemic in order to target a user community which constantly grows in numbers and diversity.

A more in-depth look into these five areas (see Fig. 1) will be provided in the next two chapters.

Fig. 1.
figure 1

EEA’s knowledge management based on the MDIAK model

The EEA recognises that common EU rules ensure data compatibility and facilitate data and information exchanges across Europe. Yet to be able to fully understand environmental trends, it is essential to cooperate not only within the EU but also at the wider European and international level. The founding EEA legislation [5] established the Environmental Information and Observation Network (Eionet) as the instrument to link with the EEA Member Countries after 1993 and together with it, and a growing set of European and international partners, the EEA is in a unique position to process environmental information and foresee future knowledge needs.

3 Evolution of Environmental Monitoring, Data and Indicators

To underpin EEA analyses and reports with evidence, we needed to establish a solid data and information base. Since the start of EEA’s mandate, this was carried out in agreement with the management board and with the support of Eionet. The EEA subsequently established data flows based on voluntary arrangements with Member Countries and increasingly serviced data reporting on behalf of DG Environment. Those data flows were established under the environmental aquis - the set of legislation related to environment available at the European level. Data flows have been growing substantially in number and volume over the past years as displayed in Fig. 2. This led to the need to modernize our data handling system Reportnet which was introduced in 2002.

Fig. 2.
figure 2

Growth in reporting environmental data

Environmental indicators are another part of the way the EEA gathers and uses data thereby contributing to its “knowledge system”. Indicators serve many purposes such as allowing data to be combined through the use of different models to represent complex relationships. Indicators are also widely regarded as communication tools to simplify environmental messages. The EEA has consolidated its indicators during the past year into a focused number of high quality core set indicators (Fig. 3). Data collaboration has also taken place with the European Statistical System (ESS). To highlight their policy relevance, EEA indicators have been submitted as part of the European contribution to the Sustainable Development Goals’ (SDGs) indicators. This international policy process is a key driver for agreeing on environmental and other sustainable development related content for the years to come.

Fig. 3.
figure 3

EEA’s core set of indicators

Such development highlights the evident growth in environmental data and information. This has been complemented by an increased collection of data by citizens and private companies. Therefore the EEA has been required to build new data partnerships between public and private, local and global data owners and collectors. A related area connects human well-being with data from classical environmental sources. This human biomonitoring increases in relevance and puts the human exposure to harmful substances and the causes and consequences of this into focus.

A further area of growing importance is the increased availability of remotely sensed data. Copernicus, the EU’s earth observation programme, has been particularly responsible for driving growth through data from satellite observations which can be combined with data reported by monitoring equipment on land and at sea.

European environmental assessments are finally taking more advantage of research data as the open availability of this data is increasing. EU funded projects are increasingly obliged to make resulting data publically available. The open data movement in general and infrastructure initiatives like INSPIRE and the Copernicus program are further instruments through which free data availability is promoted.

In addition to particular data ownership and management issues, discussed later in this paper, large amounts of data present another challenge: how to distil relevant policy relevant knowledge out of large data flows. What do we need to know to improve the environment further? Such reflections may well lead to identifying new areas to monitor, others to discontinue or monitor less frequently. It might also require building new connections between data sets. Moreover, the knowledge needs of an urban planner may be very different from those of a European policy maker. How can environmental knowledge contribute to the management of complex systems like cities?

4 Systemic Assessment and an Extended Knowledge Approach

Despite some gaps, current knowledge on the environment is impressive and our understanding of specific issues has expanded over time. However, this enhanced understanding has also highlighted the need to look at the “bigger picture”—the need for more systemic analyses, looking at entire systems, such as on mobility and food. For example monitoring air pollutant concentrations can only take us to a certain point. The EEA cannot analyse and tackle air pollution without looking at other areas such as transport, the dieselisation of the vehicle fleet, agriculture, urban sprawl, and consumption patterns. The environment is complex and our knowledge base needs to factor in this complexity. It has become clear that we will increasingly need more systemic and cross-cutting knowledge in the future. In close collaboration with partners, and in support of the European Commission’s 7th Environmental Action Program (7th EAP) [6], the EEA is contributing to Europe’s environmental knowledge base with systems-based thematic assessments. The EEA does not only focus on past trends and the current state of play, but also on emerging issues and future knowledge needs. Further details on this approach and related work can be found in the most recent EEA Multiannual work programme (MAWP) [7].

Systemic analysis constitutes the core of the EEA’s latest European environment—state and outlook report released in 2015. One section within this report has tried to assess the influence of global trends on Europe’s environment and vice versa. Thematic EEA reports also address the issue at hand within a wider systemic analysis.

This very much pushes the need to adapt our data management to the new knowledge management approach. With input sources multiplying and data volumes growing we will have to adapt our information systems. This relates to many aspects: the governance structure around them – so who guides which process, the content they represent and the IT system which is used to process the data and disseminate the results.

While we used to work with a governance building on environmental datacentres (EDCs) we move to a wider governance steered by a European environmental knowledge community (EKC). A knowledge approach is widening scope and analytical opportunities. The demand for policy relevant knowledge is seen as coming from the interaction between many institutional players and those relationships develop subsequently. Due to the diversity of them, data variety is growing. Activities move from project type to more integrated program type (Fig. 4).

Fig. 4.
figure 4

Evolution of the EU institutional structure towards a knowledge approach

5 Data Management, “Big Data” and the Impact on the Information System

Several aspects have already been highlighted which are driving the future development of both data management and the information system as such. To acknowledge this, the EEA established a data and information management framework in 2015 [8]. Data management is far more complex as the input sources are much more diverse. Figure 5 shows the main areas from which data has been collected. Challenges lie, in particular, in the different quality standards, assurance and control which needs be addressed where data are jointly been processed into European data sets. The EEA runs a common workspace for processing the national reporting data which then needs to be complemented by data sets from other input streams.

Fig. 5.
figure 5

Highlighted components for management of EEA multi-source data flows

This is especially the case when producing environmental indicators with sources which need to be integrated. It is here where we start meeting the issues which are addressed as part of the big data discussion. Out of the five big data characteristics (volume, variety, velocity, variability, veracity) the increase in volume is obvious in the area of national reporting data and the huge data stream generated by Copernicus. Variety is growing particularly when looking at research data and the new data emerging from citizen science (e.g. on species distribution) or data from the telecommunications industry. Here, spatial patterns derived from mobile phone data can be related to human movements and used to produce new population density or transportation flow related data sets. Velocity, the concept which addresses the speed in which data is generated and processed, is gaining a bigger share in the monitoring data. Here the near real time air quality monitoring is becoming more operational. Data variability grows the more the questions put forward become systemic, and finally data veracity – the differences in data quality – continues to place limits on producing integrated datasets of good overall quality.

Data publication, independent from its explicit usage in assessments, is one area which is seeing growth within the EEA. The demand to download datasets in flexible ways is increasing as is the need to create regular and interactive visualisations. The provision of map viewers, where user statistics have been monitored since 2007, has been growing steadily since 2009 and a larger variety of web services have been established and are increasing in popularity in part because they are integrated by stakeholders into their websites and information systems.

The typology of assessment products has also been changing. The number and volume of reports have been decreasing. Some elements of those reports are provided through the EEA website whilst others are more targeted, and offered in a briefing format alongside smaller information products. In general such products are not offered in hard copy format however users can often customize them to their needs and print them as PDF directly from the website.

6 Information Technology and Tool Related Challenges

As has been discussed, new data processing steps require new workflows and more integrated tools. Figure 6 explains the key components and tools used for the management of the reporting data. It incorporates the update of Reportnet and shows the tools used for data processing in the common workspace. On the dissemination side, a variety of tools has been used – depending on the type of data and the target audience.

Fig. 6.
figure 6

Tools used in the EEA management of reporting data

For a number of years, the EEA has been using cloud technology to host services and distribute data. We can be considered as a ground-breaker amongst the EU institutions. The use of cloud services has allowed EEA to better address the challenges posed by big data. Demands on our services tend to peak towards the end of a particular quarter when major aquis reporting is due or alongside the publication of popular reports such as the one on European bathing water due in June. Access to web map services can be exceptionally high.

There is one area which is not specific to the environment yet creates the need to take serious and partially costly measures. Information and communication technology (ICT) security needs to be tackled more intensely. This calls to enhance several functions in the information system: back-up solutions for data need to improve; security needs to be enhanced, that is why data access is been categorized into different security levels. Based on this, authentication levels for several of our services should be enhanced beyond the standard “public” and “restricted” designations, alongside other security measures.