Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Augmented Reality

In many ways the term ‘Augmented Reality’ (AR) is problematic in itself but as with much in the field of ‘New Media’ it appears that for the moment it has gained wide acceptance in the absence of a suitable replacement. The term was originally coined by Tom Caudell and David Mizell in 1992 for applications in aircraft manufacturing at Boeing. It was associated in the 1990s with virtual reality type headsets such as prototypes like the Touring Machine (Feiner et al. 1997) and Map-in-the-Hat (Thomas et al. 1998) which were accompanied by weighty backpacks carrying the necessary computing, GPS and communication equipment, which now fits in a cellphone. Even today the HUD (Heads Up Display) paradigm still has traction as demonstrated by the interest in Google Glass,Footnote 1 however despite Google’s intervention, the HUD as a model of AR can be still said to exist in the nostalgia of “yesterday’s tomorrows” (Bell and Dourish 2007).

This association of AR situated somewhere along the real-virtual continuum, not quite real but not fully virtual either, serves to situate the practice in a scenario which I suggest looks toward the utopian values and ambitions of virtual reality and as such runs the risk of not attending to the real value of AR, which is its ability to contextually situate data. It is necessary to further distinguish the version of AR currently available for mobile devices from the richer conceptualisation of augmented space as articulated by Lev Manovich (2006) which encompasses the gamut of distributed information resources and is not solely confined to adding context specific information overlays to a specific location in a ‘camera-view’ interface.

AR in its current popular implementation, working on mobile devices through platforms such as Layar, Junaio and WikitudeFootnote 2 and through SDKs for Android and IOS mobile devices such as that offered by String,Footnote 3 is a more prosaic affair. Designed as a device led experience it offers a limited set of procedures involving the over-layering of dynamic, context specific data over live ‘camera-view’ of physical space. Typically this information is served from geo-tagged databases, both static and realtime, supplying information such as proximity of train stations, cinemas, nearby tweets and so forth. More recent developments include a move toward augmenting print content with the display of 3D models and the ability to trigger actions, such as playing an advertising video, leading a push to monetise the technology through AR advertiser tie-ins.

It is important however to look beyond the limited nature of many of the applications currently available for the range of AR browsers to attend to the affordances of these platforms. I draw attention in particular to the ability to import and situate geo-tagged databases which can attach contextual information to any site. This offers an unprecedented opportunity for the artistic and political activation of sites with large scale data-led critiques, particularly in conjunction with physical intervention. Despite the constrained nature of the engagement possible with available AR browsers they point to a growing convergence of a burgeoning world of open and accessible data, much of it geo-tagged or available for geo-tagging, with the ability to generate overlays which attach to specific sites in real space. As an emergent technology, the application of AR is still uncertain and open to re-imagining and negotiation. Locative Media pioneer Ben Russell identified a similar openness in earlier locative technologies which he saw as seeking “grassroots and consumer level interpretation of what these devices are” (2003), in these emergent AR systems there is this sense of a technology seeking usages which are meaningful to the broadest constituency, seeking to expand rather than constrain these technologies as they begin to enter mainstream usage. This presents an opportunity to artists and activists to shape these technologies, establishing them as tools for location based annotation and critique and expanding the range of applications and understandings for these technologies as they progress from new to mature technologies.

This opportunity coalesces around two factors. The first is Open Data; the European Commission (2011) estimates that open data could result in an additional €40 billion in economic activity per annum in the EU while McKinsey management consultants estimate this increase to be worth $3 trillion globally (2013). In addition to the growth in economic activity from its enabling of innovation, the smart economy and increased Government efficiency it is also seen as integral to the promotion of democratic transparency. Whatever the merits of the Open Data discourse it has incentivised Governments and city officials resulted in the release of vast swaths of open, machine readable data on all aspects of Government and city operations representing a significant opportunity. AR platforms represent the second factor, they offer an ease of use, are available as apps for a range of location aware smart phones, and while flawed, overly defined, and with limited opportunities for customisation they represent the first step in AR, and with time they will improve.

2 Data Driven Art

In considering AR art and its relationship with data it is important to locate the discussion within an artistic tradition of using data (open or otherwise) as a tool of political critique. I see the potential for the convergence of data space and real space which AR offers as existing within this tradition and will trace this through three artists who have exerted direct influence on the NAMAland project. These are Hans Haacke with his seminal Shapolsky et al. Manhattan Real Estate Holdings, A Real Time Social System, as of May 1, 1971, Mark Lombardi’s data based drawings and Josh On’s They Rule.

The case of Shapolsky et al. is of particular interest as it was a data rich installation detailing ownership of 142 tenement properties and sites in New York City in the ownership or effective control of the Shapolsky Family. The work was based on data derived from publicly available records, assembled and refined, in the case of obfuscated records designed to conceal effective ownership, by the artist. The work reveals the city as a real estate system, uncovering its complex structure and demonstrating the ways in which the physical fabric of the city, and the arcane financial dealings designed to maximise the value of real estate holdings, are imbricated. It expands the idea of site beyond physical location to include its associated data space. This serves to activate these sites through providing a socio-political narrative, transforming individual buildings through augmenting them with data and thus situating them within a complex network of property and financial transactions, with far reaching repercussions for the space of the city and the everyday lives of the people living in these slums (Deutsche 1996: 169–181). The piece was to be exhibited in the Guggenheim Museum but the exhibition was controversially cancelled before its opening in April 1971 with the specificity of the work cited as the principle reason. The Museum Director held that social issues should be addressed “artistically only through symbolism, generalization and metaphor” (Deutsche 1996: 179). What caused the work to be suppressed was the specificity of the critique, which data supplied, whereas a generalised artistic critique would have been acceptable. This demonstrates the power of the data-based critique which through its attention to detail builds a framework, which goes beyond what can be thought of as a purely artistic stance though it is bolstered by this, upon which alternative narratives can be based.

The artist Mark Lombardi is known for his large scale data based drawings or “narrative structures” which detail the networks of power and money involved in various political financial scandals such as the collapse of the Bank of Credit and Commerce International detailed in BCCI-ICIC-FAB, c. 1972–1991, (4th Version), 1996–2000. For each drawing Mark Lombardi built a custom database culled from published information sources assembled onto cross referenced index cards, according to his gallerist Deven Golden, he had around 14,000 of them (2003), which were then condensed to create his drawings. Lombardi considered these as a method of “reprocessing and rearranging” freely available information as a way of mapping the political and social terrain (Wegener 2011). The painter Greg Stone recounts the reaction of a friend, a reporter at the Wall St Journal, on seeing Lombardi’s George W. Bush, Harken Energy and Jackson Stephens drawing, who although he was familiar with the characters in the narrative, said he “hadn’t fully understood the implications until he saw it all laid out that way” (Richard 2002).

Josh On’s web based work They Rule (Fig. 4.1) pursues a similar mission of making connections between networks of powerful individuals, this time connected though corporate directorships once again drawing from publicly available databases. They Rule provides a front-end interface to its underlying databases which allows users to make their own connections and share them with other users. As a work of art it presents a framework to interface with the data, inviting its audience to provide the narrative structure and co-construct the meaning. Originally powered from a custom database of directorships of the top 100 companies in the US, it now employs the database of Littlesis, “a free database of who-knows-who at the heights of business and government”.Footnote 4

Fig. 4.1
figure 1

Screen capture from Josh On’s They Rule (2004)

Full size image

These projects illustrate that the power of data art lies in its ability to re-present information in ways which make the connections evident, presenting the information as narrative and revealing the underlying structures and patterns. How then can ubiquitous networked location-awareness of mobile devices and emergent AR techniques add to this tradition, and in an era where data and its use has assumed a greater importance than ever before, what has art practice to contribute to this field? This raises issues of site specificity and the reality of site being described not only as specific location, which the situated artwork addresses, but also as the invisible layers of data which extend our knowledge of the complex and multi-layered interactions between site, information and audience.

At this point I introduce a case study of a project by this author which follows in the tradition of data art. It is a work which doesn’t claim any technical innovation, created for an existing platform and built using free and open source software, but it offers a powerful example of the ways in which data can politically (and artistically) activate sites and, I suggest, a model for connecting data and space to create an activist hybrid-space (Harrison and Dourish 1996; Kluitenberg 2006).

3 NAMAland

NAMAland is a mobile AR artwork, built on the Layar platform (Fig. 4.2), which uses Open Data and Augmented Reality to visualise and critique aspects of the Irish financial collapse through an over-layering of the city of Dublin with a database driven data layer identifying properties under the control of NAMA (The National Assets Management Agency).

Fig. 4.2
figure 2

NAMAland in operation on the iPhone, Conor McGarrigle (2010)

Full size image

NAMA is an Irish Government agencyFootnote 5 established in late 2009 to acquire bad property loans from Irish banks with the aim of removing them from the banks’ balance sheets as a bailout mechanism. The agency, which was controversial from the start, acquired properties (or their related mortgages) worth €71 billion but failed in its stated aim of bailing out the banks, culminating in Ireland entering an IMF/EU bailout program in November 2010 due to the imminent collapse of the banking system. Despite (or perhaps because of) its pivotal role in the financial collapse NAMA was very secretive in its workings. Legally exempted from Freedom of Information requirements, the agency was intent on shielding its property portfolio, and the individuals and corporations involved, from public scrutiny under the guise of ‘commercial sensitivity’.

It became obvious that mapping NAMA’s property holdings was essential to gain an understanding of the organisation, and the events which led to its creation, in order to open it to scrutiny and critique. The NAMAland project, as originally conceived, was to build on Hans Haacke’s treatment of the entwinement of the Shapolsky real estate holdings and New York City to create an AR portrait of Dublin seen through the lens of NAMA properties. Through the specificity of such an artistic treatment of the agency it would, I hoped, be possible to build a more generalised critique of the financial bailout in all its complexity. A critique which could demonstrate an approach for addressing the politics of austerity which were sweeping Europe, and at that time concentrated in Ireland as one of the P.I.G.S.Footnote 6 countries.

To achieve these results it was first essential to research alternative sources of data on NAMA and its property holding as all official channels were closed. I identified an activist source of information on NAMA properties published on the anonymous website NAMA Wine Lake. Footnote 7 Maintained as a Google Doc, the NAMA-bound spreadsheet was compiled from published sources of information connecting property developers known to be in NAMA, their directorships of companies, and properties controlled by these companies. Through a process of collating available data sources and correlating them with known information on NAMA the unknown author built a partial picture of the NAMA holdings from this public corporate paper trail. Each entry was well documented with links to its original public domain sources, important in a litigious climate, demonstrating the difficulties of retroactively concealing data already in the public domain. Whilst one can only speculate about the method employed to collate this data it is expected due to its scale that it was produced from automatically data-mining newspaper records and public records of company directorships. This data was, however, locationally vague, street names were typically included with vague descriptors such as “site on Mayor St” but lacked in sufficient detail to automatically geo-tag especially with the precision required for an effective AR application. Building on the NAMA Wine Lake research I enhanced this data by manually geo-tagging approximately 120 Dublin properties through visually identifying the sites in person and tagging them with a handheld GPS unit. For legal reasonsFootnote 8 the database had to be confined to properties which could be located with a high degree of certainty for which sufficient documentary evidence of their ownership could be provided. This data was then used to create a geo-tagged mySQL database to be used as the data source for NAMAland.

The application was first built in October 2010 and has been updated on a regular basis since. It employs the Layar platform which provides a development environment and software platform to create AR applications which run on the Layar App for Apple IOS and Android devices. Layar provides a standardised user interface, with limited options for modification, supplying a standard set of AR methods upon which layers can be built. It was selected for two reasons; the first was ease of use, it imports a database effectively and is a reasonably robust working AR app which can be used with a minimum of development. Secondly it provided a method of publishing a politically sensitive work on the iPhone (at the time the most popular smartphone platform in Ireland) as layers are submitted to Layar’s own approval process and publishing through their proprietary iPhone app, effectively evading the appstore gatekeeping, essential for a politically sensitive app working with grey unofficial data.Footnote 9

NAMAland in operation takes the location of the user’s phone and compares it to this database of geo-tagged NAMA properties within certain defined ranges (Fig. 4.3). An overlay of properties within the specified range is then created which can be further interrogated for ownership details (the majority of properties in NAMA are associated with a small number of individuals with vast property holdings and billions in defaulted loans). The location of each response is indicated by an overlay of a cartoon “Monopoly Man” figure over NAMA properties in the camera-view of the user’s device. It also generates a real time map of localised NAMA properties along with a list of nearby properties and their locations. NAMAland thus visualises the extent of NAMA property ownership, allowing users to identify nearby properties and interrogate specific regions of the city for NAMA connections. It was the first mapping of NAMA properties available and is still the only available mapping of NAMA properties available in Dublin.

Fig. 4.3
figure 3

Screenshots showing NAMAland in operation in Dublin city centre

Full size image

4 Reception and Activation

NAMAland succeeded in capturing the popular imagination in Ireland. It was widely reported in the mainstream media, including a report on the Nine O’Clock TV News on RTE (the Irish national broadcaster), I was interviewed for numerous radio shows and the project was widely featured in the print media. The title NAMAland has even entered common usage as a descriptor for the post IMF bailout situation. In the midst of this extended “15 minutes” of fame the project has more importantly succeeded in focusing attention on its subject matter where more traditional approaches failed. It overcame official attempts to limit information and discussion on the subject, and has acted as a conduit through which concerns over the lack of transparency inherent in NAMA could be expressed.

On one level it operated as a mobile app, a ready-to-hand source of information locating NAMA properties, as a myriad of other apps locate coffee shops and restaurants, gaining in excess of 65,000 users in the process. However as an intervention, a physical dimension to the work was of the essence. The data layer which was enabled though AR was only of significance when overlaid over real space, this is the essential quality of AR, this connection between the virtual geo-tagged dataset and the physicality of place. AR must of necessity operate in conjunction with physical actions to be effective, augmenting space rather than recreating or virtualising space, and as such is the ideal companion for interventionist practices as it redefines the practices through extending and supporting their essential aspects rather than substituting a less effective alternative. The AR structure of the project was always designed to be an enabling framework upon which a range of additional actions, interventions, discussions and so forth could be based. NAMAland was extended to include real world events such as walking tours, situated public discussion forums, public speaking engagements, media coverage and individual interventions with the work itself being an amalgam of all its constituent components. These were all supported and enabled through the data layer made visible through the application of AR technology and offered multiple points of entry and modes of engagement with the project which were not necessarily technologically dependent. This ensured that the work remained open to as broad a constituency as possible, including those without the requisite technology to view the AR.

Indeed as the project disseminated it became clear that many of the people who spoke to me of the project were not actually users, as they didn’t have a phone capable of running the application. Their experience of the project was second hand, passed to them as a story which resonated as a tale of resistance. Somebody had used mobile technology to reveal a list of NAMA properties despite efforts to keep this information from the public. It didn’t seem even necessary to see it in operation, it was enough to know that it had been done. The walking artist Francis Alÿs speaks of his work as myth making, he sets out to “keep the plot of a project as simple as possible so that it can be told as a story, an anecdote, something that can be transmitted orally without the need to have access to images” (Godfrey 2010). NAMAland similarly has a simple narrative that can be told as a story, which means that even without access to the requisite technology the project still succeeds at some level. Not only does NAMAland recount a story about NAMA and its consequences, but from the point of view of AR it speaks of the technology and its uses. For this emergent technology this is significant for it is through practices that functions and usage modes of technologies come to light and their relative value and importance are revealed.

At another level it acted as a catalyst, facilitating a range of conversations, debates and activities as part of a wide ranging critique of NAMA and the sequence of events which led to it. The project crossed boundaries from art to geography, urbanism, activism, open data, economics and politics as one would expect from work which engages critically with the space of the city and international finance. As the project became known through publicity and word of mouth another side of the project was revealed from the diversity of the discussions, from the Occupy Dublin camp one day to city-sponsored seminars on Open Data and the smart economy the next, this was its ability to function as a conduit which reconnected NAMA with the space of the city, a connection which had been deliberately severed, to preserve the idea of the agency as a by-product of obscure international financial dealings. What NAMAland contributed was an opening up of previously unavailable data and a re-connecting of this data with the fabric of the city itself. This served to add specificity in place of generalisation, fuelling debate through the provision of an infrastructure on which specific spatial critiques could be structuring. This specificity, that is the ability to overlay contextual information at the site, enabled an alternative reading of the city providing a framework for intervention whilst countering the abstraction of space fostered by the narrative of the financial crisis as collateral damage of complex international financial transactions.

5 Peripatetic Activism

The project was accompanied by a series of walks informed by the mobile application which took place in Dublin City Centre and in Tallaght, two areas characterised by a high concentration of NAMA properties. These were public, as with the NAMA-Rama walk in conjunction with Market Studios (Fig. 4.4), the In These Troubled Times walk with RuaRed Arts Centre and Ireland after NAMA with The Exchange Arts Centre, and private such as the guided walks for RTE News and Channel Four News TV crews.Footnote 10 NAMAland is essentially a walking project, albeit facilitated through AR technology, it is necessary to deploy it on the street for it to operate at all. The guided walks, through careful selection of routes, were able to maximise this impact by proceeding through areas of the highest concentration of landmark buildings and, as participatory events, functioned as walking forums facilitating participants in discussing the issues represented by NAMA and its property portfolio. In this way the project connected the abstractness of the dataset to the space of the city through a narrative contextualisation which emanating both from the framing of the walks supplied by the artist but in a more significant way from the engagement of the participants. NAMA represents a complex system of abstract financial dealings, transactions which have become so disconnected from everyday understanding but yet have significant and very real consequences. Whereas the narrative of NAMA was the narrative of the (now defunct) property market, international finance and IMF bailouts, NAMAland reconnects this to real spaces exposing their interconnectedness and the real consequences on the space of the city and in the lives of its inhabitants. The interventions which NAMAland facilitated are thus framed and enabled though the production of a hybrid space which deploys augmented reality and data overlays to re-imagine the urban intervention as the generation of data rich hybrid spaces which can materialise and dissipate with the ebb and flow of the chosen dataset.

Fig. 4.4
figure 4

NAMA-Rama walk in Dublin’s docklands

Full size image

NAMAland on one level exists as a mobile app which over-layered the city with a contextual data layer re-presenting the city as a network of property and interconnected financial transactions which have bankrupt a nation. The ambition for the project was that it move beyond a purely oppositional stance. Generalised protest had at this stage been normalised and was easily countered by a narrative which invoked the need to move forward and rebuild, for change to come from the crisis it seemed necessary to set the agenda and shape that change. This is the benefit of a data-led approach, because of the specificity of the data-informed critique the alternative narrative is immanent to the critique itself.

Through data and the locational immediacy of AR NAMAland sought to recount a narrative of the city which ran counter to the official version, through revealing, contextualising and crucially locating the NAMA data in the space of the city and letting the users’ perform their own interpretation and form their own analysis. In this data-built account the data established the foundation and the narrative was constructed, not by the artist, but through this act of participation. Themes which emerged from the project were the question of data transparency in particular the need for NAMA data be made available for public scrutiny, and the demand for vacant NAMA properties to be made available for community use. These themes, which were central to the project’s public events, were widely taken up at a community, activist, academic and ultimately at a political level resulting in substantive changes to the situation.

6 Open Data

NAMAland was built on open data which was augmented with location information, in turn this data inspired further projects demonstrating the power of data to enable a myriad of approaches and interventions from mobile apps to occupations (Fig. 4.5). In late 2011 I came into contact with groups associated with the Dublin Occupy movement who were at the time interested in extending their campaign into occupation of NAMA buildings. This was a research-based project which stressed the accuracy of their data. The focus of their campaign was to call for the unlocking of a public resource for community and social usage so it was vital that their targets were correctly identified. The campaign was built on NAMAland data augmented with additional research. Their campaign resulting in a series of short lived occupations beginning in January 2012 which highlighting the fact that many NAMA properties were vacant and decaying due to neglect while there was a shortage of affordable space for community groups. A secondary guerilla program of identifying NAMA properties through affixing banners to their exterior was begun which once more drew attention to the neglect of these properties calling for them to be made available to social and community groups. These became part of the general conversation on NAMA and have achieved results, both through foregrounding the issues of NAMA properties and their usage and in opening access to properties.

Fig. 4.5
figure 5

Welcome to NAMAland banner in Dublin

Full size image

NAMAland through its activation of these sites has informed and influenced groups and through a focus on locational specificity supported by data has introduced new approaches to the urban intervention as artistic and activist tactic. The project has acted as a resource on which further actions can be built and the cumulative effect of these interventions and their surrounding debate has achieved some concrete results. Dublin City opened direct negotiations with NAMA to access vacant properties under their control for social and cultural use. This has resulted in a city program which allocates vacant buildings for cultural uses with substantial premises being made available. This has been accompanied by the release of more information on NAMA property which, while not nearly complete, has fed the growing demand that vacant properties be opened for community use.

7 The Future of AR Art

If we step back from the particularities of the platform and the case study to consider the implications of this project and similar practices on our understanding of the practice of AR.

I argue elsewhere (McGarrigle 2012) that artistic practices which engage with emergent technologies are involving in a process of shifting the understanding of these technologies. As Richard Coyne puts it “technologies do not conform politely to predetermined or intended functions” (2010: 4), rather it is through use that functions and usage modes come to light and their relative value and importance is revealed. AR as it stands is being promoted as a marketing technology, with the principle AR browsers developing corporate tie-ins using image recognition to replace QR codes in conjunction with location based AR applications. The technology is being thus presented and developed as a method of connecting companies with their customers in real space. While these applications will be a feature of the mature practice of AR, they are, to invoke the developers of the Urban Tapestries public authoring project, “unnecessarily impoverished” (Angus et al. 2008: 44–51).

Art practices have a role to play in broadening the understanding of technologies’ application through expanding their range of application and permitted usages. NAMAland demonstrates one such application, but the potential for these tools is only limited by the datasets which can be accessed and the desire by artists and activists to engage with them as part of their practice. At an everyday level this might be the difference between AR enabling a retailer to deliver location-aware special offers and deals to a customer’s phone alongside the ability of the user to interrogate the retailer’s history on a range of issues from health and safety to their environmental record or simply customer satisfaction. This is not necessarily to privilege one over the other. Both have their place but what is of the prime importance is that multiple options co-exist as aids to informed decision making, where the user can offset say a welcome discount earned by checking-in against the company’s anti-union policies.

NAMAland is an application of AR technology which has reached a wide audience through usage, mainstream media accounts and word of mouth by addressing specific local issues (with arguably a wider import). This success establishes AR as a tool of political and artistic critique which can reveal and situate information of political and cultural significance. NAMAland points toward the potential for the development of artistic and activist practices which expand and re-define the praxis of urban intervention through the ability to identify and activate site through the deployment of AR techniques, supported by contextual static and real-time data, to produce a hybrid convergence of geographic space and data space. This ability to generate site specific data rich hybrid spaces assumes a greater importance when connected to the Open Data movement and the popularisation of data scraping techniques with services such as ScraperwikiFootnote 11 and the growing community of data journalism advocates sharing techniques.Footnote 12 As new sources of data become available there are opportunities for artists and activists to go beyond the rhetoric of the smart economy and develop critical narratives and interventionist strategies based on this newly liberated data. If AR art practices are to shape the technology, expand the range of practices and establish the technology as a tool for enhancing and critiquing everyday life, then these practices must resonate with their audience and assimilate themselves into the technology through establishing meaningful connections to the everyday. This is a challenge for AR art and one that can be addressed through the astute use of data.