Interfaces

An analysis of their history and impact on human interaction.

(Originally written for Data, Archives, & Infrastructure graduate course @ The New School with professor Shannon Mattern)

Humans are now able to retain large quantities of information, digitally stored and retrieved through networks (Arms 2000), with debates about the future of artificial intelligence overlooking the reality that human intelligence increasingly relies on the outsourcing of cognitive tasks to computers (Ware 2004). Interfaces exist within and between computers, between machine and human, and as p2p mediators on a global scale (Andersen & Pold 2011). Indeed, users live in what Andersen describes to be an “Interface Culture” (2014). Interfaces shape society, facilitating (or obstructing) sociopolitical transparency (Arns 2011).

Andersen defines an interface as a “technological artefact optimized for seamless interaction and functionality...that also draws upon cultural and artistic traditions” (2014). Interfaces are communication devices mediating memory and learning, and a tool with which users can achieve the otherwise unattainable (Quaggiotto 2018). As the computer evolves, its relevant systems are becoming more intent on influencing users, user behavior, and the user experience (Fogg 2003). Meanwhile, interface quantity continues to rise to satisfy demands and desires of individuals and markets (Andersen 2014). Interfaces mediate between user and machine, and technology and culture. Indeed, cultural activity is shaped by the interface’s effects on its performance and perception (Andersen 2014). Andersen suggests interfaces are potentially ideological constructs which balance user-machine submission and control (2014). Modern computer systems can also be considered interactive persuasive technologies (Fogg 2003 1), with negative effects on user-machine information exchange.

Computer science uses transparency to refer to the imperceptible (Arns 2011). From this perspective, a transparent interface successfully obscures itself and the underlying code, streamlining functions that please the utility-based perceptions of users. Andersen states the interface cannot be peeled away to reveal an essential core (perhaps a Cartesian fantasy in itself), as beneath surface interfaces and visualization, computer code is simply another interface layer (2014). Visualization itself is important because the process can enunciate problems within and around data, enabling quality control (Ware 2004). For Ware’s discussions of visualization, sensory visualizations are those requiring no learning, while arbitrary visualizations must be learned, as they are not grounded in perceptions.

Human perception is filtered by the many interfaces upon which individuals have come to rely (Andersen & Pold 2011). Interface theory suggests that unique individuals operating identical interfaces will report similar observations because of interface influence (Hoffman 2013 p.78). Further, interface difficulties can result in users seriously misinterpreting transmitted information (Arms 2000). Proposing a methodology for interface critique, Mattern asks if there are data or experiences incompatible with interface expression or transmission (2014), suggesting an interface could be defined both by what it handles, and what is decidedly off-limits.

Interface design has been influenced by epistemological struggles between empiricism and rationalism (Mahdjoubi 2003). Theory suggests the mechanism of the interface possesses an innate emotional modularity. Nusselder believes the nature of cyberspace related interfaces demonstrates mankind’s inability to separate technology from inner desire (2009). An interface can acts a means to visualize, or project, an object of desire before a user (2009). Computers, as information providers, are “bounteous mothers” (p.12). This gendered, maternal role is a mythological construct woven into systems; (predominantly male) fighter pilots react best to female computer voices, affectionately referred to by some as mama (p. 13). The web, a plane of intangible data on which a user may “wander around as a bodiless intelligence”, satisfies the “platonic desire for access to the immaterial realm of Ideas and Descartes’s ideal of disengaged knowledge.” (p. 57).

The nature of Screens demonstrates the ambiguity of psychic reality theories developed by Freud. Phobic objects displayed on visual interfaces generate a fascinating mix of reactions in the phobic user, who feels both threatened by the object and protected by the screen (Nusselder 2009 p.101). This example illustrates the difficulties of attempting to simplify concepts graphical user interfaces; they are too inconsequential to be real, yet too vivid to be fake. Nusselder connects these observations to Kantian and Lacanian thought distinguishing appearances from illusions (p. 101).

Fogg states that users may be unaware of their treatment of computers as living creatures (Fogg 2003). Citing the Tamagotchi craze as an extreme example, Fogg suggests that when an individual uses interactive machines, they frequently treat the experience as though it were an exchange with a living entity (2003). Indeed, colloquial consumer terminology for computer processes suggests the recognition of a living organism- machines sleep, wake up, die (p. 26). Social cues in a computer generate social responses in the user (2003). Such cues can cause human users to recognize a specific personality, perceive a computer to have a psychology, or even nuanced emotions (p. 94). Fogg states that malfunctioning computers are often perceived to be negligent, stubborn, or even “vengeful” (p.94). There is a correlation between users feeling matched with a computer of a similar personality to their own, with these users reporting the machine to be operating efficiently (2003). Fogg’s studies suggest that the persuasive abilities of a computer are enhanced in cases where users observe shared personality traits with the machine. Nusselder suggests the user-machine confusion over humanness may account for increases in online narcissism; with computers seen as humans by their users, the degree of control machines naturally invite is increasingly expected between human users interacting in cyberspace (2009).

While graphics-based interfaces are an iteration of a concept dating back hundreds of years (Andersen & Pold 2011), the interface is ancient. Hoffman suggests an evolutionary explanation for interface proliferation: the interface observer bases decisions on perceived utility. The interface observer is more fit for survival than the observer in a Bayesian model, as the interface observer selects that which maximizes utility (Hoffman 2013 p.75). Evolution has informed a human model of interface perceptions more concerned with discerning utility than detecting reality, which was lost to natural selection (2013). Indeed, the computer interface allows the user to sidestep the complexity of software and hardware and deal with objects (icons) attuned to utility (2013). The simplicity of the computer interface caters to utility measuring perceptions in a human user’s psychology (2013). However, prioritization of utility over truth does not preclude the calculation of consequence; user-controlled icon movement can indeed result in the deletion of valuable data, but Hoffman asserts that utility perceptions allow for things to be taken seriously without being taken literally (p.77).

Interfaces have contributed to epistemological and cultural changes (Quaggiotto 2018), with the carnival peep-show box thought to be a point of origin for relevant concepts of modern spectatorship emphasizing enclosure and privacy (Huhtamo 2011). Modern graphical interfaces emerged from tables of contents and bibliographies, which morphed into increasingly complex structures that were more analyzed than read (Quaggiotto 2018). The staircase became the tree, which informed the map (2018), with characteristics from all past models influencing contemporary graphic user interfaces. Like maps, interfaces have both an obvious instrument identity, and a more nuanced narrative identity; they distort knowledge passing through them (2018).

Design ergonomics emerged in the 1950s and voiced early concerns for the user experience in the design process (Mahdjoubi 2003). The mid-century science fiction concept of a helpful, endlessly reliable robot sidekick was supplanted by the emergence of the internet, which features a broad spectrum of trustworthiness and credibility (Fogg 2003). Today, knowing relates directly to user experience, and knowledge is synonymous with access to networks able to retrieve needed information on command (Quaggiotto 2018). The direct involvement of the user contributed to interface proliferation; contemporary interface production has shifted from initial software prototyping to the inclusion of network and user input, resulting in free labor for interface design (Andersen & Pold 2011). Indeed, socialization surrounds modern computer interfaces, and digital networks are able to draw attention to the social characteristics of orality that were overlooked by the advent of writing and the printing press (Quaggiotto 2018). Computing power is becoming more affordable, and the demanding calculations involved in content-based searching is fueled by the resulting availability of advanced machines (Eisenberg & Sproull 2003). Indeed, 20th century interface design contributed to the development of the internet, which triggered the proliferation of digital libraries (Arms 2000).

Digital libraries use the world wide web to transfer content (2003). A digital library is a “managed collection of information, with associated services, where the information is stored in digital formats and accessible over a network. A crucial part of this definition is that the information is managed [italics added]” (Arms 2000 p.2). Differences in terminology developed; an individual in a library is considered a reader or a patron, while digital libraries have users or end users (2000). For archives, movement toward entirely digital records traversed by user -controlled computer systems has been of relatively low cost and high reward (Eisenberg & Sproull 2003).

The processes of digital library operations involving users demonstrate the influence of user-machine interaction on the information transferred. Authentication is important, sometimes critical: international incidents have occurred from the dissemination or user acquisition of incorrect versions of international treaties that slipped through authentication processes, or avoided them altogether (Arms 2000 p.127)

The most common complaints against digital libraries relate to the quality of information, and the difficulty of information location and retrieval (2000). Curators of electronic record archives suggest that information requested frequently by users and site visitors is best stored (or duplicated) on a system separate from the long-term server (Eisenberg & Sproull 2003), as to liberate bandwidth to satisfy demand. However, designers of digital libraries are unaware of network speeds for users that navigate the results of their designs (Arms 2000), and complicated interfaces requiring significant bandwidth may act as gatekeepers of information (Fidel 2012) for the many individuals who cannot afford the required technology, nor the connection speed, to retrieve it. The digital library structure is evolving, and several decades from now, current models and concepts of digital libraries, and the internet, will be long forgotten by users (Arms 2000).

The desktop model of computing is a conceptual model, as users arrange and move files between simulated destinations as if they were tangible documents (Arms 2000). Interface design of desktop experiences concerns not only visual elements, but the manner in which functions are given to users (2000). Interfaces, the transference of data they facilitate, and user experience remain plagued by issues which threaten the integrity of recorded knowledge. Changes in user interface design have made users participants in processes they previously controlled (Nusselder 2009).

Interface data does not inherently constitute the type of knowledge informing action. As its tragedy began to unfold, the space shuttle Columbia reported accurate readings from a number of sensors to NASA programmers, who failed to promptly assemble the data into an understanding of active structural threat for the spacecraft (McCauley & Mosier 2006, p.244). The scenario model which did account for the readings (foam damage) was not integrated into programmer vocabulary, as it was not believed to produce effects as threatening as the readings were actively indicating (2006). The interface, in this example, was of little use beyond a live feed of disparate data collection points.

Users may have impairments that disrupt the user-interface relationship. Laws and regulations have drawn attention to the importance of global accessibility standards for the internet user experience (Kolar & Whitney 2015). The Fix the Web initiative aims to collect and address reports of individual instances of inaccessibility for sites (or non-compliance with relevant regulations or laws) (2015). The majority of accessibility complaints to FTW relate to screen legibility, concerning graphical user interfaces. Lacking authority on a global scale, FTW is powerless when dealing with many site operators and designers that ignore requests, lacking incentive to change (2015).

The effects of extant computer persuasion, and the potential persuasive capability of the computer interface itself, had not been considered nor studied until recently (Fogg 2003). Computers were not designed to act as persuasive platforms, they were designed for data input, storage and retrieval (2003). At the turn of the millennium, a collection of developers initiated the movement of persuasive computing technologies (2003). The introduction and popularization of the internet lead to the propagation of widespread computer-based persuasion of user behavior and thought (p.2). Some enduring examples tend to go unnoticed: Amazon does not simply facilitate transactions, it suggests purchases (2003).

Captology, an acronym taken from “computers as persuasive technologies”, refers to the study of the intersection of computing and persuasion (p.5). Fogg describes captology’s concept of persuasion to be any “attempt to change attitudes, behavior, or both” (p.15). Indeed, individuals interact through computers, but with them as well, and Captology is concerned with endogenous, intentional persuasion directly authored by the designers of the interactive technologies delivering it (2003).

Persuasion can occur through perceived or promised improvements in efficiency, and Fogg reminds readers that reduction technologies persuade through simplification of tasks, sometimes by removing steps, or reducing processes to a single click (p.33). Tunneling allows for the introduction of parallel processes, each of which may collect user data by feigning a relation to the primary task (2003). Tunneling, in some cases, can be coercive, threatening damage to user systems if users are non-compliant, falsifying a degree of authority (2003). Individuals tend to trust and comply with entities than convey a degree of authority, and this has informed the design of many persuasive programs; the anti-virus doctor is a more successful model than the tool or helper (2003). Information tailoring involves the personalization of the user experience, and the tendency for users to accept tailored recommendations has been exploited by designers for arguably unethical goals (2003). Further, “embedded” computing, occurring through machines nested in public environments and everyday devices, will allow persuasion attempts to traverse the user-machine experience largely unnoticed (pg.3).

Fogg stresses that computer-based persuasion can manifest in macro and micro changes in the user (2003). “Microsuasion” refers to the smaller persuasive elements embedded in larger operations (p.18). Microsuasion is often seen in the form of feedback that encourages continued user activity with the machine, and there are tremendous traffic returns for systems which incentivize activity in this way (2003).

Humanness is persuasive. Oscillotech made oscilloscopes that generated text-based messages for users. These messages were perceived as less personable than those of a competing brand. A redesigned, friendlier message resulted in users reporting the otherwise unchanged device to be more accurate than its predecessor (2003). This suggests a correlation between user-machine rapport, user-machine trust, and, perhaps most significantly, user opinion of machine generated data. Before content becomes a factor, the staggering volume of information handled by the computer can persuade users to blindly trust the interface (2003).

Praise, insincere or honest, measurably changes user behavior, and most users who receive praise from a machine believe that system to outperform an equivalent machine that does not offer praise (2003). Fogg’s research states users who collaborate with agreeable computers complete twice as much work as those paired with an unfriendly machine (p.109).

Responsibility for interface design and user experience is outsourced to designers and technical thinkers (Quaggiotto 2018). Quaggiotto suggests the primary epistemological implications of the outsourcing of cognitive functions enabled by the user interface relate to the externalization of memory and its effects on the structure of traditional thought (2018). These changes can be traced back to humanity’s shift from orality to literacy, with the outsourcing of memory facilitating the socialization of knowledge (2018).

Broadly, design research involves a narrow epistemological framework of Dewian philosophy, while design theory relates to centralized knowledge (Östman 2018). Dewey emphasized “practice, relations, and process” (p.2). Östman proposes the idea of design itself as design research, which addresses concerns over practicality (p.3). Indeed, design philosophy struggles to define what defines. Some approaches are recursive. Polyscopic modeling, for instance, considers a chair to be information, as it physicalizes chair making, and the experience of sitting (Karabeg 2012).

Human cognitive activities are known to be profoundly influenced by a user’s surroundings (Fidel 2012). Human information behavior (HIB) is a fairly new field of research concerned with the behavior information seeking individuals and groups (2012) which concerns user-machine interfaces and experience. Underfunded, HIB research is a nurturing practice concerned with helping others, staffed mostly by women (2012). In contrast, the technological side of human interface interaction (HII) is a male dominated field (Fidel 2012). Collaborations are elusive, and data collection can be obstructed by private sector interests. While HIB focuses on people searching with a degree of intention (2012), Surfing (searching without user intention of specific result) has been the focus of private sector proprietary research, with potentially valuable interface data unavailable to developers and researchers at large (2012).

How does HIB conduct research when the user experience occurs largely within the mind? Some methods are simple yet tremendously efficient: one study asked subjects using computer terminals to vocalize their thoughts, and their words were recorded (p.47). Unfortunately, interface researchers across disciplines clash over user-centered v. system-centered views of interface design, and priorities of HIB researchers and informational retrieval designers do not currently align in a manner that would facilitate a more epistemologically sound user experience (2012).

Andersen stresses a need to “subject our understanding of interfaces and how we co-exist with computational processes through them to critical scrutiny” (Andersen & Pold 2011, p.13). The concept of bypassing the interface, and establishing a direct connection between brain and machine, is frequently mentioned in discussions of interface design and theory (2011) and suggests a step forward. Indeed, the next leap in the field of cognitive psychology is expected to be the recognition of collaborative and cumulative cognitive abilities- individuals and groups interfaced with machines, and with each other (Ware 2004). A potential factor in a solution; distributed computing refers to the coordination of physically remote computers to facilitate coherent service (Arms 2000). Technical discrepancies between diverse computer models (operated by independent meatspace users) make distributed computing a challenge however (2000). Federated Digital Libraries overcome these obstacles by the adoption of and adherence to standards of user interoperability (2000), and perhaps users could reinterpret or adapt this model.

While U.S. and European schools of thought consider the prototype to be an outdated concept made irrelevant by design simulations, research into Japanese methodologies suggests extensive prototyping remains a focus in Japanese design (Papantonopoulos 2012). Papantonopoulos suggests this fosters (or conserves) an approach to general design based on experiential learning, indicative of the “epistemological orientation of Japanese culture” (2012). Proposed models for change to user interfaces should indeed follow this approach, empowering overqualified HIB researchers to bridge the gender gap and inform the male-dominated interface design process before interfaces are released to users.

When computers report measurements, credibility is both an ingredient in communication, and a potential casualty of inaccuracy (Fogg 2003). The standardization of credibility measurements could mitigate information degradation in user-machine interactions. Fogg distinguishes between surface credibility and earned credibility. Users evaluating based on surface inspections of an interface infer surface credibility, while positive results of long-term machine- user interaction inform earned credibility in the user for the technology used (p. 163).

Unfortunately, users tend to accept one way of making sense of experience, and this selection informs reality (Karabeg 2012). By this logic, communication between users of digital libraries, or the establishment of cross-user experience instrumentation, could preserve the integrity of information subject to user impression via interfaces. Establishing a means for short- term users to share in the reliability of long term-user credibility assessments would provide quality control for digital library interfaces. Indeed, data from team decision-making experiments suggest interface-generated feedback designed to guide human judgment result in task performance improvements due to an increase in cognitive control (Adelman et al 2006).

These additional datasets could take the form of decentralized shared feedback of user experience, potentially powered by blockchain technology to run as an independent layer atop existing digital library content streams. Arms states that efficiently designed systems do indeed provide user feedback, which could be as simple as an animation indicating processes in-progress, reassuring users patience will be rewarded (2000). Graceful Degradation, indicators which replace and correct previous inaccurate estimates of task completion, or enable access to partially-loaded data, could inform credibility decision making when shared anonymously between users (p.159). Separating these processes from systems that find and retrieve information may help to diversify utility driven human perspectives by introducing additional user voices into the digital library interface user experience, sourced horizontally via blockchain, and not from the library server.

Improvements to information degradation at the hands of user-machine dissonance could certainly benefit from ethical influences on interface design, as “computers can affect emotions, but cannot be affected by them” (Fogg 2003 p.218). Fogg argues that the ability to persuade using the technologies associated with computing involves a responsibility for programmers and designers to act ethically (2003). Fogg acknowledges that ethical concerns tend to accumulate around the idea of artificial social experiences authored by designers fully aware of the fact that users may be subconsciously misled (2003). He insists that to be an “ethical agent of persuasion”, one must assume a degree of responsibility not found in emotionally unidirectional machines or their distant programmers (p. 219).

User-machine dynamics relating to interface design continue to affect the information exchanges that characterize digital libraries, along with the user experience itself. In many ways, the humanity of users obstructs their ability to objectively navigate the digital interfaces they create. While ameliorative resources exist across disciplines, and collaboration may occur, there is little urgency to design interfaces which can address formidable user concerns and needs which grow alongside rapidly evolving technologies.

• Andersen, C.U. & Pold, S.B. (2014, January 14th) Manifesto for a Post-Digital Interface Criticism. Retrieved from http://mediacommons.futureofthebook.org/tne/pieces/manifesto-post- digital-interface-criticism
• Andersen, C.U. (2011) Minoritarian Tactics in the Age of Transparency. In Andersen, C.U. & Pold, S.B. (Eds.) Interface Criticism: Aesthetics Beyond Buttons. (pp. 7-19). Copenhagen, NV: Aarhus University Press
• Arms, W.Y. (2000). Digital Libraries. Cambridge, MA: MIT press
• Arns, I. (2011) Minoritarian Tactics in the Age of Transparency. In Andersen, C.U. & Pold, S.B. (Eds.) Interface Criticism: Aesthetics Beyond Buttons. (pp. 223-253). Copenhagen, NV: Aarhus University Press
• Adelman, L., Miller, S.L. & Yeo, C. (2006) Understanding the Effects of Computer Displays and Time Pressure on the Performance of Distributed Teams. In Kirlik, A. (Ed.) Adaptive Perspectives on Human-Technology Interaction. (pp.75-91). Oxford, UK: Oxford University Press
• Committee on Digital Archiving and the National Archives and Records Administration. (2003) In & Eisenberg, J. & Sproull, R.F. (Eds.) Building an Electronic Records Archive at the National Archives and Records Administration: Recommendations for Initial Development. Washington, D.C.: National Academies Press
• Fidel, R. (2012). Human Information Interaction: An Ecological Approach to Information Interaction. Cambridge, MA: MIT press
• Fogg, B.J. (2003) Persuasive Technology: Using Computers to Change what we Think and Do. San Francisco, CA: Morgan Kaufmann
• Hoffman, D. (2013) Public Objects and Private Qualia: The Scope and Limits of Psychophysics. In Albertazzi, L. (Ed.) Handbook of Experimental Phenomenology: Visual Perception of Shape, Space and Appearance. (pp.71-91). Oxford, UK: Wiley-Blackwell
• Huhtamo, E. (2011) Monumental Attractions: Toward an Archaeology of Public Media Interfaces. In Andersen, C.U. & Pold, S.B. (Eds.) Interface Criticism: Aesthetics Beyond Buttons. (pp.21-43). Copenhagen, NV: Aarhus University Press
• Karabeg, D. (2012, October 24th). Design Epistemology. Information. 3(4), 621-634. Retrieved from www.mdpi.com/2078-2489/3/4/621
• Kolar, I. & Whitney, G. (2015) Can We Fix the Web? In Sik-Lanyi, C., Hoogerwerf, E.J., Miesenberger, K & Cudd, P. (Eds.) Assistive Technology: Building Bridges. (pp.127 -134). Washington, D.C.: IOS press
• Mahdjoubi, D. (2003). Epistemology of Design. Integrated Design and Process Technology. Retrieved from https://www.ischool.utexas.edu/~darius/Epistemology%20of%20Design-5-IDPT
• Mattern, S. (2014) Interfacing Urban Intelligence. Places Journal. Retrieved from https://placesjournal.org/article/interfacing-urban-intelligence
• McCauley, S & Mosier, K. (2006) Achieving Coherence: Meeting New Cognitive Demands. In Technological Systems. In Kirlik, A. (Ed.) Adaptive Perspectives on Human-Technology Interaction. (pp. 241-257). Oxford, UK: Oxford University Press
• Nusselder, A. (2009) Interface Fantasy: A Lacanian Cyborg Ontology. Cambridge, MA: MIT press
• Östman, L.E. (2018) Design theory is a philosophical discipline – Reframing the epistemological issues in design theory. Retrieved from https://www.researchgate.net/publication/228625249_Design_theory_is_a_philosophical_discipl ine-Reframing_the_epistemological_issues_in_design_theory
• Papantonopoulos, S. (2012) An Epistemological Interpretation of Design Methodology in Japanese Product Development. European Union Executive Training Programme in Japan Tokyo Institute of Technology, Department of Industrial Engineering and Management. Retrieved from http://www.idemployee.id.tue.nl/g.w.m.rauterberg/conferences/cd_donotopen/adc/final_paper/26 9
• Quaggiotto, M. (2018) Images of knowledge: Interfaces for knowledge access in an epistemic transition. Retrieved from: https://www.researchgate.net/publication/266220166_Images_of_knowledge_Interfaces_for_kno wledge_access_in_an_epistemic_transition
• Thomsen, B.M.E. (2011) The Haptic Interface: On Signal Transmissions and Events. In Andersen, C.U. & Pold, S.B. (Eds.) Interface Criticism: Aesthetics Beyond Buttons. (pp. 43-63). Copenhagen, NV: Aarhus University Press
• Ware, C. (2004). Information Visualization: Perception for Design. San Francisco, CA: Morgan Kaufman Publishers