Philosophy of UX. Theories of perception for game design

Introduction

Philosophers spent centuries wondering about the nature of perception. Recently scientists coming from different subjects joined the debate. These interdisciplinary field of study is called cognitive science. However, there is one discipline left out that in the last decades showed interest in perception, the design. The publication of The Psychology of Everyday Things in 1988 by Donald Norman caught a trend among designers in those years. The idea of Norman is that a good design is strictly connected with a satisfying experience of the user; and to design a satisfying experience, the designer has to know how the experience and the perception work. This is how UX design came to light. In this paper I am going to question which perception theory, between different forms of representationalism and antirepresentationalism, is the most convenient in the UX design; in order to reach my goal I am going to analyse the conceptual tools that UX designers commonly use, making them clear, refounding them and redefining them through the perception theories. This research hase a double purpose: to clarify the multitude of theories of perception thanks to the design, but at the same time to improve the tools of design thanks to the theories of perception.

 

 

The following bibliography refers to the content of the written paper (soon online in ITA and then in ENG too).

 

Bibliography

Brewer, B. (1999). Perception and Reason. Oxford University Press.

 

Carlson, M., Taylor, L. (2019). Me and My Avatar: Player-Character as Fictional Proxy. In Sageng, J. R. (Ed.), Journal of the Philosophy of Games, 2(1). https://doi.org/10.5617/jpg.6230.

 

Chemero, A. (2003). An Outline of a Theory of Affordances. Ecological Psychology, 15(2), 181-195. https://doi.org/10.1207/S15326969ECO1502_5.

 

Chomsky, N. (1959). A Review of B. F. Skinner’s Verbal Behavior. Language, 35(1), 26-58. https://doi.org/10.2307/411334.

 

Coelho Mollo, D. (2022) Deflationary realism: Representation and idealisation in cognitive science. Mind & Language, 37(5), 1048-1066. https://doi.org/10.1111/mila.12364.

 

Cohen, N. R., Cross, E. S., Tunik, E., Grafton, S. T., Culham, J. C. (2009). Ventral and dorsal stream contributions to the online control of immediate and delayed grasping: A TMS approach. Neuropsychologia, 47(6), 1553-1562. https://doi.org/10.1016/j.neuropsychologia.2008.12.034.

 

Color theory. (2024, February 16). In Wikipedia. https://en.wikipedia.org/wiki/Color_theory.

 

Dennett, D. C. (1978). Brainstorms. Philosophical Essays on Mind and Psychology (pp. 149-173). Bradford Books.

 

Dennett, D. C. (1994). Real consciousness. In Revonsuo, A., Kamppinen, M. (Eds.), Consciousness in Philosophy and Cognitive Neuroscience, 55-63. Lawrence Erlbaum.

 

Dretske, F. I. (1969). Seeing and Knowing. The University of Chicago Press.

 

Evans, K. K. (2020). The Role of Selective Attention in Cross-modal Interactions between Auditory and Visual Features. Cognition, 196, 104119. https://doi.org/10.1016/j.cognition.2019.104119.

 

Fitts, P. M. (1954). The information capacity of the human motor system in controlling the amplitude of movement. Journal of Experimental Psychology, 47(6), 381-391. https://doi.org/10.1037/h0055392.

 

Fodor, J. A., Pylyshyn, Z. W. (1981). How direct is visual perception?: Some reflections on Gibson’s “ecological approach”. Cognition, 9(2), 139-196. https://doi.org/10.1016/0010-0277(81)90009-3.

 

Gibson, J. J. (1966). The Senses Considered as Perceptual Systems. Houghton Mifflin.

 

Gibson, J. J. (1979). The Ecological Approach to Visual Perception. Psychology Press.

 

Goodale, M. A., Milner, A. D. (1992). Separate visual pathways for perception and action. Trends in Neurosciences, 15(1), 20-25. https://doi.org/10.1016/0166-2236(92)90344-8.

 

Harley, A. (2018). Visibility of System Status (Usability Heuristic #1). Nielsen Norman Group. https://www.nngroup.com/articles/visibility-system-status/.

 

Hodent, C. (2015). Developing UX Practices at Epic Games. Celia Hodent – Brains, UX, and Games! https://celiahodent.com/ux-practices-epic-games/.

 

Hodent, C. (2017). The Gamer’s Brain: How Neuroscience and UX Can Impact Video Game Design. CRC Press.

 

Hodent, C. (2019). Understanding the Success of Fortnite: A UX & Psychology Perspective. Celia Hodent – Brains, UX, and Games! https://celiahodent.com/understanding-the-success-of-fortnite-ux/.

 

Hodent, C. (2020). The Psychology of Video Games. Routledge.

 

lriki, A., Tanaka, M., Iwamura, Y. (1996). Coding of modified body schema during tool use by macaque postcentral neurons. NeuroReport, 7(14), 2325-2330. https://doi.org/10.1097/00001756-199610020-00010.

 

Jack, A. (2018). For many, disability is part of what it means to be human. Special report Modern workplace: Disability. Financial Times. Bracken House. London. https://www.ft.com/content/f4e2dc50-3dab-11e8-bcc8-cebcb81f1f90.

 

Jacob, P., Jeannerod, M. (2003). Ways of Seeing: The Scope and Limits of Visual Cognition. Oxford University Press.

 

Jeannerod, M., Decety, J., Michel, F. (1994). Impairment of grasping movements following a bilateral posterior parietal lesion. Neuropsychologia, 32(4), 369-380. https://doi.org/10.1016/0028-3932(94)90084-1.

 

Johnson, J. (2020). Designing With the Mind in Mind: Simple Guide to Understanding User Interface Design Guidelines. Morgan Kaufmann Pub.

 

Johnston, M. (2004). The Obscure Object of Hallucination. Philosophical Studies: An International Journal for Philosophy in the Analytic Tradition, 120(1/3), 113-183. https://doi.org/10.1023/B:PHIL.0000033753.64202.21.

 

Joyce, A. (2019). 10 Usability Heuristics Applied to Video Games. Nielsen Norman Group. https://www.nngroup.com/articles/usability-heuristics-applied-video-games/.

 

Kaley, A. (2018). Match Between the System and the Real World (Usability Heuristic #2). Nielsen Norman Group. https://www.nngroup.com/articles/match-system-real-world/.

 

Kanizsa, G. (1955). Quasi-Perceptual Margins in Homogeneously Stimulated Fields. In Petry, S., Meyer, G. E. (Eds.), The Perception of Illusory Contours, 40-49. Springer, New York, NY. https://doi.org/10.1007/978-1-4612-4760-9_4.

 

Klevjer, R. (2012). Enter the Avatar: The Phenomenology of Prosthetic Telepresence in Computer Games. In Sageng, J., Fossheim, H., Mandt Larsen, T. (Eds.), The Philosophy of Computer Games. Philosophy of Engineering and Technology, 7, 17-38. Springer, Dordrecht. https://doi.org/10.1007/978-94-007-4249-9_3.

 

Klevjer, K., Günzel, S., Sternagel, J., Mersch, D., (2022). What is the Avatar?: Fiction and Embodiment in Avatar-Based Singleplayer Computer Games. Revised and Commented Edition. transcript publishing.

 

Klevjer, R. (2023). Avatarhood and Selfhood. In Mersch, D., Rey, A., Grunwald, T., Sternagel, J., Kegel, L., Loertscher, M. L. (Eds.), Actor & Avatar. A Scientific and Artistic Catalog, 38-44. transcript publishing. https://doi.org/10.14361/9783839467619-004 (Book: https://doi.org/10.14361/9783839467619).

 

Laubheimer, P. (2015a). Preventing User Errors: Avoiding Unconscious Slips. Nielsen Norman Group. https://www.nngroup.com/articles/slips/.

 

Laubheimer, P. (2015b). Preventing User Errors: Avoiding Conscious Mistakes. Nielsen Norman Group. https://www.nngroup.com/articles/user-mistakes/.

 

Logue, H. (2015). Disjunctivism. In Matthen, M (Ed.), The Oxford Handbook of the Philosophy of Perception, 198-216. Oxford University Press UK.

 

Mack, A., Rock, I. (1998). Inattentional Blindness: Perception Without Attention. In Wright, R. D. (Ed.), Visual attention, 55-76. Oxford University Press.

 

McDowell, J. (1994). The Content of Perceptual Experience. The Philosophical Quarterly (1950-), 44(175), 190-205. https://doi.org/10.2307/2219740.

 

Müller-Lyer, F. C. (1896). Über Kontrast und Konfluxion. (Zweiter Artikel). Zeitschrift für Psychologie und Physiologie der Sinnesorgane, 10, 421-431. http://echo.mpiwg-berlin.mpg.de/MPIWG:0AG8GCV7.

 

Nanay, B. (2015). Perceptual Representation/Perceptual Content. In Matthen, M (Ed.), The Oxford Handbook of the Philosophy of Perception, 153-167. Oxford University Press UK.

 

Nanay, B. (2018). Multimodal mental imagery. Cortex, 105, 125-134. https://doi.org/10.1016/j.cortex.2017.07.006.

 

Nielsen, J. (2006). 6 Ways to Fix a Confused Information Architecture. Nielsen Norman Group. https://www.nngroup.com/articles/fixing-information-architecture/.

 

Nielsen, J. (2024). 10 Usability Heuristics for User Interface Design. Nielsen Norman Group. https://www.nngroup.com/articles/ten-usability-heuristics/.

 

Nielsen, J., Chan, M. (2024). Mental Models. Nielsen Norman Group. https://www.nngroup.com/articles/mental-models/.

 

Noë, A. (2004). Action in Perception. The MIT Press.

 

Norman, D. A. (2008). Signifiers, Not Affordances. ACM Interactions, 15(6), 18-19. https://doi.org/10.1145/1409040.1409044.

 

Norman, D. A. (2013). The Design of Everyday Things: Revised and Expanded Edition. Basic Books.

 

Ore, J. (2017). Mario Kart 8 Deluxe and the spinoff franchise that just keeps rolling. CBC News. https://www.cbc.ca/news/entertainment/nintendo-mario-kart-8-deluxe-1.4075987.

 

O’Regan, J. K., Noë, A. (2001). A sensorimotor account of vision and visual consciousness. Behavioral and Brain Sciences, 24(5), 939-973. https://doi.org/10.1017/S0140525X01000115.

 

Perenin, M. T., Rossetti, Y. (1996). Grasping without form discrimination in a hemianopic field. Neuroreport: An International Journal for the Rapid Communication of Research in Neuroscience, 7(3), 793-797. https://doi.org/10.1097/00001756-199602290-00027.

 

Putnam, H. (1963). Brains and behavior. In Butler, R. J. (Ed.), Analytical Philosophy: Second Series. Blackwell.

 

Ramly, A. A. (2018). Brewing a Meaningful Experience in Coffee Talk. Toge Productions. Medium. https://medium.com/toge-productions/brewing-a-meaningful-experience-in-coffee-talk-c492a24f2c3b.

 

Ramly, A. A. (2019). Enlightening Navigation Design in She and The Light Bearer. Toge Productions. Medium. https://medium.com/toge-productions/enlightening-navigation-design-in-she-and-the-light-bearer-8fe77221092e.

 

Retractable Floor Spike Schematic. (2021, November 15). In Fortnite Wiki. Fandom. https://fortnite.fandom.com/wiki/Retractable_Floor_Spike_Schematic.

 

Rock, I. (1985). The Logic of Perception. Bradford Books.

 

Salen, K., Zimmerman, E. (2003). Rules of Play: Game Design Fundamentals. The MIT Press.

 

Sathian, K., Lacey, S. (2022). Cross-Modal Interactions of the Tactile System. Current Directions in Psychological Science, 31(5), 411-418. https://doi.org/10.1177/09637214221101877.

 

Schell, J. (2008). The Art of Game Design: A Book of Lenses. CRC Press.

 

Sekuler, R., Sekuler, A. B., Lau, R. (1997). Sound alters visual motion perception. Nature, 385, 308. https://doi.org/10.1038/385308a0.

 

Sellars, W. (1956). Empiricism and the Philosophy of Mind. University of Minnesota Press.

 

Simons, D. J., Chabris, C. S. (1999). Gorillas in Our Midst: Sustained Inattentional Blindness for Dynamic Events. Perception, 28(9), 1059-1074. https://doi.org/10.1068/p281059.

 

Singhal, A., Monaco, S., Kaufman, L. D., Culham, J. C. (2013). Human fMRI Reveals That Delayed Action Re-Recruits Visual Perception. PLoS One, 8(9), e73629. https://doi.org/10.1371/journal.pone.0073629.

 

Ungerleider, L. G., Mishkin, M. (1982). Two cortical visual systems. In Ingle, D. J., Goodale, M. A., Mansfield, R. J. W. (Eds.), Analysis of visual behavior, 549-586. The MIT Press.

 

Universal design. (2024, January 30). In Wikipedia. https://en.wikipedia.org/wiki/Universal_design.

 

Van Donkelaar, P. (1999). Pointing movements are affected by size-contrast illusions. Experimental Brain Research, 125(4), 517-520. https://doi.org/10.1007/s002210050710.

 

Verganti, R. (2009). Design-driven innovation: changing the rules of competition by radically innovating what things mean. Harvard Business School Press.

 

Weinschenk, S. (2011). The Secret to Designing an Intuitive UX. UX Magazine. https://uxmag.com/articles/the-secret-to-designing-an-intuitive-user-experience.

 

Weiskrantz, L. (1986). Blindsight: A Case Study and Implications. Oxford University Press.

 

Wright, W. (2015). Nonconceptual Content. In Matthen, M (Ed.), The Oxford Handbook of the Philosophy of Perception, 181-197. Oxford University Press UK.

 

Zipoli Caiani, S., Ferretti, G. (2017). Semantic and pragmatic integration in vision for action. Consciousness and Cognition, 48, 40-54. https://doi.org/10.1016/j.concog.2016.10.009.

 

Zipoli Caiani, S., Ferretti, G. (2018). Vedere e agire. Come occhio e cervello costruiscono il mondo. Il Mulino.

 

Images

Adelson, E. H. (1995). Checkershadow illusion. https://persci.mit.edu/gallery/checkershadow.

 

Hill, W. E. (1915). My wife and my mother-in-law. Puck, 78(2018), 11. https://loc.gov/pictures/resource/ds.00175/.

 

Philips, M. (2018). Exploring the Gestalt Principles of Design. Toptal Designers. https://www.toptal.com/designers/ui/gestalt-principles-of-design.

 

Rabbit-duck illusion (1892). Fliegende Blätter, 2465(145). https://doi.org/10.11588/diglit.2137#0147.

 

Selket (2007). Visual pathways. Wikimedia Commons. https://commons.wikimedia.org/wiki/User:Selket#/media/File:Ventral-dorsal_streams.svg.

 

Videos

Brown, J. (2018). Bridging the Gap Between UX Principles and Game Design. GDC. https://youtu.be/73Pqsk74Jc0.

 

Brown, M. (2015). Why Nathan Drake Doesn’t Need a Compass. Game Maker’s Toolkit. https://www.youtube.com/watch?v=k70_jvVOcG0.

 

Brown, M. (2017). Finding the Fun in FPS Campaigns. Game Maker’s Toolkit. https://www.youtube.com/watch?v=-Bx5t0baXhc.

 

Brown, M. (2018a). Making Games Better for the Deaf and Hard of Hearing | Designing for Disability. Game Maker’s Toolkit. https://www.youtube.com/watch?v=4NGe4dzlukc&list=PLc38fcMFcV_vvWOhMDriBlVocTZ8mKQzR&index=2.

 

Brown, M. (2018b). Making Games Better for Gamers with Colourblindness & Low Vision | Designing for Disability. Game Maker’s Toolkit. https://www.youtube.com/watch?v=xrqdU4cZaLw&list=PLc38fcMFcV_vvWOhMDriBlVocTZ8mKQzR&index=4.

 

Brown, M. (2018c). Making Games Better for Players with Motor Disabilities | Designing for Disability. Game Maker’s Toolkit. https://www.youtube.com/watch?v=Ufe0i26DGiA&list=PLc38fcMFcV_vvWOhMDriBlVocTZ8mKQzR&index=5.

 

Brown, M. (2019). Improving Games for Those with Cognitive Disabilities | Designing for Disability. Game Maker’s Toolkit. https://www.youtube.com/watch?v=ObhvacfIOg0&list=PLc38fcMFcV_vvWOhMDriBlVocTZ8mKQzR&index=6.

 

Brown, M. (2022). How Game Designers Solved These 11 Problems. Game Maker’s Toolkit. https://www.youtube.com/watch?v=rJZyPdYIbZI.

 

Griesemer, J. (2010). Changing the Time Between Shots for the Sniper Rifle from 0.5 to 0.7 Seconds for Halo 3. GDC. https://www.youtube.com/watch?v=8YJ53skc-k4.

 

Noë, A. (2017). Alva Noë – Why a Mind-Body Problem?. Closer To Truth. https://www.youtube.com/watch?v=deIrE6RofEs.

 

Philipp, D., Bullock, B. (2017). Next Level Creature Sound Design. GDC. https://www.youtube.com/watch?v=Rz7WwwlDjOA&t=2079s.

 

Schatz, E. (2018). Ask a Developer: Emilia Schatz Talks Level Design. Critical Path. https://www.youtube.com/watch?v=CRDaXvkge8Y&t=0s.

 

Yano, C., Giovannetti, A. (2019). How Slay the Spire’s Original Interface Almost Killed the Game | War Stories | Ars Technica. Ars Technica. https://www.youtube.com/watch?v=r_BPJzNPF6M.

 

Games

Alien: Isolation (2014). Creative Assembly. Sega.

 

Apex Legends (2019). Respawn Entertainment. Electronic Arts.

 

Call of Duty: World War II (2017). Sledgehammer Games, Raven Software, Infinity Ward, High Moon Studios, Beenox, Toys for Bob, Demonware. Activision.

 

Checkers (circa 1500). Traditional.

 

Chess (circa 850). Traditional.

 

ChromaGun (2015). Pixel Maniacs. Pixel Maniacs.

 

Coffee Talk (2020). Toge Productions. Toge Productions, Chorus Worldwide.

 

Crash Bandicoot (1996). Naughty Dog. Sony Computer Entertainment.

 

Dark Souls (serie) (2011). FromSoftware. Bandai Namco Entertainment, FromSoftware.

 

Fortnite (2017). Epic Games. Epic Games.

 

Gears of War (2006). Epic Games, Psyonix. Microsoft Game Studios.

 

Halo 3 (2007). Bungie. Microsoft Game Studios.

 

Mario Kart 8 Deluxe (2017). Nintendo EAD, Nintendo EPD, Bandai Namco Studios. Nintendo.

 

Rocket League (2015). Psyonix. Psyonix.

 

Scrabble (1948). Alfred Mosher Butts. James Brunot.

 

Serious Sam: The First Encounter (2001). Croteam. Gathering of Developers.

 

She and The Light Bearer (2019). Mojiken Studio, Mojiken, Toge Productions. Toge Productions.

 

Slay the Spire (2019). Mega Crit. Humble Bundle.

 

Tic-tac-toe (circa 100 BC). Traditional.

 

Titanfall 2 (2016). Respawn Entertainment. Electronic Arts.

 

Uncharted 3: Drake’s Deception (2011). Naughty Dog. Sony Computer Entertainment.

 

Uncharted 4: A Thief’s End (2016). Naughty Dog. Sony Computer Entertainment.

 

Undertale (2015). Toby Fox, 8-4. Toby Fox, 8-4.

 

The Witness (2016). Thekla, Inc., Nvidia Lightspeed Studios. Thekla, Inc.