Good Old Research

This week, Rae and I revisit our previous ideas about combining our prototypes and determine a new approach. Additionally, I describe my findings from doing some reviewing of the literature in Social Systems, Experimental Design, and Scale Validation.

Research Through Design

This week, Rae and I began bringing together our two projects into the environment I had made the prior week. However, this quickly revealed several of the design problems we had yet to tackle in our project including:

1. How many social interactions would there actually need to be?

2. Who are the central characters of our combined project?

3. When would entities spread gossip?

4. How could Rae author gossip events as a narrative designer?

5. What is the timeline for our implementation?

Thus, knowing what we now knew thanks to our fail fast approach, we went to the drawing board to come up with a solution that would address each of these issues. In the end our solutions were as follows

1. We would separate our project into three phases of implementation: 

   1. The Introduction / Gathering Scene

   2. The breaking off to separate rooms Scene

   3. The Final Gathering Scene

2. For the Introduction Scene we would limit the scope to only 4 NPC agents and the NPC companion character

3. Creation of a Time System

   1. The scene would be represented as a series of time segments

   2. During the party, each time the player interacts with a guest, they would advance the party clock forwards to the next time segment

   3. For every time segment, each NPC agent would have a set position in the room they would walk to representing the conversation they are apart of

   4. If two NPC’s are in the same conversation group, they will gossip amongst themselves sharing the info they have obtained so far

4. Custom Arcweave Gossip Attribute

   1. Ability to mark Arcweave nodes as gossip spreaders

   2. Ability to mark Arcweave branches as requiring gossip for execution

5. The creation of a project priority task list for managing project development while maintaining flexibility

   
   

Based on our discussions, I then went through and created the custom Arcweave gossip attribute system and schedules for the test. For this initial phase, I have simplified the transitions to a teleport during a fade to black sequence between each time segment. In future prototypes, I may look into creating a simple animation showing each of the NPCs moving to a different location. 

In addition, I also dedicated time this week to begin conducting a deeper dive into similar projects with my focus being on three different social systems; 

• Prom Week (CiF) by McCoy et al (2011)

  • A social simulation game in which the player is tasked to complete a series of “social puzzles” in order to progress through set storylines

  • The “social puzzles” used a new type of gameplay mechanic referred to as “social physics” which would allow for tracking chain reactions from social interactions 

  • The player would be able to control any character to initiate a social event and then the AI would calculate the resulting social consequences based off the predefined social physics

  • Players would have a limited number of social events they could execute for each level

• Talk of the Town by Ryan and Mateas (2019)

  • A social simulation game in which two players face off through their interactions with the people in the town

  • One player would need to use the games ability to spread misinformation in order to hide their identity from their opponent

  • The other player would need to leverage their social connections to try and follow the chains of information back to their opponent in order to determine their identity

  • The town would be made up of hundreds of NPCs all of which are simulating their social citations amongst each other ranging anywhere from romantic relationships to self-owned businesses.

  • Each NPC would be able to have their own internal memory of what they know with that information being mutable allowing for it to change over time due to misremember or forgetting as well as misshared due to lying or misconstruing

• Neighborly by Johnson-Bey (2022)

  • A “Rational Reconstruction” of Talk of the Town

  • Essentially a reimplementation of Talk of the Town that allows for other developers to implement their own versions of the solution without needing extensive rewrites due to the hard coding of several systems with other systems and the plot

  • Provides a breakdown of what separate Talk of the Town from other social engines and an overview of a variety of different engines already developed

  • Provides an overview of how to use the Neighborly system

Overall, after reading these solutions, I feel more confident in the uniqueness of what Rae and I are working on since we are trying to create a low weight version of these two different social systems. Both social systems require an AI backend that can take up to a minute at times to compute results which hurts a game's ability to feel reactive. Additionally, Talk of the Town / Neighborly focuses on large-scale society simulation while Prom Week is for smaller groups of NPCs but still relies on shared knowledge systems between NPCs. Additionally, both systems utilize a large amount of emergent storytelling resulting in greater difficulty to tell a more directed story like what we are trying to do. 

Brainstorming Session Images

Independent Studies

Gaydar

For this week, I continued to study how to design an experimental study using a control group and a treatment group. So far, I’ve been able to learn about internal validity issues that, while on the surface seem obvious, did not initially occur to me.

These include:

1. That it is not truly a control group with no X, since the control group will be doing something while X is being given the treatment. For example, in our game, we will be testing the impacts of trans content on queer views against gaydar which will also be including other queer content. So as a result we will not be comparing it against if the game was never played but rather if they had played another queer game.

2. Intrasession History: If we do only one session to test the control group and then another session to test the treatment group, then the session itself could be a factor in the results. Similarly, if we test all the control groups in multiple sessions first then the experimental group, the time could be a factor in the results. The suggested approach we will need to be aiming for is to either randomly have each participant be scheduled or to have pairs be tested together, or have them all be tested together. However, we would also need to have them in the same room for joint testing or else the moderator could be a factor

3. Morality: If people are dropping out of the treatment group but not the controlled group, and those who stay are the ones who are compared then morality could be a factor in the results. The recommendation for this is to use a design 4 strategy of having a pretest to check for this possibility and report the results.

With these points in mind, I will begin looking next week into the threats towards external validity and hopefully beginning to design the study.

GDC Talks

For this week, I went through and watched a talk called “'Disco Elysium': Meaningless Choices and Impractical Advice” given by Justin Keenan. In his talk, he breaks down Disco Elysium’s dialogue system and approach to dialogue. The main key points he focuses on are having micro-reactivity, a rhetorical approach, minimal gamification, and radical asymmetry. I would say the most relevant aspects for my own research would be his advice on micro-reactivity and minimal gamification. Essentially, he describes the importance of having even small choices having impact which could be a good way to view the benefit of a gossip system. Additionally, for minimal gamification, he brings up the good point of trying to make dialogue scenes feel like conversations and less like games or else players will engage with dialogue that way. This reminds me of how Prom Week when playing can feel like you're working on a puzzle rather than actually watching a scene play through due to how much gamification they incorporate. 

Moving forwards, based off this talk I will be considering the following questions:

1. How can we create social systems that allow for micro-reactivity?

2. How can we minimize the amount of gamification used in our systems while still making it clear how the system works?

Believability

For this week, I began reading several papers on past scales that have been developed and spoken with several professors on the matter. My main goal has been to learn how to go about showing a scale is both reliable and valid. Through my research so far, I’ve been informed I will need to show both convergent and divergent validity. Essentially, I want to show that my scale yields similar results to scales that in theory should be similar such as those used for Narrative Transportation. Additionally, I want to show that my scale is different from those that are different. For example, the scale should show a relationship with narrative transportation but not with say action coupling. Additionally, I have begun looking into a validity testing method called confirmatory factor analysis which confirms that each of my categories are actually measuring the same elements of believability. 

Additionally, I went through and created a visual to show my current understanding of believability which can be seen below.

Believability Tree

The Wrap Up

Overall, the main focus of this week ended up being on the advancement of my research through design projects.

Work Cited

Campbell, D. T., & Stanley, J. C. (2011). Experimental and quasi-experimental designs for research. Wadsworth.

Johnson-Bey, S., Nelson, M. J., & Mateas, M. (2022). Neighborly: A Sandbox for Simulation-based Emergent Narrative. 2022 IEEE Conference on Games (CoG), 425–432. https://doi.org/10.1109/CoG51982.2022.9893631

Levine, T. R. (2005). Confirmatory Factor Analysis and Scale Validation in Communication Research. Communication Research Reports, 22(4), 335–338. https://doi.org/10.1080/00036810500317730

McCoy, J., Treanor, M., Samuel, B., Mateas, M., & Wardrip-Fruin, N. (2011). Prom Week: Social physics as gameplay. Proceedings of the 6th International Conference on Foundations of Digital Games, 319–321. https://doi.org/10.1145/2159365.2159425

Ryan, J., & Mateas, M. (2019). Simulating Character Knowledge Phenomena in Talk of the Town. In S. Rabin (Ed.), Game AI Pro 360 (1st ed., pp. 135–150). CRC Press. https://doi.org/10.1201/9780429055119-13