Human-in-the-Loop Behavior Modeling

This project is geared towards developing modeling approaches to data from games that uses a combination of visualization techniques, state-action representations and sequence modeling techniques, such as Dynamic Time Warping. Using these tools we use qualitative analysis and a human expert to allow analysis and labeling. Publications explaining more on the approach is below.

Contributions: granular analysis and modeling of human behavior in games that include contextual details and individual models surpasses current techniques.

Visualization Systems used

Stratmapper

Stratmapper is a spatio-temporal visualization system that displays data from a game and superimposes it on the game map. The visualization system allows users to filter data, annotate or label the data and export this data for further analysis.

Glyph

Glyph is an interactive visualization system with a co-synchronized two window visualization. One window shows state-action transition, showing a descriptive view of the participants or players' problem solving patterns. The other window shows a clustering of players' behaviors. Users can select different nodes from the clusters which would highlight the problem solving patterns in the other window making analysis and comparing and contrasting problems solving patterns an easy process.

Publications

Ahmad, S., Bryant, A., Kleinman, E., Teng, Z., Nguyen, T. H. D., & Seif El-Nasr, M. (2019, October). Modeling Individual and Team Behavior through Spatio-temporal Analysis. In Proceedings of the Annual Symposium on Computer-Human Interaction in Play (pp. 601-612). [PDF]

Kleinman, E., Ahmad, S., Teng, Z., Bryant, A., Nguyen, T., Harteveld, C., & Seif El-Nasr, M. (2020, September). ” And then they died”: Using Action Sequences for Data Driven, Context Aware Gameplay Analysis. In International Conference on the Foundations of Digital Games (pp. 1-12). [PDF]

Nguyen, T., Seif El-Nasr, M. , & Canossa, A. (2015, June). Glyph: Visualization Tool for Understanding Problem Solving Strategies in Puzzle Games. In FDG. [PDF]


Seif El-Nasr, M., Nguyen, T., Canossa, A., Drachen, A. (in press). Game Data Science. University of Oxford Press.

Videos

Microsoft AI and Gaming Research Summit 2021 - Understanding Players (Day 2 Track 1.2)

Team: Sabbir Ahmad, Andy Bryant, Troung-Huy Nguyen, Eric Kleinman, Zhaoqing Teng

Partners: Charles River Analytics.

Funders: Office of Naval Research