Games based on Markov Decision Processes: several projects

Potential supervisors: 

This master’s project will be part of an ongoing research collaboration with Chalmers, Karolinska Institute, and the National Centre of Competence for Rare Diseases (Ågrenska). The overall goal of the project is to develop games with high pedagogical and recreational value for different patient groups, including patients with intellectual disabilities.

Among the most widely used psychometric tests are Raven’s Progressive Matrices [1]. Each test item consists of 8 figures that are arranged in a 3x3 matrix, with the entry at the bottom right being left empty for the participant to figure out. These tests have several drawbacks: (i) they are not well defined mathematically, (ii) they are adequate for adult humans, but not for children or animals; and (iii) they are not culturally independent, for example, familiarity with the XOR function is helpful [2].

In this project we propose a different kind of psychometric test, which is at the same time a game. Each test item or game is a Markov Decision Process (MDP), which is not visible to the test participant or player. The player can only observe the present state of the MDP and explore the consequences of the available actions. Certain actions in certain states lead to reward and the task is simply to collect as much reward as possible in the given time. Each MDP can be given different “skins”, by means of physical or digital designs of the states, actions, and rewards.

Project suggestion 1

  • Develop the above-mentioned game environment for MDP games
  • Evaluate how the choices of MDPs and skins relate to the performance and user experience
  • Help design psychological tests and analyze the results.

Project suggestion 2

  • Develop a Markov game based on the Ecotwin ecosystem simulator [3]. This includes adding manual control of animals, designing scoring systems, and designing ecosystems (levels) of varying complexity. This leads to a multiplayer game, where the human players take the roles of animals that cooperate or compete, side-by-side with NPCs that are animals controlled by AI.
  • Evaluate this game with respect to performance and user experience.


  • Good programming skills
  • Interest in gaming, interaction design and psychology
  • Experience of game design
  • Experience of Unity is a bonus
  • Projects are suitable for groups of 2 students


[1] Raven, J. (2000). The Raven's progressive matrices: change and stability over culture and time. Cognitive psychology, 41(1), 1-48.

[2] Strannegård, C., Cirillo, S., & Ström, V. (2013). An anthropomorphic method for progressive matrix problems. Cognitive Systems Research, 22, 35-46.

[3] The ecosystem simulator Ecotwin (2021).

Date range: 
September, 2021 to September, 2024