Hi, my name is Audrey and I’m an UX designer/researcher on the Méliès is Gone project. I was in charge of setting up playtests for our project.
But what exactly are playtests?
A “playtest” is a development phase where the game is played and evaluated by testers, recruited from outside the development team. These sessions are used to identify and correct gameplay problems, bugs and balancing issues, and to gather feedback on the overall gaming experience.
We had the opportunity to test our prototype in a laboratory of ergo-design.
The prototype consisted of an introductory riddle and a more complex main riddle. The player must discover the elements to be filmed with the cinematograph, based on clues and scenery and then integrate them in the right place in a movie scene which has missing elements.
We were lucky enough to be able to write our protocol with a professional who is used to carrying out user tests.
From the template which he had prepared, we detailed several key aspects :
The device: Our game : “Méliès is gone”
Goals and of tests: Our aim was to validate the general concept of the game, evaluate the gameplay, measure interest for the proposed content and verify the effectiveness of onboarding.
Profile of participants: We targeted five people with a passion for investigative games and/or cinema, to ensure that the feedback would be relevant.
Optimistic assumptions: we made optimistic assumptions that we hoped to validate during testing, especially with regard to the game’s accessibility and the clarity of instructions.
Measurement methods: Because of the nature of the game, which can be solved in different ways and in different orders, we chose to measure the time taken to solve the onboarding and the main riddle as performance indicators.
Observations: We decided not to ask users to verbalize their thinking process. In fact, verbalizing could have disrupted their concentration on the riddle and distorted the results, especially with regard to time resolution metrics. In terms of the guidance we wanted to give playtesters, we chose to help them only if they were not at ease with English or if they couldn’t read certain documents. We didn’t want to help them any more than that because these playtests would allow us to gauge the difficulty of the various puzzles and whether they were feasible.
We asked the user to wear eye-tracking glasses in order to follow their gaze and record their spontaneous reactions.
We developed a pre-test questionnaire, a rainbow spreadsheet, a post-test questionnaire and an interview grid.
This methodical and structured approach enabled us to establish a strong protocol for our user tests, the reliability and relevance of the results obtained.
We wrote an accurate unfolding :
-
- Welcome and information on the testing process (2 min)
- Technical manipulations (3 min)
- Pre-test questionnaire (3 min)
- Set-up and instructions (2 min)
- Manipulations (25 min)
- Post-test questionnaire (3 min)
- Interview (5 min)
Thanks to the tests, we were able to highlight a number of problems that we may or may not have previously identified. Here are a few examples (not exhaustive).
-
-
- We were confronted with some unexpected behavior on the part of certain users during the tests. Although two of them succeeded in solving the two proposed puzzles, none of them fully understood how to solve them. This is interesting, as one might have thought that passing the riddles would mean having understood them, but thanks to the post-test questionnaire and interview, we were able to understand that they succeeded by forcing or by chance.
-
-
-
- We found that the cinematograph instructions were often ignored or misread. They never looked at the inventory part of the instructions, which hampered the progress of the players, who had difficulty finding the inventory of filmed objects.
-
-
-
- Confusion arose during the final puzzle, when some players delayed entering the left-hand room. As a result, they were slow to find most of the clues that were precisely in that room. Although it could be assumed that they had never noticed the adjacent room, eye-tracking analysis revealed that they had indeed examined the door leading to the left. Thanks to the post-test interviews, we understood that this confusion was due to the fact that they thought they were coming from this room.
-
-
-
- No one understood that it was snowing, yet this was essential information, because to solve part of the main puzzle, you had to know whether it was summer or winter. This could be one of the reasons why nobody understood the whole puzzle.
-
-
-
- One thing we knew for sure was that the UI was unsuitable: it made you want to drag filmed objects straight into the real world and blocked access to room changes, but thanks to the test, you can clearly see how annoyed players were with it.
-
To conclude, playtesting is a really important part of game design. It’s a crucial process that enables designers to gather essential feedback from players, detect any problems we hadn’t noticed, and make any necessary improvements. Playtests offer a valuable opportunity to evaluate the user experience, test gameplay, check the clarity of instructions and gauge the game’s level of difficulty. By analyzing players’ reactions and gathering their feedback, designers can adjust and fine-tune every aspect of the game to deliver an optimal gaming experience. In short, playtests are an indispensable tool for guaranteeing quality and player satisfaction, and iterating with them saves precious time and enhances game quality.
A une prochaine fois !