Via the Wowhead liveblog of the Q&A:
You’ve said Fun is the ultimate metric that drives the game forward. How do you measure fun?
If you listen to the Activision Earning Call and you hear MAUs, that’s Investor speak and not what the developers use. All they want to do is as engaging as possible. If there’s something that is over rewarding or has a risk/reward ratio that is out of wack, then they see participation rewards (e.g. The Mechanar Problem: In BC, it was the fastest, easiest dungeon).
Some other metrics are:
* How frequently do you log in?
* How long do you log in for?
* What activities do people do?
I think this needs to be said. The “other metrics” listed here are not all that different from the MAU measurement philosophy, and worse, they’re still bad metrics.
Let me explain:
How frequently do you log in?
This metric is a problem because WoW is designed to encourage people to log in even if they’re not having fun, in order to stay “current” or to not miss out on chances at things they’re interested in obtaining. Examples: PVP/M+ chests, Darkmoon Faire profession bonuses, emissary completion bonuses that disappear every 3 days, and the weekly Island Expedition AP bonuses.
People do lots of these things, not always because they’re fun, but also because they lead to rewards they want or feel obligated to pursue. It’s all about the carrot and nothing about the stick.
How long do you log in for?
This metric is also a problem because, again, it doesn’t take into consideration how much fun you are having. Back when I actually played this game, I would log in daily, do stuff for AP, set up my mission table, down world bosses for loot chances, do stuff for emissary chests, and then go, “okay, time to do it on my alt.”
After that was done, I would sit back and go, “finally, now I can play the game for fun.” At this point I’ve played for 45-60 minutes already, and all I’ve done are chores.
What activities do people do?
Again, this is a problem, because it ignores that some people are only participating in content for the rewards, and not because they find the base content entertaining.
For example, Warfronts and Island Expeditions are not entertaining game modes, but when I was playing, I still did them, because they provided me with substantial character stat bonuses that I felt obligated to pursue.
As another example, during BFA S1 I arena’d and RBG’d my mug off so I could get the S1 PVP mounts (thought the hippo was cool) and the “Veteran of the Horde” title. But most of that was a frustrating, not-fun experience. My main focus was doing just enough of the content so I could get the rewards I wanted. It was all about the carrot, and nothing about the stick. I stopped as soon as I got what I wanted.
This is the problem with all the metrics you mentioned. Some (possibly many) people are consuming your content just so they can say, “whew, thank God I never have to do that again!” And by looking at simple factors like how long they are playing and what they are doing, you are getting a distorted view that these are successful gameplay modes and features, when in actuality they are just frustrating obstacles between the players and what the players want.
TLDR you need better metrics. Don’t focus on what people are doing. Focus on what people do after they fulfill their progression content requirements.
Do people do more than the “required” Island Expeditions after they get the AP bump? What about after they collect a majority of the toys?
Do people continue to do Warfronts after the gear is irrelevant or they have collected all the transmogs? Or do they stop?
When players hit approach the ivl cap for the patch, do they continue to push M+ keys, or do they stop?
Make metrics meaningful again.
Thanks.