Game analyst tools to prevent and solve a crisis
How can you as a game analyst help prevent a crisis? As we mentioned in the first part of this series (The Game Analyst Guide to Deal With a Crisis), it’s normal for your game/product to go through some crisis situations from time to time. As a game analyst, diagnosis and damage control are part of your job. So, once again, grab your detective cap, and let’s explore methods to address this challenge.
Diagnosis: separating signal from noise
Monitoring tools are the best tools you can provide your teams (and yourself) with to have a good grip on the health of your game/product.
Most dashboard/visualization solutions have alert systems that you can use for this effect.
You can train sophisticated models to help you monitor. In this case, I’m very partial to using signal processing principles for real-time anomaly detection and trying to keep it as simple and self-explanatory as possible. The last thing you want is to spend hours trying to understand and explain a false positive.
Documenting events that can potentially impact KPIs
This is self-explanatory but having a single source where you can check all events that happened on a specific date can help you understand some consequences and help your team course-correct before you’re faced with an all-hands-on-deck crisis.
Interestingly enough, less experienced product teams often have a big resistance to contributing and documenting these live-ops events. My theory for this behavior is that there’s a temptation to think that release notes are synonymous with these live-ops events. However, great world-class live-ops teams understand that there are multiple factors that can move KPIs.
Some examples of what we might want to document on a live-ops events list are:
- Dates and content of releases.
- Dates and contents of patches/hotfixes
- Date/time when a noteworthy bug is discovered
- Marketing events
- Viral events such as mentions on a popular website/youtube video/social media and app store featuring.
- Any sort of live configuration changes (these often go untracked and can be the root cause of many crises), including AB test and other player segmentation/personalization.
- Past crises (it’s easy to forget the exact day we had a crisis 6 months after the fact)
One of the best ways I’ve seen this Live-Ops Diary concept implemented is using a live-events slack channel. All team members are encouraged to post changes (accompanied by extra information whenever applicable).
Pre-Flight/Normal Checklist for a Game analyst
As I mentioned in the first part of this series (The Game Analyst Guide to Deal With a Crisis), a pro tip to deal with a crisis (especially if you have some trouble remaining calm when you’re getting 20 different people asking you what’s going on and 10 other people asking you unrelated requests), is to have a checklist. This idea is borrowed from an aviation pilot technique, the normal checklist, where you have a checklist to make sure you cover all bases when you’re going through a certain procedure.
It’s normal that you might not want to see any of the work that you created under stress and label most of your work under ad-hoc analysis destined to be forgotten.
However, there’s a big potential to repurpose that content, even it isn’t the best/most efficient content you’ve ever created. You can create quick private dashboards/code snippets with your considered segmentations to be reused for the next crisis. You’ll thank your future self.
So far, we’ve covered extensively some of the work that game analysts can expect during a product crisis, as well as some tools to prepare and improve response time.
By this time, we’ve been able to help our teams solve the issues that were affecting the product. We’re starting to see the first signs that we’ve course-corrected. However, we shouldn’t forget that our job isn’t over. Check the final part of this series: Game analyst guide to deal with the aftermath of a crisis.