Microsoft
Study Background
Businesses struggle to close the loop from insights to timely, targeted actions, especially across thousands or millions of business objects. There is a high dependency on individual developers and these experts resort to manual, repetitive, reactive actions. Businesses are often looking for patterns over time but it is difficult to iterate rapidly and share what they learn as well as make it fast and scale. The Data Activator is a novel product meant to be the “action” leg of the Trident ecosystem. It will drive timely decisions from data by empowering business domain experts to determine critical trigger points at which they will be notified to take
action.
Objectives
The main objective of this usability study was to learn about the target customer and how they presently close the “signal, condition, action” loop, how Reflex facilitate their processes, what their perceptions and behaviors are while using Reflex and what measures may be taken to make Reflex more usable.
Key Research Questions
- Can users easily accomplish core tasks using the experience?
- What are the most pertinent challenges to completing selected tasks? Why?
- What features or capabilities would make the design more intuitive and effective?
- What are users’ understand of product goals and terminology?
-
What potential use cases can participants identify related to data monitoring and notification?
Main Takeaways
Users in the study explicitly requested various features to improve usability:
It was observed that the majority of users in the study expressed that the new interface was an improvement and relatively intuitive.
7 out of 8 users in the study ranked Data Activator as either mostly useful or very useful and the average net promoter score was 50 out of 100.
It was observed that discoverability of the create trigger from the column header in data mode had low task level satisfaction and highest error rate (average of 2.29 errors across participants) during the cognitive walk-through.
There was an average of 5.14 errors across all tasks. It was observed that errors were mostly concentrated in create trigger and edit trigger tasks. All other tasks had average error rates closer to zero.
It was observed that edit trigger also had relatively low task level satisfaction compared to other tasks but there was a consensus that it was learnable once discovered. Some users expressed desire to add a second trigger rather than…
It is recommended that an “edit trigger” button is added and/or a draft mode
When asked how satisfied they were overall with the product, 88% indicated they were either satisfied or very satisfied. 12% were neutral on satisfaction.
It was observed that total task times ranged from 10 to 30 minutes (average time 20.06 minutes).
It is recommended that the edit trigger feature is made more intuitive with “edit trigger” button and more on screen instructions throughout the product (particularly for create and edit trigger).
Users in the study explicitly requested various features to improve usability:
Compound conditions Customized alerts Draft mode
On-screen instructions Edit trigger button In-depth analyses
It was observed that the majority of users in the study expressed that the new interface was an improvement and relatively intuitive.
Survey results indicate that 6 out of 8 individuals ranked the interface as easy or very easy to use. The remaining 2 individuals felt neutral on the ease of use. The new design was perceived as familiar by several users with experience in PowerBI. Those who had used previous versions of Data Activator prefer the current version for its more streamlined flow.
Illustrative quote: “The navigation part itself, I think it was intuitive. I could figure it out just by scanning the monitor where I need to go and what I need to click on. I guess it follows the same pattern as PowerBI.” – Analyst, Hitachi Solutions