WordCamp San Francisco, @johnbillion and I conducted some user tests. We focused on three feature plugins planned for 4.1: Focus, Author Select, Session Manager. When possible, we created an editor account for users and had them participate on there own device. When they didn’t have a device, they participated on one of our laptops.
Below are the notes.
Dan – developer. using a macbook
“Wow” was first reaction to focus. Kind of felt like it was shocking and he needed some frame of reference. Maybe keep 5px or so of menu so you know you can go back to it. Fading of buttons kind of make it feel like they are disabled.
Author select – Immediately knew what to do. super natural
“Where are you logged in” – knew to go to profile page, found the panel right away.
Michelle – designer/developer. using a macbook
It’s disconcerting not to see all the things. “If the point of things is to be distraction free, it’s weird not to see publish since that is what people want to do”. Fading of buttons seems a bit weird. Not sure she like automatically going into distraction free. It’s jarring.
Author select -didn’t search, just selected.
Janneke – Developer. using an iPad mini
Author select on iPad mini – Super hard to use. Can’t really scroll down. Not at all intuitive.
Profile manager – Looks horrible on iPad mini
Focus – She is building it, so I didn’t test it
Josh – Developer. using a macbook
Likes not having the sidebar menu. Losing the meta boxes seems odd. Might be weird for some people.
Author select. Select 2. Using display name instead of user name. Seems odd.
First looked to tools to find where he was logged in.
Joe – Developer. using a macbook
Lag on the editor screen makes things feel slow and confusing.
Author – Not used to to having it enabled by default is different. Seems to be able to use
Cory – WP business owner. using his macbook
Loves the focus. Seems super natural.
Author – Very usable.
Profile – Didn’t quite understand question at first, but once he did, made it and found that info right away.
Mel – designer. using our laptop
Focus – It’s weird that it’s not centered. Less things on the page makes it harder to focus.
Author – “I heart Select2″
Jen – Freelance writer and photographer. Has her own blog, using a MacBook.
Initially wasn’t enthusiastic about the fact that the meta boxes and menu disappeared, “oh, what’s going on?”. Was confused about how to get them to re-appear, erroneously clicked an unrelated button in the browser in an attempt to get the UI to reappear. After figuring out that the UI re-appears when you move your mouse or tab away, actually very much liked the behaviour.
I mentioned that we were considering adding a feature pointer pointing to the DFW button. Jen said she thinks this will be important. User has no other idea of what’s going on otherwise.
Couldn’t find where the user sessions may be listed. Mentioned that she is the only user on her site so she would have no interest in it. Expected it to be on the dashboard, or on the same screen as Jetpack analytics.
Christen – blogger
“Create a new post” – First went to quick draft. When I sent her to the regular draft screen, it seemed natural to her. She liked the “Distraction free writing”. It felt natural to her. Once it was in place though, she didn’t really look for anything else on the page.
Andre – Theme Builder. Design and build.
Focus Was natural. Really liked and it seemed 100% natural to him.
Author – Select two was a natural choice and he really likes it.
Went to tools page to find sessions.
Matt – developer and project manager
Not enthusiastic about using the editor focus improvements himself, but said he would leave it enabled for his clients who write more long form content than he does. Suggested that some form of A/B testing may be desirable if we have the time. No other feedback.
User session list will be of interest to him as a project/site manager, not so much use to his clients themselves. Suggested that we could implement a dashboard widget which displayed the “last accessed” time for the current user, or listed other current sessions if there is more than one. Also suggested adding an “active sessions” column to the user list screen, visible to site admins.