Andrea Middleton 9:46 pm on May 16, 2014
Tags: surveys ( 3 ), WordCamps ( 239 )

One Survey to Rule Them All

In our never-ending quest to make WordCamps easier to organize and better for attendees, I’d like to improve the attendee survey experience. The survey is frequently an after-thought for organizers, and when Polldaddy started requiring a WordPress.comWordPress.com An online implementation of WordPress code that lets you immediately access a new WordPress environment to publish your content. WordPress.com is a private company owned by Automattic that hosts the largest multisite in the world. This is arguably the best place to start blogging if you have never touched WordPress before. https://wordpress.com/ account, things got complicated.

I’ve always been loathe to ask everyone to use the same survey because I don’t like to limit individuality in WordCampWordCamp WordCamps are casual, locally-organized conferences covering everything related to WordPress. They're one of the places where the WordPress community comes together to teach one another what they’ve learned throughout the year and share the joy. Learn more. organizing, but at the same time it would be incredibly useful to have all WordCamps judged on the same scale so we can give organizers reliable data on how their event directly compares to other events. Currently we provide a default survey template to organizers so they can customize it, but teams can write their own as well. My experience is that most teams don’t customize it very much, if at all.

Proposal: Make one centralized survey available on WordCamp CentralWordCamp Central Website for all WordCamp activities globally. https://central.wordcamp.org includes a list of upcoming and past camp with links to each. with a drop down for the WordCamp name and questions that should apply to every WordCamp. All camps can then tweet/email out the same link, making the survey easy to find. WordCamp Central then just exports the results to the organizing team after a period of 2 weeks. For V2, we could even set up an automated system so that CampTix would send an email about the survey to all attendees the day after the event, and maybe even automate a scheduled post about the survey on the blog as well.

Here is an example/proposed survey: http://central.wordcamp.org/wordcamp-attendee-survey/

Please comment with your thoughts on a centralized WordCamp Attendee Survey. If you have suggestions about the survey questions themselves, please also leave those in the comments.

#surveys, #wordcamps

Aaron Hockley 3:19 am on May 18, 2014

Seems like a good idea. As an organizer I know it was always nice to be “done” with the event after everyone had left, and having to manage the followup survey was never an exciting chore.

The other benefit of having a centralized survey is that it might be easier to present last year’s survey results to next year’s organizers without relying on local turnover.
Ian Dunn 4:41 pm on May 19, 2014

One drawback is that it wouldn’t include detailed feedback on individual speakers, but #5 and #6 will still get the highlights from that, and regardless it’ll be better than the current situation, so +1.

The automated e-mail and scheduled post should be pretty easy, so I think that could be V1 if you wanted.
samuelsidler 5:59 pm on May 19, 2014

Definitely better than the current situation, but I think we should have a plan for adding back detailed feedback on individual speakers in the future. Ideally, something like that could be built-in to a WordCampWordCamp WordCamps are casual, locally-organized conferences covering everything related to WordPress. They're one of the places where the WordPress community comes together to teach one another what they’ve learned throughout the year and share the joy. Learn more. app, such that you could leave feedback about a speaker right in the app and right after the talk, before you forget.
David Laietta 6:03 pm on May 19, 2014

Despite fear of over-complication, I agree with Sam. Perhaps this could be built into the WordCampWordCamp WordCamps are casual, locally-organized conferences covering everything related to WordPress. They're one of the places where the WordPress community comes together to teach one another what they’ve learned throughout the year and share the joy. Learn more. site as a tool? We’re already placing speakers on the site, as well as registering users there. Enough information is already available in one place to allow organizers not to have to retype things, and attendees not to have to do extra work around filling out a survey.

While it shouldn’t be an issue, quite honestly the first question of the example survey makes me think that there will be people who forget to put a camp, or put the wrong one down. Tying it to that individual site could make that easier.
- Andrea Middleton 8:14 pm on May 19, 2014
  
  I’d really like to have a way to get feedback on/for each individual speaker also, yes. I think an app would be a great way to do it, as well as on the site. I’ll bring this up on a different thread so we don’t go too far down the app tangent, though. 🙂
  
  I agree that people might choose the wrong city on the drop-down, yes. Currently, I’m trying to weigh the amount of time it would take for Central to add or subtract WordCamps in a certain time period against the threat of someone reviewing Denver as Dayton.
  - samuelsidler 3:23 pm on May 20, 2014
    
    Can we do specific URLs to prevent this? i.e., something like survey.wordcamp.org/?denver will auto-select Denver? Seems like that’d be an easy way to get rid of the drop down entirely, when that variable is set.
David Bisset 6:05 pm on May 19, 2014

I think this would be a great idea for smaller or newer camps, but I would rather see this as an opt-in and not something… well required is the best word I’ve got. Forgive me if that wasn’t the idea.

I’ll state way: larger camps like to ask specific questions (like we are doing now in our survey as we send it off to our attendees). Organizers like to be in control of the surveys on that level – or at least the organizers of WordCampWordCamp WordCamps are casual, locally-organized conferences covering everything related to WordPress. They're one of the places where the WordPress community comes together to teach one another what they’ve learned throughout the year and share the joy. Learn more. Miami. I think the survey you provide Andrea is a great step and if you haven’t been applauded of it before, i’m doing it now! 🙂

The statement “useful to have all WordCamps judged on the same scale” doesn’t sit well with me. I try not to compare WCMIA to other camps because it truly is unique, or at least what i’ve been told. I’m sure on some levels there are simulatories that could be applied, no doubt about that.

Long story short: Make it’s opt-in if organizers want to use it. Sounds like a great tool for some, especially those who don’t need or have time for a customized survey.
- Andrea Middleton 6:27 pm on May 19, 2014
  
  I think my mention of judging WordCamps on the same scale was ill-phrased. 🙂 What I meant to say was that being able to share with WordCampWordCamp WordCamps are casual, locally-organized conferences covering everything related to WordPress. They're one of the places where the WordPress community comes together to teach one another what they’ve learned throughout the year and share the joy. Learn more. organizing teams the kinds of answers they get on their surveys compared to other WordCamps that host similar numbers of people or are held in similar regions might be interesting for all. My goal is to share successes rather than highlight challenges. 🙂
  
  For WordCamp Miami, for instance, I’m positive we’d see a lot of praise about how that event promotes personal connections that would be instructive/intriguing/motivating for other WordCamps of a similar size. I think each WordCamp is unique, and don’t want that to ever change. However, if we can all learn from each other’s successes and hiccups, organizing gets easier and attendees benefit across the board.
  
  I think the same effect would be achieved if the survey results were published to each WordCamp’s website, but when people aren’t aware that their responses will be published, they’re frequently harsher than they might otherwise be. I’ve not been very successful getting camps to be transparent with their survey results in the past years. 🙂
  
  On the individualized surveys, that’s definitely a thing. Opt-in to the centralized survey could work, or we could add branching to the centralized survey and let people add event-specific questions if they wanted to. That way we’d get the same comparable data but also be able to gather event-specific data. I’d actually quite like to do that for those events that are trying new things.
  
  Thanks for the pat on the back for the survey template; it’s always nice to hear whether or not something is helpful. 🙂
  - David Bisset 10:54 pm on May 20, 2014
    Completely understand on “judging WordCamps on same scale”.
    
    So from the second paragraph, if i read between the lines, are you concerned that WordCamps aren’t sharing the complete (negative especially) feedback? It seems to me that having that information would be a benefit or result of sharing and WordCamp CentralWordCamp Central Website for all WordCamp activities globally. https://central.wordcamp.org includes a list of upcoming and past camp with links to each. hosting a common form.
    
    While i believe in being open and HEAVILY support the spirit of cooperation, I personally wouldn’t be open to having survey results on the WordCampWordCamp WordCamps are casual, locally-organized conferences covering everything related to WordPress. They're one of the places where the WordPress community comes together to teach one another what they’ve learned throughout the year and share the joy. Learn more. website. Some negative results spurred upon by events at the conference might be taken out of context. Some organizers might be more paranoid to do cover things because they fear the public survey results. On some level you can count me in that group. However, I firmly believe organizers should share their experiences with surveys like I did last year (http://www.davidbisset.com/wordcamp-miami-2013-feedback/).
    
    As you said, this is about data. The same way WordCamp Central no doubt examines the WordCamp websites to determine what tickets they sold, in a similar fashion the post-even survey data would be a nice to have for WordCamp Central.
    
    Summary:
    
    I don’t believe survey results for a WordCamp should be shared publicly unless the conference organizers agree. If the raw numbers aren’t displayed, some form of “post mordem” should come from the organizers so others can learn – we should make that alot easier to do IMO.
    
    Maybe there’s a way to export the raw data that WordCamp Central wants from those camps who choose to create their own surveys through PollDaddy. Depends on how the data is being collected.
    
    I’m still up for providing a generic survey for those camps that don’t have the time, resources, etc. to create something custom. This could be the default and what’s been provided already is awesome. Did i say that already? 🙂 AWESOME.
    
    I would love to hear from other camps (especially larger ones) how they do surveys in the first place and what they do AFTER the results are in. Haven’t heard too much of that. It would make a good forum discussion. 🙂
    
    For many organizers, WordCamp surveys are as much of a reflection on them as they are on the event itself. I would imagine some would treat these personally.
Siobhan 7:56 pm on May 19, 2014

I like the idea of having just one survey. I agree that the survey is often an afterthought and having a central one would mean one less thing to take care of.
Tony Scott 5:26 am on May 21, 2014

I agree with David Bisset’s comments.
Karissa 12:00 am on July 20, 2014

As an organizer for WordCampKC, I like this idea but would like to be able to add custom ones that may be specific to us. I agree that it still would be good for the WC Foundation to see all the data too.

What about Parent/Child Polldaddy accounts for those that want to customize? Where you can clone the master and add the unique items?

I would like to be able to login to review the data vs just the export as it presents it in a different way that is more readable.
Taylor Dewey 6:17 am on July 31, 2014
Q3’s likert scale is a bit confusing:

Specifically, what’s the difference between “Completely true” and “Very true”. The statements for the scale should be equally spaced. Our community already trends toward the positive, we don’t need to reinforce this with how we structure the surveys.

I would recommend sticking to a standard five point scale and omitting the “N/A”
- Strongly disagree
- Disagree
- Neither agree nor disagree
- Agree
- Strongly agree
Respondents who would normally submit “N/A” cleanly fit into the “neither agree nor disagree” response.

Weighting the questions equal distance in this way allows us to score the ratings using integers where 0 is “neither agree nor disagree”. “Strongly agree” and “Strongly disagree” would be scored as -2, or 2 and “Agree” and “Disagree” are scored -1 or 1 depending on the format of the question (e.g. is agree “good” or is disagree “good”).

Assigning responses to integers allows us to easily compare questions and aggregate survey results. If the entire survey can be scored and distilled into an aggregate number it gives wordcampWordCamp WordCamps are casual, locally-organized conferences covering everything related to WordPress. They're one of the places where the WordPress community comes together to teach one another what they’ve learned throughout the year and share the joy. Learn more. organizers a measurable result. Statistics can be performed on individual questions. Generally the data becomes a lot more valuable. As it stands now, the responses, from Q3 at least, would be hard to decipher in any measurable way.

-Taylor
- Andrea Middleton 8:24 pm on August 1, 2014
  
  Q3’s likert scale is a bit confusing
  
  That’s a good point, Taylor! Thanks for your suggestions; I’ll edit Q3. 🙂
Amanda Giles 8:47 pm on September 28, 2015

I just took this survey and it was great and easy to take. The one piece of feedback I have about it though is that questions 3 and 4 both have scales (high to low), but they go in opposite directions. So Q. 3 feedback goes from “Strongly Disagree” on the left to “Strongly Agree” on the right. However question 4 goes from “Extremely Valuable” to “Not at all Valuable”.
Pingback: WordCamp Central Proposes Centralizing Post WordCamp Surveys - PressSearch

Welcome!

Get Involved

Let’s Talk

Learn More

One Survey to Rule Them All

Welcome!

Get Involved

Let’s Talk

Learn More

Share this: