Surveys
The Office for Institutional Effectiveness strives to expand the capacity for assessment at Clemson to enrich data-driven continuous improvement and innovations. Towards this goal, OIE provides resources and services for creating and using surveys and other indirect measures.
Survey Software
All Clemson faculty, staff, and students have access to Qualtrics online survey software for university-related work. Employees can also use Anthology Baseline for conducting surveys. We recommend that you use Clemson's Qualtrics enterprise license for most surveys. Qualtrics is a feature-rich, functional survey software solution and has met the requirements for Clemson's IT Solutions Registry. Please see our Qualtrics Terms of Use.
Use of Google Forms and Microsoft Forms is discouraged for confidential or restricted data. When working in with external survey vendors, keep in mind that directory information (such as names and email addresses) should not be transmitted without satisfying the stipulations in the South Carolina Family Privacy Protection Act (FPPA).
Please see the links in the sidebar for more useful information on how to design, implement, and execute your survey. If you would like to distribute a survey at Clemson, need help developing your survey in Qualtrics or Baseline, or have any other survey-related question, please use the links below or email surveys@clemson.edu.
Create a Data Management Plan for Surveys
Ask a Question About Surveys or Related Issue
Design and Distribution
Indirect measures such as surveys can be useful tools for measuring agreement, gauging satisfaction, and identifying population demographics. Indirect measures should be written, administered, and analyzed with thoughtful planning. The Office for Institutional Assessment aids programs in the design, distribution, and analysis of surveys and focus groups.
Tips for Writing SurveysSurvey Development Guide (PDF)
Sample Sizes and Sampling (PDF)
Survey Terminology
Below is a list of common survey terms and how they are used in our office.
Please note that terminology has been simplified and may vary from other sources.
-
Target Population
The group of people you wish to study with your survey. Example of the target populations would be all undergraduate students at Clemson or all faculty in the College of Business.
-
Sampling Frame
The list of people from which your sample will be drawn. A sample frame largely overlaps with the target population. By default, our office excludes the following students from the student sample frame: 1) students under the age of 18 and 2) students who have asked for their directory information to remain confidential.
-
Sample
The term sample is used in two ways, which can cause confusion.
- During the survey design and data collection phases, the sample of potential respondents is the list of people who are invited to take the survey. Samples are randomly drawn from the sample frame and designed to be representative of the desired population.
- Once the survey data is being analyzed, sample more commonly refers to the collection of valid respondents and their responses (also called observations). When making statistical inferences, be sure to use the characteristics (such as the size and representativeness) of the actual respondents. See sample size below.
-
Sample Size
The number of valid survey responses obtained or needed. Sample size is important when determining whether how well you can generalize your survey results to the larger population. See the example below for more information.
-
Convenience Sample
A sample which is used because it is easy to access, but is not representative of the population being studied. If the target population for your study is college students but you only survey students at Clemson, then your sample is a convenience sample because Clemson students are not representative of all college students. Convenience samples are best used for pilot studies.
-
Census
A survey of an entire population, with the aim of collecting data from every single individual of the population. Surveys of entire populations at Clemson are rare and only conducted with the approval of the Executive Leadership Team (ELT). Sometimes called a population survey; however a true census will include multiple modes of data collection to achieve as close to a 100% response rate as possible.
-
Response Rate
The percentage of responses collected out of all survey invitations. The average response rate for student surveys without incentives is between 10-20%. If your response rate is low, think about how to motivate survey invitees to take your survey through incentives or improved communication.
-
Completion Rate
The percentage of survey responses that are fully completed. If the completion rate is low, the survey may be confusing or too long. Completion rates of 100% are possible with good survey design.
-
Survey Bias
Bias refers to choices made in the survey design or data collection which result in systemic errors that distort the responses. There are many sources of bias in surveys which can occur at all phases of planning and executing a survey. In particularly, the difference in characteristics between your initial intended sample and those who responded to your survey may result in non-response bias.
-
Incentives
Incentives refers to tangible rewards (usually gift cards) are offered to survey respondents as a reward for completing a survey. Incentives can be given to every participant or raffle-based. See the Inventive Cards Procedure for more information on how to purchase incentive cards through Procurement for use with your survey.
Example
-
Survey Example
Let's assume you want to study the full-time, undergraduate students at Clemson. You know from the Interactive Factbook that the full-time, undergraduate student population in Fall 2023 is 22,144 students. After submitting your survey request and receiving your survey sample, you send out 3000 survey invitations. Of those, 450 students open the survey link and begin to take the survey, but only 360 respondents complete the survey.
Your target population is comprised of all 22,144 currently enrolled, full-time undergraduate students at Clemson.
Your sampling frame is a subset of the target population which excludes those students who are under the age of 18 or have requested confidentiality of their directory information. In this example, the sample frame consists of 21,890 students, or 99% of the target population.
Your survey sample is a list of 3000 students randomly drawn from the sampling frame. This random sample is representative of your target population.
If 450 students started the survey, then your response rate is 15%:
450 survey responses / 3000 survey invitations = 0.15 or 15%
But if only 300 survey responses are completed fully, then your completion rate is 80%:
360 completed surveys / 450 started surveys = 0.80 or 80%
It is important to note that when you perform your statistical calculations, your sample size is 300 (the number of complete responses) and not 3000 (the number of surveys you sent out). Furthermore, while your initial sample was represent of the target population, your response sample may not be. Because of your low response rate, there may be non-response bias if one segment of the population is more likely respond.
For more discussion on sample size and statistical significance, please see Penn State's open education resource on Statistical Concepts and Reasoning.
To determine if your sample was large enough, you can use the Qualtrics Sample Size Calculator. You must first determine what confidence level and margin of error is acceptable for your study. A confidence levels of 95% is typical for research, but a confidence level of 90% may be acceptable for operational surveys.
Based on the example above, if your goal was to have a margin of error of 5% and a confidence level of 95% for a population of 22,144, you would have needed 378 responses. However, you can see from the Qualtrics Margin of Error Calculator that your margin of error is only slightly higher at 5.6%.
