UNIVERSITY OF CALIFORNIA, SANTA CRUZ; Tonay; Psych 80b; Winter 2005
Step 1: Choose a topic
--Is it best measured with a survey? (see text chapter 2)
--Define what you are measuring. Make sure your definition is
clear.
--Define your population: Whom are you studying? Make sure to
consider diversity variables, such as age, geographical location,
religion, socioeconomic status, gender, sex, ethnicity, educational
level, and so on.
Step 2: Generate your items. You'll need an item pool. Items must be logically related to what you are studying. Try to write items which address several of these item levels:
1) overt behavior (actions)-- e.g., 'I have had sex with more than three people in the last 3 months.'
2) covert behavior (feelings)--e.g., 'I believe I daydream about sex more frequently than most people.'
3) physical reactions (symptoms)--e.g., 'I have never gotten sexually aroused in public.'
Under those three main categories are several subcategories. You might include items from several of these item types:
1) personal characteristics (what does subject think s/he does or is or believes?--see #2);
2) wishes/interests ("I would rather do x than y");
3) biographical facts (possible past experiences--"I have had a homosexual experience.");
4) attitudes & beliefs ("I think sex is healthy");
5) peer evaluations ("Others tell me I'm...");
6) symptoms ("Saliva terrifies me").
Think about using a mixture of true/false, likert rating scale (1-7 with 1 = 'not at all like me,' 4 = 'neither like nor unlike me,' 7='very much like me'), and open-ended questions (versus questions that can be answered with a yes or no--you get more info that way, but then you have to find a way of making sense of those narrative answers).
Step 3: Item validity. You might want to keep in mind
the following questions when developing your items:
--Will I know if my subjects are responding honestly? (Include
items which every honest person would answer one way, e.g., "I
have never told a lie" would be False.)
--Will I know if my subjects are responding randomly? (include
duplicate items and score to ensure they were consistent)
--Are the items sensitive to my subjects' possible response set?
(Include equal numbers of items keyed true and false; reverse
the likert scale order (7=low, 1=high.)
[Step 4: Questionnaire validation: first administration. Now that the items have been developed, is the survey reliable and valid? If this were a REAL instrument you were developing, your survey would be validated by being administered to large numbers of people. Here, you would need to be sensitive to the characteristics of your subjects, their ethnicity, age, gender, where and when it is given, under what conditions, etc. You would need to determine whether or not it demonstrates split half, odd-even and/or test-retest reliability, and if it demonstrates internal consistency (items are measuring the same thing), convergent and/or discriminant validity (text). Obviously, for a lower division class, you are not required to validate your questionnaire! But you would want to do that next.]
Step 5: Administer your survey. You may ONLY administer it IN OUR CLASS (otherwise, you'd have to get it evaluated and approved by the Human Subjects Committee!). Because 480 people is a LOT of data, you might just make 75 copies and randomly distribute them.
Step 6: Analyze your data. See a TA for help here. You will at the least want to get mean responses and standard deviations for each variable of interest.
Step 7: Write up your results. Start by saying WHY you thought this was an important topic, briefly survey the literature on the topic, say what you did, and what the results were. Include your survey form! Considering the items you chose and the subjects to whom you administered your questionnaire, what does it really measure and in whom?