Things to remember and have when testing

I finished testing for my PhD yesterday! Hurray!!!

In three years of PhD, I’ve done 9 studies:

Year 1:

  1. Behavioural test on the modality switching costs of perceptual modalities and linguistic dimensions (48 participants);

Year 2:

  1. EEG study on the modality switching costs (16 participants);
  2. Norming of 452 sensory adjective metaphors;
  3. Behavioural test on shallow processing of metaphors (28 participants);
  4. Behavioural test on deep processing of metaphors (40 participants);

Year 3:

  1. Behavioural test on the effect of time constraints on shallow processing of metaphors (48 participants + 8 pilots);
  2. Behavioural test on the effect of time constraints on deep processing of metaphors (54 participants + 8 pilots);
  3. Behavioural test on the within-participant comparison of shallow vs deep processing (48 participants);
  4. EEG study on the within-participant comparison of shallow vs deep processing (16 participants).

A couple of lessons I learned from testing.

1. Always double-check.

The reason why my first study took a year to finish (and it actually spilt over to second year) was that I ran it three times. It took me several months to select the right material. I ran the study first time and found no effect whatsoever. Since it was the first stage of my study, we decided it might be a good idea to re-examine our material. We felt the task may be too hard for people (the accuracy rate for fillers was about 60%), so we tweaked on the material and ran the study second time. Still no effect. We decided to proceed to the EEG study which would only use the material that actually had shown the modality switching cost. But when I went on to choose the material, I realised I did not presented to my participants all the material. The participants were supposed to be in 3 different conditions, but I had given them the same material for all three conditions. (I really wanted to kick myself when I found out.) So, I could do nothing but to run the study the third time. That was at least 96 participants, 48 hours wasted. So, always double-check the material, the programme, the counterbalancing sheet. Even though data-peeking is not a good practice, still look at the data and see if you’ve done the right thing. Better to discover a mistake half-way through the study than realising it at the end.

2. Sort out your paperwork.

Apart from the necessary ones, i.e. the Information Sheet, the Consent Form, the Debrief Form, you may also need a sheet to record how many participants there are in each counterbalancing condition and probably a sheet for expense. I keep all my paperwork in one expanding file, and each file goes in one pocket. It might sound very simple and even pedantic, but it really saves me a lot of time each day to pull out the papers needed.

3. Recruiting, recruiting, recruiting…

During this three years, I tested about 300 participants in lab, and also another 100+ online. Recruiting participants has always given me a headache. Here are some ways out.

  1. Use course credits as much as you can. Lancaster, so do most other universities, has course credits for psychology students if they participate in research. There are different kinds of incentives for them. For instance, they can get access to the same course credit system (SONA) in their 3rd year. They can get a bump in their Year 1 final grade. They can also participate in paid studies once they’ve collected enough credits. Most students are very happy to participate. So it helps if you schedule your studies at the beginning of each term (first 3-4 weeks). This enthusiasm dies down around Week 5 when they start to have assignments piling up.
  2. Speaking of SONA, it could be helpful to advertise for it on its own at the beginning of each term, because I realise a lot of students outside of the Psychology Department are very interested in participating in psychology research but do not know of SONA at all. And a lot of paid participants I ended up having were ones that signed up on SONA because of other people’s studies and then discovered that they could get their coffee money for half an hour’s time. I feel the best way is for the PsychSoc to advertise for SONA during the freshers’ week. I might bring this up to someone next time. 😛
  3. Join some Facebook groups and advertise your study there. I also have left flyers and stuck up posters at different corners on campus. But I doubt how useful they were.
  4. For a more effective way to advertise your studies, especially if they were paid, you can also email different department secretaries and ask them to do you a favour and circulate your advertisement. Most of them will be quite happy to help, or even participate themselves (True story). However, I do think this should be your last resort. Maybe don’t email them every time. I used this way to recruit for my final study and managed to find enough people even during the spring break.
  5. Collect good karma. In the psychology department, with other departments that do human research such as linguistics, computer science and so on, you will understand what it means by what goes around comes around. Your fellow students will know your pain of recruiting and are more likely to help you in your study. To cultivate this culture, it helps if you participate in other people’s studies once in a while. Plus, most of them are pretty fun.
  6. Some ways I haven’t used and could potentially be useful includes organising a mailing list for potential participants, especially those that wanted to participate in your study last time but did not get a timeslot (you can also exchange mailing lists with others); finding out times for different lectures and ask for 5 minutes at the beginning to speak to the students and leave your flyers there; holding up a sign that says “Psychology study for £7/hour” on campus and sign people up there.

To be honest, recruiting adults is considerably easier than special population such as infants and autistic children. The studies itself is much less time consuming as well. I may dedicate another post for my experience with EEG.

Bonus Question: What do you do when you are testing?

My studies usually take about 30 minutes, so it absolutely cut my time into pieces. I can hardly do anything substantial. Therefore, I usually try to do some housekeeping stuff like managing my to-do list and replying to emails. Sometimes, I try to mark some students’ report or write a paragraph for the method section of the study. For my last two studies, I even brought a good novel and enjoy.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s