By: Seema Nath, Faculty of Education, University of Cambridge
About a month ago, I embarked on the second phase of my PhD journey and started my fieldwork. The pilot I carried out last summer helped me immensely in shaping up my research questions and strengthening my research instruments and my overall research design. However, it is only now, after starting my fieldwork, that the full realisation of the benefits of piloting have dawned on me.
To give some background, my research aims to critically understand the teaching and learning for children with special educational needs and disabilities within mainstream classrooms in India. This blog post is a reflection on how my pilot research helped me in shaping my research design and helped me to plan my fieldwork.
As individuals in social sciences know, pilot studies can be employed primarily for two purposes – either to test out the feasibility of a study or to test out the research instruments (Teijlingen and Hundley, 2001). Sometimes pilot studies are done with the dual purpose of both determining the feasibility of the study and to test out the research instruments. In my case, I carried out the pilot research with the aim of testing my research instruments. I was able to pilot my research instruments with a small sample within my study population. For me, the pilot study was also a means to establish the validity of these research instruments within the context I was planning to use them. However, even though I started out with the aim of just piloting my research instruments, eventually it also influenced my research design.
Initially I planned a mixed method research design with a combination of questionnaires and interviews. My research instruments consisted of an attitudinal scale and teacher questionnaires (quantitative instruments) and semi-structured interviews with multiple stakeholders (qualitative instruments). I managed to pilot all these instruments with a small sample of stakeholders in the school system; two things led me to change my research design:
- Though the quantitative instruments allowed me to include more respondents, it was not providing me with the depth of information that the semi-structured interviews provided. In the process of obtaining supplementary data, I realised that the data I was getting was not up to the mark. The amount of time that was spent collecting the quantitative and qualitative data hampered me from gaining depth of information.
- One of the things I had not foreseen (and I thank my assessors and supervisor for bringing it to my attention) was data management and whether I would be able to utilise (through transcription, coding, data analysis, etc.) all the generated data to finish my PhD within the stipulated time. (Some of the best advice I received was to clearly define the scope and limitations and always keep the research aim in mind).
Thus keeping these factors in mind, I changed my research design to be qualitative in nature and increased the scope to include more in-depth interviews and do away with all my quantitative tools. In addition, this change was also brought about because the pilot made me reflect on my research questions; it shaped my second research question into its current form. (In many ways, the pilot felt like a miniature version of what was in store for the actual PhD research).
The pilot also strengthened my research instruments. First, it helped me to understand if the interview questions meant the same to my respondents as they did to me. Establishing a common meaning for the questions was very important. Second, it contributed to my understanding of the school context where I would be working – for example, how their classes were structured and the activities they did through the day. These and other details aided in shaping my final research design and instruments.
Third, the pilot research also provided me with an opportunity to plan my data management strategies, the amount of time required for transcription and translations, and a chance to perform rudimentary data analysis. A very important thing I learned from my pilot study was how to schedule the time to obtain informed consent and conduct the interviews with the various stakeholders. This helped me to plan the timeline for data collection for my actual research in a more realistic manner. For instance, I had initially planned my teacher interviews as Part 1 and Part 2 because I had estimated that I would not have enough time to do the entire interview in one session. However, the pilot study revealed that it was possible to conduct Part 1 and Part 2 in the same session. In fact, it provided much richer information as continuity was maintained between the questions. As a result, I am now able to schedule all my interviews and classroom observations in a more planned manner because I am already familiar with the structure of these schools and thus stay on track with my data collection timeline.
As parting advice, I would say an ideal period for the pilot study is right after finalising the research design and instruments. At the Faculty of Education in Cambridge, I would say a good time to consider the pilot study will be anytime after Lent term (or earlier if ready). It is important to account for enough time after the pilot study for data analysis and write up before the final submission of your upgrade report.
I would not do justice to this blog post if I do not list out few of the limitations of pilot studies. At the onset, in terms of practicality, the availability of finances does affect whether a pilot study is possible, especially if your research is in a different country. There is also a fear regarding contamination of the data pool (Teijlingen and Hundley, 2001). If the instrument changes based on the pilot study, then re-administering the modified tools to the same respondents may influence the data. However, if the instruments stay the same then this does not affect the data already collected during the pilot research. Nevertheless, in order to avoid these pitfalls, it is advisable to not use the same sample in the pilot study and the actual research.
In conclusion, though a pilot study might not always guarantee a smooth sailing in PhD field research, it does help to avoid many pitfalls and makes life in the field a lot less stressful. It helps deal with the uncertainties of social science research with more confidence. I am very happy with the decision to do a pilot research and it is helping me enormously as I move forward with my fieldwork.
Van Teijlingen, E. R., & Hundley, V. (2001). The importance of pilot studies. Nursing Standard. 16, 40,33-36. doi: 10.7748/ns2002.06.16.40.33.c3214
Seema Nath is a second-year PhD student at the Faculty of Education and Hughes Hall and a Cambridge Trust scholar. She is also a member of the Research for Equitable Access and Learning Center (REAL). Her research focus is to critically understand what constitutes teaching and learning that includes all children (children with disabilities and typically developing children) in mainstream low resourced schools in India. Her research interests include international education, inclusive education, disability, and education in countries of the South. You can follow Seema on Twitter: @seemanath.