The paradata research project

What is paradata?

Paradata is generally seen as the by-products of social research, usually referring to survey research. It relates to two types of paradata:

• ‘macro paradata’ which covers the length of interviews, how long it took to arrange the interview and the like, and
• ‘micro paradata’  which refers to more qualitative comments and perceptions found, for example, in the marginal notes on a survey booklet. For examples of notes on the 1968/69 Townsend survey go to Marginal notes on the questionnaires.

Paradata can provide a wealth of insights into both the topic of the survey and the methods used to conduct it. Its examination can help improve the quality of data collection.

Sod of an interview heading 6-36-0239

Copyright: 

Townsend archive

The fieldworkers's comments on an interview for the 1968/69 Townsend survey (from 6-36-0239)

The research project

‘Poverty in the UK: advancing paradata analysis and open access’ project was an ESRC-funded collaboration between the National Centre for Research Methods and the Poverty and Social Exclusion research team.  It focussed on the 1968-69 Poverty in the United Kingdom (PinUK) study led by Peter Townsend with the Poverty and Social Exclusion: UK 2012 study (PSE UK) used for comparative purposes. The project had two main aims:

• To conduct a comparative narrative and statistical analysis of paradata, including on contextual materials, comparing the Poverty in the United Kingdom 1968-69 survey with the  Poverty in the UK 2012 survey . This enabled an exploration of shifts and continuities in the social process of gathering household survey data about poverty.
• To provide open access to the data and contextual material relating to the PinUK survey for the wider national and international social science community and build understanding of its substantive and methodological potential and use.

For the comparative analysis, thematic and narrative analyses of marginalia in selected PinUK survey booklets were compared with the thematic analysis of 25 transcribed audio-recorded PSE UK survey interviews and narrative analysis of one transcribed audio-recorded PSE UK survey interview. The main stage of the project ran from April 2013 to July 2014.

• Listen to an interview with Professor Ros Edwards, the Principal Investigator of the project, on the importance of paradata and the implications of the findings for survey research today, including insights into issues around survey quality and costs, on the National Centre for Research Methods podcast page here.
•  Read an overview of the project by Rosalind Edwards, David Gordon, and Ann Phoenix 'Peter Townsend’s Poverty in the UK study – a methodological investigation' in the Spring 2014 issue of Policy World, pp. 22-23 here.

The project also tracked down and interviewed a number of people involved in the orginal research including some of the field workers for the 1968/69 survey and members of the research team. The full list of interviewees can be downloaded here.  For further details go to The video interviews - an overview. The video interviews are in the process of being uploaded on to the website and will be accessible through the left hand menu. They provide unique insights into the process of conducting the survey and its impact.

In addition, the project digitised a large number of documents relating to the 1968/69 survey for open access use. These include the pdfs of the orginal survey questionnaires which can be found The 1968/69 survey questionnaires and notes and documents relating to the 1968/69 survey and to Townsend's 1979 book, Poverty in the UK, which can be found in Original notes and documents from the Poverty in the UK study.

Findings

The project’s final report built on the analyses of the PinUK and PSE UK by-products, interviews and contextual materials to develop an account of the technological, social and professional role changes that have occurred over the past 45 years and discusses how the conditions of production have an impact on the data produced. The report finds that a key skill for field interviewers is the ability to be flexible in interviews, in the face of organisational concerns about standardisation. However, overall, there is now less possibility for spontaneous and flexible communication, and less opportunity to see the research as collaborative work. The report concludes that, overall, the substantive data is as much a ‘by’-product of the survey interview process as the conventional view that paradata is a by-product of substantive data gathering.

• Download the 'Advancing Paradata project and Open Acess final report' here.
• Read 'How paradata can illuminate technical, social and professional role changes between the Poverty in the UK (1967/1968) and Poverty and Social Exclusion in the UK (2012) surveys' by  Rosalind Edwards, Ann Phoenix, David Gordon, Karen Bell Heather Elliott and Eldin Fahmy in Quality and Quantity,
  November 2017
  , Volume 51, Issue 6, pp 2457–2473.

The analysis of the marginal notes on the survey questionnaires identified an emergent typology of the kinds of paradata being used by the interviewers. For more details about this emergent typology, read:

• ‘What can we do with marginalia, notes and letters appended to large scale surveys?’ by Ann Phoenix, Heather Elliott, Janet Boddy, and Ros Edwards, in NCRM Methods News, Winter, 2012, p. 4. 
• ‘The Possibilities of Narrative Analysis for Paradata: An Historically Situated Exploration’ by Ros Edwards, Ann Phoenix, Heather Elliott and Janet Boddy, the final report of a prior National Centre for Research Methods project.

The Townsend survey marginal notes help to illuminate substantive issues around at the time of the survey and provide notes on the social context as well as raising methodological issues as to the impact of the interviewer on the data collected. For examples of the range of marginal notes made on the Townsend questionnaires, go to Marginal notes on the questionnaires. To explore all the marginal notes in the questionnaires go to The 1968/69 survey questionnaires.

Project team

The ‘Poverty in the UK: advancing paradata analysis and open access’ research project was a collaboration between the University of Southampton’s National Centre for Research Methods (NCRM), the University of Bristol’s Townsend Centre for International Poverty Research and the NCRM’s research node on Narratives of Varied Everyday Lives and Linked Approaches (Novella) at the Institute of Education, University College London. The research team members were:
NCRM Hub, University of Southampton:     Rosalind Edwards (PI)
NOVELLA, Institute for Education, UCL:    Ann Phoenix (Co-I), Heather Elliott
Townsend Centre, University of Bristol:    David Gordon (Co-I), Eldin Fahmy, Karen Bell

The open access for this project was overseen by Joanna Mack, Honorary Senior Research Fellow, Bristol University and Visiting Fellow, The Open University.

Downloads

Below you can download the various reports, documents and articles related to the ‘Poverty in the UK: Advancing Paradata Analysis and Open Access’ project.  These downloads also include the setup syntax file of the PinUK dataset, made available for analysis of the survey data in the ESRC archive. The downloads cover:

• 'Advancing Paradata an Open Access project final report' (Advancing-paradat-final-report.pdf)
• Advancing Paradata research bid by NCRM, Novella and PSE (Advancing-paradata-research-bid-NCRM-NOVELLA-PSE-29-08-12.pdf)
• 'The Possibilities of Narrative Analysis for Paradata': An Historically Situated Exploration’ (Possibilities-of-Narrative-Analysis-of-Paradata_Hub_Novella_Final-Report.pdf)
• The list of people involved in the Poverty in the UK survey who were interviewed for the research study and their roles (List-of-interviewees-and-roles.xls)
• The topic guides for the video interviews for researchers, fieldworkers and administrators (Topic guide for interviewees.zip)
• The 'Paradata project information sheet' given to interviewees which gives an overview of the research (Paradata-project-information-sheet-for-interviewees.pdf)
• The Paradata project interviewee consent form (Paradata-project-interviewee-consent-form.pdf)
• The PinUK SPSS setup syntax file [To come]
• List of the Poverty in the UK survey sampling areas with codes and interviewers for each area (Poverty-in-the-UK-sampling-areas-and-interviewers.pdf)
• List of interviewers who conducted the Poverty in the UK survey (Poverty-in-the-UK-fieldworker-list.xls)
• List of the survey questionnaires included the pilot phase by study ID, area and interviewer (PinUK-Survey-booklets-pilot-phase-fields-by-fields.xls)
• List of the survey questionnaires included phase 2 by study ID, area and interviewer (PinUK-Survey-booklets-phase2-field-by-fields.xls)
• List of the survey questionnaires included phase 3 by study ID, area and interviewer (PinUK-Survey-booklets-phase3-fields-by-fields.xls)
• List of people involved in the Poverty in the UK 1968/69 research and their past and current roles (List-of-interviewees-and-roles.xls)
• Allocation of orginal notes and documents relating to the 1968/69 Poverty in the UK survey and to the Townsend book to folders (Allocation-of-original-Townsend-documents-to-folders.pdf)

This page was authored by Joanna Mack, University of Bristol and The Open University.