Research and Measurement Guide

If you have worked through the first steps of the blueprint process – identifying your purpose, objectives, and research questions – you are now ready to collect data. It is helpful to make a data-collection plan before collecting data, which should cover:

• The type of data to collect (i.e., quantitative, qualitative),

• Who will collect the data,

• How often the data will be collected (e.g., pre/post, once a quarter), and

• How to access the data (e.g., state dashboard, school records).

While completing a data-collection plan, consider collecting quantitative data (numbers) and qualitative data (texts, interviews, or observations). Some data may already exist – such as student scores, discipline records, or state dashboards. Be mindful that you may need to acquire access or certain permissions from other data sources such as libraries or government databases.

To ensure that data best represents the population or subgroup you are studying, consider collecting different types of data that you can use to confirm your findings. For example, if you are investigating student growth in mathematics, beyond looking at student test scores (quantitative data), it may be informative to collect student reflections (qualitative data) about their learning in their math classes or conduct observations of a math lesson (qualitative data).

As another example, if you are working on a needs assessment or designing a new pilot or program, you might want to take a more qualitative approach and conduct empathy interviews or have your team complete a self-assessment.

Access to data varies according to your role (e.g., district leader versus building leader). However, it is critical that data is collected from the perspective of seeking what can be learned rather than confirmed to avoid preconceived, biased perceptions of the populations involved in your research. Every student, teacher, and community member exists within a unique context (e.g., socioeconomic status, learning level, preferred language); therefore, data should be collected both quantitatively (with numbers) and qualitatively (with stories) to gain an insightful understanding of their unique experiences.

Gathering Rich, Descriptive Data

Rich, descriptive data is detailed and comprehensive information that provides a deep and nuanced understanding of a particular phenomenon. It is often collected using qualitative methods such as interviews, focus groups, or open-ended surveys, and it is essential for capturing the perspectives of the participants and ensuring more equitable outcomes.

Using Subgroups

By analyzing data by subgroup (e.g., gender, race or ethnicity, grade level, neighborhood), you can acquire tailored and specific insights. Subgroup analysis helps identify variations and patterns that might otherwise be hidden when analyzing the entire dataset. Most importantly, subgroup data enhances the depth and applicability of findings that help inform decisions and targeted interventions with equity in mind. Take a look at the following to help identify meaningful subgroups:

Organizing data is a critical precursor to effective data analysis. Since many schools and districts already use Google for Education, tools like Google Drive and Google Sheets can make excellent resources – assuming you are using a district account and have first checked your student data privacy policy for adherence. (Note: The process described below would also work with Microsoft SharePoint or OneDrive and Excel.)

Organizing and Analyzing Quantitative Data

These best practices will help you gather all of your data files and then organize your data into a database:

• Create a folder that is accessible to the members of your research team. To protect the privacy of participants, ensure that only the people who need access to the data have permission to do so.

• Store an original copy of all raw data files you export from surveys, assessment platforms, or other data systems (such as your student information system or human resources database). If you make a mistake during your analysis, you can still refer back to this original data source.

• Maintain a shared document to keep track of all procedures you use to clean and analyze your data. By cleaning the data, we mean removing any personal identifiers to protect the privacy of your participants as well as columns or rows that may not be necessary from different exports. In this shared document, make a note of any items that you rename or data points that you decide to remove.

• Import all quantitative data into a Google Sheets (or Microsoft Excel) workbook and organize it around a common identifier such as a student or teacher ID number. Consider using color-coded and labeled tabs to categorize the data in your workbook.

• Once you have a single dataset for analysis, ensure it is set to ‘View Only.’ Encourage your research team to copy that dataset before running any analyses to prevent accidental edits.

In addition to Google Sheets or Microsoft Excel, research teams may choose to use other statistical programs such as Tableau, R, SPSS, or Jamovi to organize and analyze quantitative data. Regardless of the tool you use, make sure to save all files or outputs in a shared folder.

Organizing and Analyzing Qualitative Data

Although qualitative data provides a wealth of insight into the thoughts and perceptions of the population(s) you are examining, it can quickly become overwhelming. Organizing and analyzing data should be completed with efficiency and integrity to ensure accuracy.

There are several platforms such as nVivo, ATLAS.ti, and Quirkos to support qualitative analysis. For this guide, the strategy card below walks you through a series of steps using Google Sheets or Microsoft Excel to capture and analyze answers from open-response survey questions, transcripts from interviews or focus groups, and/or researcher notes.

When analyzing your data, you might take an inductive approach, through which you allow themes to emerge as you read, or a deductive approach, in which you start with preconceived ideas. The following steps provide general guidance for analyzing your qualitative data:

1. Read through the data, paying careful attention to the frequency and commonalities of terms or ideas that may be emerging. Make sure to document your thinking as you go.

2. Begin organizing responses. Think in terms of codes or keywords as well as broad themes and sub-themes. Keep the open-response/interview/focus group questions in mind.

3. Begin applying your codes, themes, or sub-themes to your data.

4. Document how you are defining different codes or themes and determine why you think they are relevant.

5. Review the responses, codes, themes, and sub-themes again to determine your findings. In this last step, some themes may be combined or even split to create new ones.

The following external resources can also help organize and analyze qualitative data:

• Step-By-Step Guide: Qualitative Data Analysis

• Four Simple Steps to Do Qualitative Analysis

• Your Ultimate Guide to Qualitative Research (with methods and examples)