New Report Outlines More Effective Use of Data

Complete College America (CCA), a national non-profit organization with the mission of raising postsecondary attainment in the U.S., has released a report titled, "Using a Measurement System to Strengthen Student Success Reforms" along with a companion workbook that provides step-by-step guidance and tools toward the effective and impactful use of data.

The goal is to build and sustain a culture of data and measurement for institutions and states. At a time of declining enrollment and retention as well as other changes to higher education due to the pandemic, the report notes that it is essential to make informed decisions on how to improve student success and completion. 

“If data is supposed to be depicting student reality so you can improve that reality, you have to know on the front end what type of data you’re looking for,” said Charles Ansell, vice president for research, policy and advocacy at CCA. This involves looking at metrics and measurements in alignment with the strategies to identify the necessary data.

“If you care about things like racial equity and college attainment, then identify the metrics you care about, and then from there identify the data you need,” said Ansell. “We’re concerned first and foremost with making sure that you’ve identified the full measurement system that you need to be able to track whether you’re meeting your mission or not.

“That way, no reform effort becomes a mere fad, but rather it becomes part of your ongoing operations,” he added.

The guidebook and its tools can help a college or university know whether it is meeting its student success mission, identify institutional performance gaps, develop and assess the impact of specific strategies and initiatives, inform decision making and generate shareable information.

Student success reform efforts, if well executed, improve graduation rates in a way that closes equity gaps, said Ansell. A key part of that is effective management of the reform efforts. To be able to reach goals, there needs to be structure and strategy.

Dr. Michaela Rome, vice provost for institutional effectiveness at Sacred Heart University, said data is absolutely critical to making decisions about strategic ways of best supporting students. She said all her positions in higher education have involved supporting and fostering a culture of assessment and the use of data, and there is room for expanding the understanding of best uses.Dr. Michaela Rome

One component outlined in the CCA report is clarifying language. Then, building measurement systems, for which the report identifies four cornerstones: measure what matters, source the data your college needs, use the National Student Clearinghouse PDP (Postsecondary Data Partnership) and have regular conversations about data.

“Given the landscape for higher ed and limited resources, being responsible for good stewardship of resources, we really need to use what we have in a way that is going to make the biggest impact for the areas of highest need and our priorities to make sure we’re moving in the right direction,” said Rome.

Ansell mentions the student debt crisis, noting, “If you’re not applying principles of strong management that lead to increased graduation rates and closures of gaps, then you’re not doing your fullest part to make sure that higher education is truly that equalizer it’s supposed to be.”

If there is simply a deluge of data rather than targeted data, then it won’t lead to effective solutions, said Ansell. He suggested that institutions look at their strategic plan and if they don’t have one, make one. Identify key performance indicators.

“From there, figure out how they would quantify the attainment of the goals in the strategic plan and then know what changes on a frequent basis so that way their work is always serving the higher goals,” Ansell said. As an example, he cited a community college tracking median earnings after degree completion or percent employed in their area of occupational training, which are ways to measure market value of a degree or credential. Ultimately, build a full list of the relevant metrics of overall institutional success.

The report notes that it is important to examine disaggregated data, to which Rome agreed. “If we have groups of students that are not performing or not succeeding in alignment with other groups, we want to make sure that we have the resources to support them to be successful,” she said. “When you look at the overall numbers and the overall picture, it can often hide inequities within those numbers if you don’t pull the data out and disaggregate it.”

The information detailed in this report can be utilized in community colleges, four-institutions and even graduate schools. Ansell said he hopes institutions use it to build a full measurement system that embeds predictors of college completion as found in the PDP.

“Then use that to figure out not just what data they need but what are the daily, weekly, monthly dialogues they need to have, with which precise people, such that any reform efforts enter an extremely efficient, effective machine for managing success for all students and making sure this all promotes equitable attainment,” said Ansell.