A Review of the Department of Education’s New College Scorecard Website

Over the weekend, the U.S. Department of Education released its long-awaited College Scorecard. Originally envisioned as a way to help the Department in determining whether institutions should remain eligible to offer financial aid, the focus is now on student-as-consumer information.  According to President Obama in his weekly radio address:

“Americans will now have access to reliable data on every institution of higher education.”

“Well, kinda.” – Russ Poulin, not President and not running for President or I would have had to call people names.

The College Scorecard Home Page

The College Scorecard Home Page

I realize  I have much less stature than the President, but I was asked last year by the Department to be on a data panel when they began their quest to create (what was then called) the Postsecondary Institutional Ratings System. Several issues were raised by that panel. Some were solved. Many remain.

I did not have time for an exhaustive study of the site, but I do have some initial impressions. Some of my questions might be answered with more experience and research.

The Good!

Student-as-Consumer Focused.
The data is intended to help students make informed college selections. This is a great goal and is much better than their original plan of grading colleges based upon Departmental data sets that were ill-equipped to do so. Institutions respond to student focused info or ratings. Look at how they try to game the U.S. News rankings or those who have already touted their standing in the Scorecard.

Design.
They focused on a simple, relatively easy-to-understand, mobile-centric layout. The site is easy to navigate and the graphics and pleasing and informative.

It’s Open.
The data sets are open for others to use and create tools to enhance the site. I enjoyed their “Under the Hood…” revelations of how they created the site, in which they say: “By giving developers access to an API, even more customized tools will be created, providing students more options than ever before to find the right school for them.

Click for More Info.
When students conduct searches, they are given several colleges (often over several pages) from which to choose. Students can click to find even more information about a college that might interest them.

There is No Information about Distance Education.
You may thing that’s odd coming from me. Thankfully, the old College Scorecard page has been relegated to the “Page Not Found” graveyard. In my testimony last year, I showed how the “Distance Education” option lead you to only those handful of institutions that are fully at-a-distance. Well, sine the vast majority of colleges now have distance education, that leaves out many, many distance education options. Being silent on distance education is better than being so misleading.

The…Ummmm…Not-So-Good

Some colleges are missing.

Webpage that shows colleges surrounding the F's and there is no Front Range Community College

Front Range Community College is Missing on this Alphabetical List of Colorado’s Associates Degree-granting Colleges

I live n the Front Range Community College district in Colorado. I thought I would check the data on them. I could not find them. The President did say “every institution of higher education,” didn’t he? Most of the colleges are there, but I noted some other absent institutions (this is not an exhaustive list, just some research to show that this was not an isolated case):

  • Colorado – Aims, Front Range, Pueblo, and Otero Community Colleges.
  • Arizona – Rio Salado College.
  • California – Bakersfield College.

Perhaps I did something wrong, but I searched both by variations on their names and by looking the state lists of colleges.

The dreaded first-time, full-time completion rates are used.
I knew this would be the case, but it really irks me. Under current data collected by the Department’s IPEDS surveys. They the group on which they base their “Graduation Rate” as: “Data are collected on the number of students entering the institution as full-time, first-time, degree/certificate-seeking undergraduate students in a particular year (cohort), by race/ethnicity and gender; the number completing their program within 150 percent of normal time to completion; the number that transfer to other institutions if transfer is part of the institution’s mission.”

This rate has long been a massive disservice to institutions focused on serving adults and community colleges. Here are some example rates: Empire State: 28%, Western Governors University: 26%, University of Maryland University College: 4%, Charter Oak Colleges: no data, and Excelsior College: no data.. The problem is that these numbers are based on incredibly small samples for these schools and do not reflect the progress of the bulk of the student body.

I won’t quote data for community colleges because they are all negatively impacted. They often serve a large number of students who are not “first-time” or define “success” in other ways

I know that they are working on a fix to this problem in the future. Meanwhile, who atones for the damage this causes to these institution’s reputation. This data display rewards colleges who shy away from non-traditional or disadvantaged students. Is this what we want?

Chart shows "Data Not Available" for Average Annual Cost and Graduation Rate.

Why is “Data Not Available”?

Why is “Data Not Available” for Some Measures?

For some measures for some colleges, the data field is “Data Not Available.” I have good guesses as to why that is. Will the average student? There should be an explanation.

Is the Focus on Salaries Good Public Policy?
I understand that this is put out by the Department of Education and they want to make sure that students enter fields in which they can repay their loans and make a good living. I’m all for that. But is that the highest and only goal of higher education? Frankly, I’m hoping that we continue to graduate a few social workers, teachers, nurses, and actors. They don’t measure well on this scale.

Some observations on this:

  • One of the first three charts that appears in the initial search for every college is the “Salary After Attending.” This gives salary great importance.
  • Salaries are compared to a national average, but salaries vary greatly by region. I can understand wanting to make it simple, but is this an over-simplification?
  • Salaries are institution-wide. They may vary greatly by program.

The “Are You Kidding Me? Charts
I’m not going to say much about these charts they provided, on “30 four-year schools with high graduation rates and low costs” and “Schools with low costs and high incomes.”   Williams College on a list of low-cost colleges? That’s interesting. Again, the site seems to be rewarding colleges with highly selective admissions. I’m biased because I don’t really think these colleges need yet another boost.

In Conclusion
The site is a good beginning at addressing the needs of the traditional student leaving high school and seeking a college. It leaves much to be desired for the non-traditional students who now comprise a very large portion of the college-seeking population.

I applaud the consumer-focused vision and hope that feedback continues to improve the site. I actually think this could be a fantastic service. I just worry that in the haste to get it out that we did not wait until we had the data to do it correctly.

What do you see when you look at College Scorecard?

Russ

Russell Poulin
Director, Policy & Analysis
WCET – WICHE Cooperative for Educational Technologies
rpoulin@wiche.edu

If you like our work, join WCET!

4 Comments

  1. Posted September 15, 2015 at 8:55 pm | Permalink | Reply

    Dan –
    Looks like you are correct. Institutions that granted more certificates than degrees in a year do not appear to be listed. According to the great work by Phil Hill that leaves out 17% of community colleges.
    http://mfeldstein.com/17-of-community-colleges-are-not-included-in-college-scorecard/

    That leaves out some major community colleges.
    Russ

  2. Tim
    Posted September 15, 2015 at 12:18 pm | Permalink | Reply

    In the “The…Ummmm…Not-So-Good” section I think you are being unnecessarily defensive. There is a large number of schools providing an extremely broad range of products. Higher education is not a single market with a single segment. I think you should focus on your market and segments. What’s good and what’s bad?

    Droning on about the inequities between the schools with huge endowments and everyone else is simply not productive. Harvard and Williams can charge a ridiculous amount and they can afford to offer huge discounts. So what?

    Don’t waste your limited energy hoping for the world to make sense. It doesn’t and it won’t. Who is your market? What are the segments? How can you serve them to the maximum. Can you get coverage in the NYT or on TV? I don’t know. But if you don’t focus on your challenges then your just wasting time and under serving your market.

    • Posted September 15, 2015 at 3:36 pm | Permalink | Reply

      My complaint was not with those institutions spending their own funds and endowments. You make the point that higher education in not a single market. I agree. I just think it is bad public policy for the Department of Education to advantage one segment over the others, especially when those advantaged are already well-funded.
      Russ

  3. Posted September 15, 2015 at 9:18 am | Permalink | Reply

    MISSING COLLEGES?

    Hi, Russ,

    The College Scorecard Help Desk suggests that since the aim is to list predominantly degree-granting 2- and 4-year institutions, if a school crafts a variety of local certificates, and provides more certificates than degrees in a given academic year, the institution doesn’t qualify for the College Scorecard.

    Instead, the school is categorized as a “predominantly certificate-granting institution” and not listed.

    That might account for at least some “missing” schools.

    Dan

11 Trackbacks

  1. […] take this argument and see where it takes us. Just under a year ago, Russ Poulin from WCET and I wrote a series of posts calling out major flaws in the College Scorecard, culminating in a joint article […]

  2. […] take this argument and see where it takes us. Just under a year ago, Russ Poulin from WCET and I wrote a series of posts calling out major flaws in the College Scorecard, culminating in a joint article […]

  3. […] from the database. This was pointed out recently in a list of such institutions carried by a WCET blog, a component of the Western Interstate Commission for Higher Education, a nonprofit that works to […]

  4. […] the College Scorecard was announced, Russ noticed a handful of missing schools. When I did the whole data OCD thing, I discovered that more than 700 […]

  5. […] the College Scorecard was announced, Russ noticed a handful of missing schools. When I did the whole data OCD thing, I discovered that more than 700 […]

  6. […] the College Scorecard was announced, Russ noticed a handful of missing schools. When I did the whole data OCD thing, I discovered that more than 700 […]

  7. […] looking at the new College Scorecard and its inherent flaws in the data, often starting with observations from Russ Poulin at WCET. Today Susan Svrluga, education reporter at the Washington Post, posted a new article […]

  8. […] from the database. This was pointed out recently in a list of such institutions carried by a WCET blog, a component of the Western Interstate Commission for Higher Education, a nonprofit that works to […]

  9. […] week Russ Poulin was the first person I read who noted the missing colleges in the new College Scorecard. I subsequently did some research (with additional pointers from […]

  10. […] Poulin noted in his WCET post that several community colleges were missing from the Scorecard based on a quick spot […]

  11. […] Russ Poulin at WCET has a handy summary of the new College Scorecard produced by the Education Department (ED) and the White House. This is a “first read” given the scorecard’s Friday release, but it is quite valuable since Russ participated on an ED Data Panel related to the now-abandoned Ratings System, the precursor to the Scorecard. Russ describes the good, the “not so good”, and the “are you kidding me?” elements. One area in particular highlighted by Russ is the usage of the “dreaded first-time, full-time completion rates”: […]

Post a Comment

Required fields are marked *

*
*

%d bloggers like this: