The U.S. News & World Report law school rankings methodology may appear simple at first glance, but it is also widely (and deeply) misunderstood. In this blog, we hope to explain why and how.
The official rankings methodology can be found here. Visually represented, the assignment of weight to each score category is represented below.
U.S. News Rankings Raw Score Weight Breakdown
Ambiguous Metrics
While U.S. News does tell us each of the metrics it factors into the rankings and the precise weights they are given, it does not publish precisely how all of these metrics are calculated. It is impossible to know their exact methods for certain metrics, and thus impossible to replicate with perfect accuracy. For example, it appears that some data is rounded, but U.S. News does not disclose how it rounds that data. Another issue preventing perfect accuracy is job category weights. U.S. News does not disclose how much weight most job types are assigned (e.g., how much is a full-time, short-term, professional position worth versus a full-time, long-term, JD-advantage position?). This is particularly problematic for schools that tend to have more of the non-full credit job outcomes. Further, U.S. News does not disclose the source of test-score percentiles it uses. All these combine to make a perfect replication quite difficult.
However, the basic methodology U.S. News uses is not, ultimately, that complicated. Certain things are convoluted—the way that they transform certain data, the different weights assigned to job categories, etc.—but U.S. News does not take active measures to prevent people from replicating their methods (at least, not beyond declining to disclose certain parts of those methods), and even for those things they don’t disclose, they generally provide enough “hints” to allow reasonably accurate replication.
The Unintuitive Weighting System
One of the easiest mistakes to make in understanding the U.S. News law school rankings has to do with the pie chart above. The weighting of the various metrics, as presented by U.S. News, is deceptive due to the way they standardize and scale the data. Taken literally, one might assume that a raw score would reflect that each piece of data—Median LSAT, Library Resources, Peer Assessment, etc.—contributes its respective percentage to a given school’s raw score, similar to how one might calculate an overall grade for a course from several assignments with different weights. That is not the case, and it is a common and key misunderstanding.
How, then, is the score calculated? Within each metric, U.S. News scales and standardizes scores (Z-scores), and then weighs the resulting score by that factor’s respective assigned weight. Unlike in our example of finding a grade for a course from a number of assignments, in the U.S. News formula, it is quite possible to achieve “grades” above 100% or below 0% for each given factor. In fact, this is quite common. This distorts the expected “weight” of each factor, in many cases quite drastically. The Librarian to Student Ratio is an example of this—despite the fact that it only makes up 2% of the U.S. News formula, Yale Law School overperforms so much in this metric relative to other schools that, in the most recent rankings edition, it was responsible for its number one position.
This is all to say, scores well above the average for each factor are strongly rewarded, while scores far below the average are severely punished. The rewards—or punishments—are stronger the larger the factor weight is, in accordance with the score multiplier factor shown in the pie chart above. It is this understanding that is critical: the schools that are the most highly ranked are those that have scored significantly above their competitors in a few specific factors. When examining why one or a few schools are ranked above another, it usually comes down to just a few categories wherein the higher-ranked school outperformed, or the school in question underperformed.
Yale, again, exemplifies this principle. By almost every U.S. News measurement, Yale is no different from Harvard, Stanford, Chicago, or others among the most elite law schools (sometimes even below them), except in one key regard: the Librarian to Student Ratio. Yale is so far above the average here—almost six standard deviations—that even when weighted for that category’s relatively small 2% contribution to overall score, Yale receives over a full point more in score contribution than the next highest-performing school in this metric. Without this metric, Yale would not be tied for first.
How U.S. News Rescales Rankings Scores
After being multiplied by their respective score multipliers, the weighted scores from each category are then added together. The totals are then rescaled: each school is weighted as a percentage of its total sum compared to the highest-scoring law school (historically Yale, though theoretically it does not have to be). This 0-100% score is rounded to the nearest whole number, and that is a school’s reported U.S. News raw score. For example, if the sum total of each of a school’s weighted scores from each category added up to 56% of Yale’s sum total, this would result in a raw score of 56. These raw scores are then ranked.
Rankings Inflation
It is important to recognize what this “ranking by relation to overall average” means for an individual school’s ranking. The U.S. News rankings are not static, and neither are the actions of schools that make them up. Almost every school seeks to improve itself—quality of students, reputation, job outcomes, etc.—and thus, the average and the overall bell curve of many of the U.S. News metrics go up over time. We refer to this as “inflation” in the rankings. This is a frequent source of frustration among law school administrators and leadership. They look at their regular increases in test scores, in money spent, in the quality of faculty hires, and in other categories and wonder why their rankings are not improving in turn. This is because of the rankings inflation described above. Every year, a given school must improve more and more to yield just the same rankings outcomes as they did in the past.
Explaining the Substantial Changes to the 2023-2024 Law School Rankings
We would be remiss not to discuss the changes to the rankings U.S. News made in 2023. These changes were a result of many law schools choosing to “boycott” the rankings—that is, to opt not to submit data in response to U.S. News’ annual questionnaire. The boycott began when Yale Law School announced it would not participate in the U.S. News rankings going forward, which sparked a number of other law schools to decide not to participate. In total, about one-third of law schools chose not to participate in the 2023-2024 edition rankings data collection.
The rankings boycott forced U.S. News to make changes to their formula. For years, a few schools have not provided all the data U.S. News asks for. Typically these schools have been lower-ranked, usually in the unranked category of law schools or close to it. In those instances, U.S. News either used publicly available data or, for data not publicly available, imputed a placeholder value to the non-reporting schools. This placeholder was usually equal to 1 standard deviation below the national average for a given category. So, for example, if the national average for Expenditures for Instruction was $50,000, and the standard deviation was $10,000, U.S. News would assign an Expenditures for Instruction value of $40,000 for the non-reporting schools.
However, this approach only worked so long as the non-reporting schools were largely confined to the lower ranks and limited in number. If higher-ranked law schools were assigned rankings values equal to 1 standard deviation below the national average, their overall rank would, of course, drop quite dramatically compared to schools that reported all their data and received full credit. Once multiple highly-ranked schools decided not to submit data to U.S. News, they had no choice but to change their approach and eliminate any metrics that they could only obtain from schools self-reporting. Fortunately for U.S. News, the American Bar Association makes a significant amount of data publicly available through 509 reports, bar passage data, and employment data.
Eliminating those metrics that are not publicly available meant that U.S. News had to reassign 19% of the formula’s weight. Further, U.S. News announced that it would reduce the weight of the two assessment score metrics, which were also the target of significant criticism. U.S. News also decided to reduce the weight assigned to Median LSAT and Median GPA, which was something they had not announced in advance of the rankings release. In all, that meant there was quite a lot of weight to re-assign.
U.S. News Metric Weight Assignment, 2023 vs. 2024 Editions
U.S. News chose to focus the vast majority of its weight reassignment on outcome measures. Employment at 10 months more than doubled in weight, going from 14% to 33%. First-Time Bar Passage went from 3% weight to 18% weight. U.S. News also introduced a new bar-passage metric, Ultimate Bar Passage Rate, which received 7% weight. All told, the post-graduation outcome measures now account for 58% of the total formula weight. This is a dramatic shift. Those three outcome-based metrics now account for more than all the other metrics combined. They also significantly decreased the weight of the admissions metrics.
U.S. News also made other small changes within their calculations of certain metrics. The most important was to adjust the weight given to certain specific employment outcomes. U.S. News bases its ten-month employment rates on assigning different weights to the several dozen categories reported to the American Bar Association. Historically, jobs that were funded by the law school received less credit than jobs in the same category not funded by the law school. For example, a school-funded, JD-required, full-time, long-term job received about 75% of the weight the same job not funded by the law school did. In response to substantial criticism, U.S. News changed that this year. All school-funded jobs now receive the same weight as any equivalent job not funded by the law school. Finally, U.S. News now assigns full credit to graduates who are pursuing a graduate degree full-time 10 months after graduation.
The U.S. News rankings are in a period of great change, and the extent to which this will impact law schools, applicants, and students remains to be seen.