Q: What does it mean to say that you measure “Legislative Effectiveness?”

In our book and related research, we define legislative effectiveness to be the “proven ability to advance a member’s agenda items through the legislative process and into law.” In defining legislative effectiveness in this way, it is important to note that our definition consists of four separate components: proven ability, advancing legislation, members’ agenda items, and progression through the legislative process into law. Our measure of legislative effectiveness was developed to incorporate each of these four components into a summary measure of legislators’ “effectiveness” in lawmaking. 

More specifically, the LES is constructed to measure how successful a given Representative or Senator is at moving his or her own legislative agenda items (meaning, the bills that he/she sponsors) through different stages of the legislative process, where each of those bills is also coded for relative substantive significance. The specific formula for the LES is defined more explicitly in our methodology section.

Q: Why can’t I find out the Legislative Effectiveness Score of a Representative or Senator in the current Congress?

As described on the Methodology page, a Representative’s or Senator’s Legislative Effectiveness Score is a parsimonious metric of how successful she was at moving her agenda items through each of five stages in the legislative process, where each agenda item is coded for substantive significance, in comparison to all other members of the House of Representatives (or Senate). In order to calculate this metric, it is necessary to know how successful each member of the House (or Senate) was at advancing her agenda items through Congress, which can only be ascertained at the conclusion of a given Congress. 

Likewise, one aspect of the coding protocol — identifying whether a bill that was introduced in a Congress is “substantive and significant” — can only be ascertained by consulting Congressional Quarterly publications and Project Vote Smart listings, including those that are published up through the conclusion of the relevant Congress. Hence, it is substantively inappropriate (and technically impossible) to replicate our methodology to calculate Legislative Effectiveness Scores for Representatives (or Senators) who are sitting in a current 2-year Congress before that Congress has concluded.

Q: If a Representative or Senator has a low LES or ILES, does that mean that she is not an effective lawmaker?

Not necessarily. As described above, the LES is designed to measure how successful a given Representative or Senator is at moving his or her own legislative agenda items (meaning, the bills that he/she sponsors) through different stages of the legislative process, where those bills are also coded for substantive significance. A relatively low LES results from few introductions or from sponsored bills not progressing particularly far.  Likewise, given that an ILES captures how successful a Representative or Senator is at moving bills in a particular issue area, it is quite plausible that a Representative is extremely successful at advancing bills in issue areas that are most important to her, while devoting little attention to bills that engage with issues that are less important to her, leading her to have a high ILES in some issue areas, but not others.  Such variance in scores would reflect a Representative’s or Senator’s different issue priorities, rather than effectiveness in lawmaking, per se. 

That said, other efforts that may be commonly considered “legislative effectiveness,” such as working behind the scenes to help others’ bills pass, serving as Speaker of the House or party leader, or blocking proposals of opponents, are not included in calculating the LES.

Q: What is the range of the Legislative Effectiveness Scores and Interest and Legislative Effectiveness Scores?

Legislative Effectiveness Scores are normalized to be an average of “1” within each Congress. That said, the scores exhibit significant variance within and across Congresses, ranging from a low of “0” for those Representatives who did not introduce any bills in a given Congress, to a high of 18.7, for Representative Charles Rangel (D-NY) in the 110th Congress (2007-2008). In the Senate, scores range from a low of “0” for those Senators who did not introduce any bills in a given Congress, to a high of 10.2 for Senator Howard Cannon (D-NV) in the 96th Congress (1979-1980).

Interest & Legislative Effectiveness Scores are normalized to be an average of “1” within each issue area, within each Congress.  The scores exhibit significant variance within and across Congresses and issue areas, ranging from a low of “0” for any Representative or Senator who did not introduce any bills in a particular issue area in a given Congress (which is quite common), to occasional scores in excess of 100.  Such outliers occur most frequently in issue areas with very few bills in a given Congress.

Q: Why is the Legislative Effectiveness Score calculated based solely on H.R. bills or S. bills?

We are interested in understanding why some legislators are more effective than others at advancing bills that have substantive consequences for changes in policy. While other legislative vehicles have the potential for facilitating substantial changes to public policy in the United States (in particular, House Joint Resolutions that engage proposed constitutional amendments), most resolutions are largely symbolic in substance, and have no de jure force of law. We disregard all legislative vehicles that cannot lead to a direct change in U.S. Code if signed by the President.

Q: Does a Legislative Effectiveness Score capture if a Representative or Senator is effective at stopping legislation from moving forward?

No. The Legislative Effectiveness Score is constructed to measure how successful a given Representative or Senator is at moving his or her own legislative agenda items (meaning, the bills that he/she sponsors) through different stages of the legislative process, where each of those bills are also coded for relative substantive significance. Hence, it only captures how successful (i.e., effective) a Representative or Senator is at moving bills that she has sponsored forward in the legislative process. Any success that a Representative or Senator might have in slowing down, or stopping, the progression of others’ bills through the legislative process is therefore not captured by the measure.

Q: Does a Legislative Effectiveness Score account for Representatives’ or Senators’ amendment activities?

The Legislative Effectiveness Scores that are available on this website, and were the focus of much of the analysis in the book and related published research do not account for the impact of Senators’ and Representatives’ amendment activities on their respective effectiveness. In Appendix 2.1 of Chapter 2 of Legislative Effectiveness in the United States Congress: The Lawmakers, however, we present analysis where Representatives’ Legislative Effectiveness Scores are influenced by their success and/or failure in amending legislation, as well as whether their legislation attracts (or defends itself from) amendment activity. As demonstrated in Appendix 2.1, the main substantive points of analysis that are presented in our work do not change upon accounting for Representatives’ amendment activities.  We present similar analysis in the online appendix to “Legislative Effectiveness in the United States Senate” (Journal of Politics, 2018), and we demonstrate that the LES for the Senate is robust to the inclusion of Senators’ amendment activities in a variety of ways.

Q: Does a Legislative Effectiveness Score account for language from a Representative’s or Senator’s sponsored bills being incorporated into other legislative vehicles?

Beginning with the 117th Congress, Representatives and Senators receive credit for later lawmaking stages (e.g., passing their home chamber and becoming law) when a substantial portion of the language in their sponsored bills is incorporated into other legislators’ bills that become law. Such bill language is detected using 5-gram Jaccard similarity comparisons, coupled with criteria accounting for bill length, introduction dates, companion bills, and other idiosyncrasies of the contemporary lawmaking process in Congress.

Prior to the 117th Congress, Legislative Effectiveness Scores followed the methodology described in Legislative Effectiveness in the United States Congress: The Lawmakers (Cambridge University Press, 2014), which only assigned credit based on the fate of a lawmaker’s standalone bills, without considering language inserted into other legislative vehicles. Datasets with both methodological approaches (LES Classic and LES 2.0) are available for downloading. Such data illustrate that these two approaches, while distinct, yield highly correlated scores.

Q: Is the sponsored legislation itself “effective”?

We are not able to answer this question, or related questions about the efficacy of the legislation that is being sponsored by members of the U.S. House of Representatives or Senate. Our methodology allows us to measure how effective Representatives and Senators are at moving their agenda items forward in the legislative process, but it cannot speak to any questions about the relative effectiveness of the bills that are proposed and/or the laws that are ultimately promulgated.

Q: Are Legislative Effectiveness Scores comparable across time?

Legislative Effectiveness Scores (LES) and Interest and Legislative Effectiveness Scores (ILES) are normalized to average “1” within each Congress. Such a normalization facilitates easy Representative-to-Representative or Senator-to-Senator comparisons within a given Congress. So, if one Representative in the 110th Congress (2007-2008) had a LES of “1” and a second Representative in the 110th Congress had a score of “2” it would be appropriate to say that the second Representative was twice as effective in lawmaking (according to our metric) as the first Representative. 

Because the scores are normalized to average “1” in each Congress in each chamber, however, care must be made when trying to compare legislators across different Congresses or across chambers. It is appropriate to compare the relative ordinal rankings of Representatives across Congresses within the same chamber (for example, to say that Don Young was the most effective Republican in the 110th Congress, but only the fifth most effective in the 111th Congress).