Watch: The History of the Legislative Effectiveness Scores

Transcript as follows:

Craig Volden (00:00):

Welcome everybody to the next episode of Batten Expert Chats today, we’re talking about scoring effectiveness in Congress. My name is Craig Volden. I’m a Professor of Public Policy and Politics at UVA’s Batten School.

Craig Volden (00:14):

I’m Alan Wiseman, the Co-Director of the Center for Effective Lawmaking with Craig Volden, as well as being a Professor of Political Science at Vanderbilt University, where I’m also serving as the department chair of political science.

Craig Volden (00:27):

We wanted to thank you all for joining. This’ll be a nice exchange back and forth Alan, and I will ask each other questions, but for those of you joining us live, you can ask questions down in the Q and A tab down at the bottom of the screen. And before we dive in, I just wanted to thank everybody for coming as well as thanking our support staff, Millie and Allison and Jeff who put this together for us.

Craig Volden (00:52):

Great. So we wanted to start by telling you a little bit about the background of the research project that we’re engaged in. And then, Alan will dive in and talk about scoring effective members of Congress and I’ll bring people up to date on some current research that we’re doing. Um, so this idea of thinking about who are the effective members of Congress is something that Alan and I were working on back when we were both faculty at different universities, we were both at Ohio State a little over a decade ago. And we were thinking about this question of not just, you know, what’s different about Democrats and Republicans or liberal or conservative members of Congress, but who actually gets things done, who are the ones who are willing to do the work to propose the bills, to gain the expertise necessary to, you know, follow them through the lawmaking process and all the way into law.

Craig Volden (01:45):

And as we were wrestling with that question, we were finding that we weren’t comfortable with the types of measures that were available, you know, at that time to approach that question and we were hoping that we could, you know, kickstart a bunch of scholarship in that area as well, partly by developing measures of effective lawmaking, but partly just raising a series of questions that we felt had been neglected, both in a scholarly arena, but also in terms of, you know, what was going on in American politics at that time. So we started engaging in a research project. Alan will tell you in a minute about how we generated our legislative effectiveness scores. But with those scores in hand we moved forward with a bunch of research that resulted in a book that we published in 2014, we released our scores broadly and at that time had some nice receptivity from the scholarly community, but also gaining interest from media, from Congress members, from their staff.

Craig Volden (02:49):

And then from a number of foundations that were asking, “well, are you interested in keeping the scholarship going? Are you interested in engaging more in the, in the public life of Congress?,” And when we said “yes, all of the above,” we were able to receive some nice funding from a variety of sources, that led us towards setting up then at that point, the Center for Effective Lawmaking at Vanderbilt and at UVA, and set us on a longterm course for studying effectiveness in Congress.

Alan E. Wiseman (03:23):

So building on what Craig was saying, you know, central to a lot of our efforts i to try to understand why some members of Congress are more effective lawmakers than others. And to engage with that question, as well as a bunch of other auxiliary questions that follow from that, the first thing we need to do is come up with what we think is a meaningful metric of lawmaking effectiveness. So what we did, dating back to the early days of the project back in 2008, 2009, is that we, and by we, I really mean some very well technically trained, undergraduates, including someone who we only half-jokingly say who now works at the national security agency, essentially went to congress.gov and his predecessor was Thomas, which is the Library of Congress’s website to essentially rip down all information from every bill. And by that, I mean a public bill.

Alan E. Wiseman (04:13):

And by saying public bill, that means it’s a bill that if it’s ultimately passed, actually changes US law. Every bill that was introduced into the U S house from 1973, up until 2008, that was the original datasets. We’ve since expanded that from 1973, up until 2018. So our most recently completed Congress, which just ended right around the midterm elections. And we’ve also expanded the scope of analysis to not only include the House, but also the Senate. So having said that, as I said, we ripped down all information for every public bill that was introduced into both the House, as well as the Senate and for every bill what we did is identify every stage that it went through in the lawmaking process, from introduction, until in some cases actually being signed into law by the president, as many of you know, who follow Congress or worked in Congress or study Congress even casually, as you know, it’s extremely challenging to actually have a bill ultimately make its way to become law.

Alan E. Wiseman (05:10):

In some congresses, you might see as little as three to 4% of public bills that are introduced, ultimately being signed into law by the president. But that being said, there’s a lot of variation that occurs from introduction until ultimately conclusion at the president’s desk or conclusion earlier stages in the process. Some bills just get introduced and we never hear from them again. Some might be the focus of pretty meaningful and substantial committee activity. Some of them might perhaps be the focus of committee markup and might be reported out of committee might even pass either the House or the Senate, but then they die at that point. And what we want to do is come up with a metric that cumulatively captured the journey that every bill takes through each step in the process. So what we did is that we identified five status steps in the lawmaking process that we think are essentially crucial hurdles that every bill needs to pass through in order to ultimately become law.

Alan E. Wiseman (06:01):

And more specifically, what we did is identify for every public bill that was introduced into the United States House, for example, or pardon me for every member of the US House, how many public bills did she introduce? How many of those bills received any sort of action and committee? And by that, I mean, were they the focus of a hearing or were they the focus of markup or some other activities such as that? How many of those bills received some sort of action beyond committee? How many of those bills ultimately passed the chamber in this case, the US House, then ultimately, how many of them were signed into law? Now it also goes without saying that not all bills are, let’s say, equally challenging to try to pass the lawmaking process. It serves to reason that it’s probably a bit easier to advance a bill that deals with commemorative matters such as renaming a post office.

Alan E. Wiseman (06:47):

And I mean that literally not figuratively, in comparison to advancing legislation that could really engage with very meaningful reform for financial services or major healthcare reforms. And what we want to do is account for the relative substantive significance of every bill that’s introduced into the chamber across the two year session of Congress as well. So what we did then is every bill was coded in our data set as either being substantive and significant, which basically means is the focus of significant media attention across the two year period of Congress or two year session, a commemorative, which essentially means that it engages with a variety of commemorative matters, such as renamings, literally commemorative bills that strike commemorative coins or metals, for example, or other relatively salutary measures. And then all other bills, which is really the overwhelming majority of legislation that is introduced into two year session of Congress is denoted as substantive bills or substantive legislation.

Alan E. Wiseman (07:39):

So with each of these three categories, commemorative, substantive, and substantive and significant, and for every bill that goes through one of possibly five stages, (it could be introduced, it could advance through some action and committee, action beyond committee, passed the House or ultimately be signed into law) we essentially have 15 different data points for every piece of legislation that’s introduced by every member of the US House or alternately US Senate in a two year session of Congress. And then what we do is we use this data to generate what we refer to as a “legislative effectiveness score” for every member of the US House or US Senate across the two year session of Congress that serves to parsimoniously capture how successful every member of the chamber is and moving his or her legislative agenda items through every stage in the legislative process where we’ve adjusted or we’ve coded every legislative agenda item for relative substantive significance.

Alan E. Wiseman (08:34):

Some of you are familiar with our website, thelawmakers.org, where we present our scores publicly. And if so, you know that the average score of every legislative effectiveness score in every Congress is 1. We normalize them to make sure that the average score is 1. And what that means is that if a member of the House has a score above one, we would essentially say that they’re above average. Uh, if they’re below one, they’re below average, That being said, we also want to make sure that we’re presenting our data in a public forum. People are aware of some things that might naturally influence how high or how low scoring a member of Congress might be. So for example, as would probably be, as you’d probably expect members of the majority party score significantly higher than members of the minority party in our metric.

Alan E. Wiseman (09:20):

Which makes sense if we think about anything we believe we know about Congress, given that it’s a majoritarian institution given that members of the majority party tend to have disproportional access to resources and the legislative agenda, it should be unsurprising that they tend to be more successful on average the numbers of minority party. But we want to account for these tendencies, when we’re presenting our scores and the public. So people won’t draw incorrect inferences about who’s more or less successful, or why they’re more or less successful at advancing their legislative agenda. So if you look at our website and if you click on any state delegation, Tennessee, or Virginia, for example, you’ll be able to identify every member of the House and every member of the Senate that represented that state within a two year session of Congress, you’ll identify their legislative effectiveness score, just the raw score.

Alan E. Wiseman (10:06):

And we also present what we refer to as a benchmark score. And what that represents is essentially what we would expect that member of the US House or alternately US Senate to be scored as based on whether or not they’re in the majority party, whether or not they’re a committee chair or a subcommittee chair, and likewise how senior they are, how long they’ve been in the chamber, which is also highly correlated with how successful or how effective they are at advancing their legislative agenda items. And after we identify what each member of the House and Senate’s benchmark score is, and as I said, that’s essentially what we would expect them to be at given the relative seniority and position in the chamber. We can then identify whether or not they’re consistently scoring above their benchmark, mean they’re essentially over-performing expectations, or if they’re scoring below their benchmark, they’re essentially underperforming our expectations for them.

Alan E. Wiseman (10:57):

And in addition to presenting their raw score, their benchmark score, as well as identifying whether or not they’re above or below expectations, we also present where they are in respect to their total party. Meaning if we think about them being a member of the majority party, are they among the highest scoring members of the majority party, or are they relatively low scoring in comparison to all other members of the majority party? And we do the same for members of the minority party as well. And that hopefully presents you with a pretty basic nuts and bolts perspective, but where thee data comes from what we’re trying to measure, how we present the data, which we ultimately use in quite a bit of academic research, but also how we try to present in the public domain so that it conveys what we want it to convey.

Alan E. Wiseman (11:38):

And that being once again, a summary metric of how successful any individual member is of the US House or Senate at advancing her legislative agenda items from introduction until possible signing into law by the president where we’ve also accounted for the relative substantive significance of each bill in comparison to all other members of the chamber across the two year session of Congress. And just to build on the point that Craig raised, if anyone has any questions along the way either in response to things that I’m saying or what Craig is going to say in a second, and obviously we’ll have Q and A in a second as well, please don’t hesitate to use the Q and A function below, and we’ll just answer them in real time. But with that, Craig, do you want to pick it up?

Craig Volden (12:16):

Well, absolutely. I think it’s important to think about, right in this space, what the scores are and how they’re used. And so questions on that front are very welcome, you know, I’ve seen Alan present this dozens of times and I’ve presented it dozens of times as well. And so we know some of the common questions we have. And so I’ll just ask Alan one, now and see where he goes with that.

Craig Volden (12:40):

How have these scores been used?

Alan E. Wiseman (12:44):

Well, do you mean in the academic environment or in the public domain?

Craig Volden (12:48):

Little of both.

Alan E. Wiseman (12:48):

Okay. So I think building on the point that Craig was saying a little bit earlier, when him and I first started this project, I think it’s fair to argue that most scholars of legislative politics, which this might sound surprising to those that aren’t in academics, but honestly, most scholars of legislative politics weren’t really focusing much of their academic research and trying to understand why some members of Congress are just generally better than others at advancing laws. They engaged many other questions that were relevant to what we all think about as happening on a day to day basis in Congress. Many of them focused on trying to understand under what conditions are party leaders really influential, or are they influential in the contemporary Congress?

Alan E. Wiseman (13:26):

How does the division of labor through committees influence the course of lawmaking? But you know, when we started this project, which now it’s been about 12 years ago, compared to almost anyone else studying Congress at this point in time, we were relatively unique and trying to advance some pretty basic questions about what makes for an effective lawmaker. Why is it that some people are just generally better than others t advancing their legislative agenda items? What is it about people’s past experiences either in the chamber or before they even came into the chamber that makes them an effective lawmaker. So for those of you have any background in management studies or work in consulting or thought about doing so, you know, there’s a wide array of academic and practitioner based research that engages with precisely these types of questions and almost any profession you could think of, whether it be in athletes, teachers, management consultants, attorneys, but there’s very little academic scholarship to engage with these questions with regards to law making, and specifically with regards to Congress.

Alan E. Wiseman (14:22):

And I think it’s fair to say that’s changed pretty significantly over the past 12 years as we’ve engaged with this project. Our scores now are used very widely, both by scholars of the American Congress, scholars at state legislatures as trying to engage with these concepts and Craig will talk a little bit later about the ways in which we’re actually moving into this space to actually generate legislative effectiveness scores for essentially almost every state legislature in the United States over the past 25 years. It’s also the case that our approach has been emulated scholars of legislatures in other countries. In trying to think about ways to capture in a very objective way, how successful members of their own legislatures are advancing their legislative agendas. So from an academic perspective, I think what we’ve done or our work has really set the foundation for putting scholars and observers of legislative politics in a variety of contexts, and a very good position to start to engage in lots of these questions that, yeah, I think it’s fair to say many citizens and just observers of politics in the real world will think about a lot, or at least when pressed will think about it a lot.

Alan E. Wiseman (15:24):

Why is it that someone’s just good at their job in Congress? Why are they incompetent in some aspects, but very competent and others. And we’re in a position that we start to engage with these questions from a data driven perspective. In terms of practitioners in the real world, we’ve had lots of interesting interactions, I’d say 99.42% of them have been extremely positive in the sense that interested and engaged citizens and observers of politics have been very excited to learn about our scores and, uh, try to use them to inform their perspectives on politics. We’ve also engaged very directly with current as well as former members of Congress, themselves to either talk about what we’re measuring and the way this comports to their own experiences. In some cases, some are members of Congress have been extremely receptive to what we’ve been doing, and have actually been trying to draw some inferences about the lessons that we provide or suggestions we have for how one can become a more effective lawmaker.

Alan E. Wiseman (16:23):

One high scoring member who we actually profiled in our 2014 book is a strong proponent of our research. His chief of staff actually purchased our book and purchased copies of the book for everyone in the office. And they apparently had a workshop about trying to identify the ways in which, what we refer to as the “habits of highly effective lawmakers,” could or did apply to their own member and what he could be doing to enhance his lawmaking effectiveness. Outside of Congress, some of you might be aware that the scores have also generated quite a bit of attention in the public domain, especially in regards to election campaigns. The data that we use and our website actually went live in October 2014, right around the midterm elections. At that point, even though the data was in the public domain and really hadn’t gotten much traction with people who were running or thinking about running for Congress, but what we’ve seen is that cycle after cycle, after cycle, since then the data, the scores that found their way into the public domain, both in the sense that the media and by that, I mean, newspaper coverage, journals, the news, et cetera, are drawing attention to the scores and in respect to how to potentially evaluate candidates, incumbents, and their lawmaking effectiveness. High performing incumbents on our scale have definitely highlighted the scores in their own campaigns.

Alan E. Wiseman (17:40):

And in some cases, low performing incumbents have been the focus of criticism by challengers, focusing on our scores. Most recently, some of you might be aware that when Senator Amy Klobuchar was running for president, she actually highlighted her legislative effectiveness score in a variety of contexts. In our most recent scoring of representative and senators through 2018, Senator Klobuchar was actually identified as the most effective democratic lawmaker within the United States Senate. She made this a point of pride in her campaign as a point of illustration as a way that she could be counted on to be a very effective negotiator and bargain maker to work with the legislature in a constructive way. This actually became the focus of one element of the New York Times endorsement of her and Senator Warren, where the Center for Effective Lawmaking was actually cited as grounds for why they were choosing to cite Senator Klobuchar. And, as Craig knows, we also received some great pictures from the Iowa caucuses where there apparently were campaign shirts for Klobuchar bouncing around for awhile, where identified that she was the most effective democratic lawmaker in the United States Senate. Sadly, neither of us got any of the swag that came with that, but we’re still hoping. And other than that, Craig, are there any other aspects you’d like me to flesh out?

Craig Volden (18:59):

So Trudy asked a question about the use by pollsters, as well as the use by a campaign. And I think you covered most of that… In terms of how people are using this information during elections. It started out kind of the way we’re pitching it, which is, you know, our top 10 lists and the highly effective lawmakers and so on. Within a cycle or two pivoted to negative ads against those who were doing poorly on our metrics, a sort of classic how’s American elections playing out, I would say on that front.

Alan E. Wiseman (19:33):

Yeah. I think that’s a fair and accurate statement. Yes.

Craig Volden (19:37):

Yup. So then I wanted to turn to basically this broader sense of how we at the Center for Effective Lawmaking are using these scores and largely we’re engaged in a variety of research endeavors. So I’ll talk broadly about our Building a Better Congress project. And then we can turn to how that information and not just the scores alone have found their way into the public dialogue. So in terms of our Building a Better Congress project, we’re engaged in a lot of separate kind of research also projects or endeavors, in three main areas there. The first is what we referred to as identification. Could we identify the characteristics of people who if they were to choose to run for Congress and be elected would likely be more effective once they get there? Now, because members of Congress are public figures and we know a lot about them, we do know about their backgrounds.

Craig Volden (20:36):

And so we’re able to see what are the backgrounds of people and how did those affect how they behave as a effective or not so effective lawmakers. And our findings,, for example, point out that on average women are more effective than are men, as one example. And we find there is that women, especially when they are in the minority party, keep pushing their issues forward and keep building coalitions across party lines in ways that men on average don’t do as much. Well, what this points to then is a variety of organizations, like She Should Run or Running Start that are trying to encourage more women to run for Congress, that they can make a strong case that it may be difficult to run, and there may be a lot of obstacles up front.

Craig Volden (21:24):

But once you get there, you can make a difference. And that’s an appealing message for women who are thinking about running. Also in this identification space, we’ve been doing work looking at those who were in state legislatures. And there we find that service in a professional state legislature, the ones that meet year round and have staffs and pay enough that it’s their full time job. Once they get into Congress they tend to be more effective than other members. Unfortunately those who serve in citizen legislatures like we have in Virginia, that aren’t meeting much and don’t pay much and don’t have a lot of staff for lawmaking, once they get into Congress. We find that they’re not any more effective than anybody else and, in fact, look a little bit less effective than those who don’t have state legislative experience at all.

Craig Volden (22:12):

And there, when we talked to them kind of qualitatively, we get messages back like, “well, Congress seems to work a lot differently than our state legislature.” And in many times it’s, “I wish Congress were to work like our state legislature.” We would argue that they need to kind of pivot at that point and say, well, you’re no longer in your state legislature. And indeed it is quite different and if you want to be effective you need to kind of follow along with the habits of what’s necessary here to succeed. We’ve also been looking at educational background and prior employment. There’s some work that the Center published this summer on veterans, those who served in active duty post 9-11 seem to be more effective once they’re in Congress. And so in that identification category, there’s a lot of work and a lot of kind of interesting findings. Our second big category of research for the Building a Better Congress project is what we refer to as cultivation.

Craig Volden (23:10):

Now, you’re in Congress: What could you do to cultivate an agenda that makes you more effective? And there, we base that on a variety of research projects, one of them being out of our book, we identified the 20 most effective lawmakers from the past 40 years in the House of Representatives. And then we did a deep dive into looking at what they did differently than other members of the House. And from that perspective, we identified five habits of highly effective lawmaking that we’ve written up and talked about in various formats. They range from building a legislative agenda based on what your district cares about in order to align your electoral goals with your lawmaking goals, to building on your expertise and passion and why did you get to lawmaking in the first place, to really being entrepreneurial in Congress and building coalitions and how that process is done.

Craig Volden (24:07):

Also in this cultivation space, we’ve done research on different staff that especially freshmen members instead of hiring their campaign staffs, if they hire people who have served on Capitol Hill in a staff role for a decade or so, they can hit the ground running and really be more effective lawmakers. But also in this space we’re talking about cultivation of Congress as an institution to get things done. There, I highlight for example, the difficulty that subcommittee chairs and committee chairs have in just wrestling with their material in gaining expertise there in helping their committee tackle major public policy problems. And our data seemed to show that it gets better and better at being a subcommittee chair or a committee chair over time. But as Congress in the mid 1990s put on term limits on committee chairs and subcommittee chairs, a lot of that expertise fell by the wayside. Chairs and subcommittee chairs were really getting their committees going really quite well by their third term and fourth term.

Craig Volden (25:14):

And that’s about when they were term limited out – after the third term in the process had to start all over again. Lots of loss of expertise in Congress based on those sorts of activities and reforms. And then our third area, moving from identification, who’s an effective lawmaker, or would be an effective lawmaker if they were to run to cultivation, what’s going on in the institution. Our third category there is what we refer to as accountability. Do voters know how effective lawmakers are? Do lobbyists? Do campaign contributors? And how do they base their behavior on those effective or less effective lawmakers? There we find, for example, that on average, most people don’t know how effective their lawmakers are. But when they’re getting information, they’re going to process that in kind of some interesting ways. If the information comes from the campaigns themselves, voters seem to be dismissive. If it comes from opponents voters seem to be dismissive. In other words, that’s just biased information. But when it comes from a source like we have at the Center for Effective Lawmaking, it seems to be taken more seriously. And that’s probably why campaigns have used a lot of our information to provide that credibility to what they’re announcing out there. So that’s the broad overview of the project.

Alan E. Wiseman (26:33):

Craig, there’s a question from an audience participant asking, which of these or what of these research projects are you most excited about that’s?

Craig Volden (26:44):

That’s great! So whatever I’m doing today, I think I find most exciting. And so, today, we’re working a lot on state legislators. So in addition to thinking about broadly, who are the classes of state legislators who do well in Congress, and we find those who served in professional legislatures, we’ve had a project going on with our post-doctoral scholar at Vanderbilt Peter Bucchianeri, who has looked at every state legislator in almost every state. We have some difficulties in Kansas where people don’t take credit for their bills or avoid blame for their bills. But in 49 other States over the past decade or two we’ve been able to score those state legislators. And then we’re able to see, you know, does doing really well in the States transfer into doing really well in Congress?

Craig Volden (27:38):

That’s classically in our identification category. And it does, particularly in those professional state legislatures, but also we’re able to see across the States, what institutions really help gain effective lawmakers? What are the ones where minority party ideas are taken seriously and move through the process or where committees seem to be working really well or whereby the time you’re in your second term, you’re already contributing well, despite having to ramp up pretty quickly to learning the lawmaking process? And those types of States that are doing really well tell us about some potential institutional reforms that we could see either spread across the States or be adopted in Congress. In terms of how these types of projects are used then,, we take our Habits of Highly Effective Lawmakers, and we talk with members of Congress about them.

Craig Volden (28:32):

We’ve taken the advice that a variety of organizations give to new freshmen members of Congress. And we kind of have distilled that down to “well, which pieces of advice are about effective lawmaking.” And some of that advice is followed and some is not followed. And, you know, based on our scores, the people who are following that advice in some cases do much better. And in other cases there’s not much difference. And so that advice is something that we wouldn’t really endorse by the Center. There are also reform proposals going on before Congress, they have currently a Select Committee on the Modernization of Congress, and we’ve produced some recommendations to that Committee, and spoken at length with our staff about our proposals there in our research.

Alan E. Wiseman (29:25):

There’s another question asking how much the variation in effectiveness outcomes are you able to explain? Are there potentially important determinants in effective lawmaking that you’re unable to quantify? So that’s the first question. Do you want to handle this or do you want me to take a stab? I’.

Craig Volden (29:41):

I’m sure that, I mean, that goes back to the scores and so you might want to talk about that. But you can tell them sorts of things that are in our models that help explain it.

Alan E. Wiseman (29:52):

Sure, sure. So, in response to that question the first part of it, how much of the variation and effectiveness outcomes are you able to explain? The most direct way to engage that question is to basically run pretty reasonably straightforward statistical analysis to see which variables that we have consistent and clean data on correlate with our legislative effectiveness scores. So there’s a variety of things that we would expect to be correlated with legislative effectiveness, building the points I was saying before .We’d expect people in the majority party to be more effective than those, the minority party. We’d expect the committee and sub committee chairs to be more effective than those who are rank and file. And would likewise would expect more experienced more senior members of the chamber to be more effective than those who are relatively junior. And in fact, we do find this to be the case. More broadly speaking, if we collect roughly about 15 variables that we now view as kind of the standard battery or standard correlates of legislative effectiveness, and look to see how strongly they predict the legislative effectiveness scores I would say depending on the Congress that we’re thinking of anywhere between about 45 to 60% of the variance and legislative effectiveness scores is usually captured by this battery of variables that we look to explore. That being said that there’s a lot of other variances out there, obviously. I mean, it’s not… In a perfect world we have a model in which we can predict 70, 80, 90% of the variance in legislative effectiveness based on observable indicators. So from our perspective, there’s lots of things that are out there that could be influencing legislator effectiveness. A consistent theme that we hear, or we experienced based on our own research and also talking to the members themselves and their staffs is that there obviously are activities that happen behind the scenes, for which we just simply can’t observe them, or alternatively can observe them in a consistent way that we can actually code up the data in a way to measure it.

Alan E. Wiseman (31:52):

That’s not to say that they’re not important. We unambiguously believe that there are some individuals who are either themselves working behind the scenes for their own legislative agendas and such ways that we can measure it. Or alternatively, they’re aided by others who are working behind the scenes, such a way that we can’t systematically measure those. So from those, from that perspective, this suggests we need to become a bit more creative about identifying what is going on with these members. That’s not particularly are easily observable. Looking back to our 2014 book, for example, for all intents and purposes, this sort of served as the foundation for the sixth chapter in that book that we titled the “habits of highly effective lawmakers”. Because what we did in that chapter is we took a look at our entire sample, and that was looking at all members of the United States house of representatives in 1973, up until 2008 at that point in time.

Alan E. Wiseman (32:42):

And we had demonstrated in the earlier parts of the book that, okay, we know if you’re a committee chair or we know if you’re a party leader, you’re a majority party leader, you’re going to be relatively more effective than the rank and file or people that aren’t committee chairs. That being said, let’s say we throw out all those people from the sample. So now we’re only looking at people that are essentially rank and file members. They didn’t hold a committee chair within a given Congress, they weren’t majority party leaders. If that’s the case, if we’ve thrown away the most obvious predictors of legislative success, or legislative effectiveness who’s left and are there people that consistently emerge as being high performers, Congress after Congress, after Congress. And what we found is even after you throw away all the legislative leaders and all the committee chairs, there are a handful of people, and we identify 20 of them, as the most effective rank and file members of the House across this data period from 1973 to 2008.

Alan E. Wiseman (33:38):

And then what we did is we dove into this, this data, this very small sample 20 members to try to identify across their careers, what strategies did they employ, or alternatively, what were the set of circumstances they found themselves in that contributed to their legislative effectiveness, given that for all intents and purposes, they were essentially outliers? There were people who were consistently high performing as effective lawmakers, even though they didn’t clearly benefit from any obvious institutional advantage or leadership advantage. And in that chapter, we identified the “five habits of highly effective lawmakers.” It’s based largely on qualitative analysis and to some degree is somewhat suggestive. What we’re saying upfront that if we look at this relatively small sample, these are patterns of behavior that some of them well, all of them engaged in at least one of these patterns of behavior and some of them employed multiple patterns of these behaviors. And we provide this as some suggested guidance, for some other strategies, the lawmakers might want to think about employing themselves. And to some degree that really served as the research foundation for a lot of our projects right now, where we’re using those habits that we articulated in 2014, as ways to motivate pretty substantial data questions to think about what data could we collect that we could use to more systematically explore these habits. And that’s where we’re going right now.

Craig Volden (34:54):

Yeah. One example there is a, um, you know, the habit that says build unexpected coalitions. Well, we didn’t know who was doing that and who wasn’t until we engaged in a research project, then that looked at co-sponsorship networks. And how much do you co-sponsor bills by members of the other party, are you willing to be bipartisan along those lines? And do they in return co-sponsor your pieces of legislation? The finding there is really intriguing that by reaching across the party line for co-sponsoring other pieces of legislation by members of the other party, they’re more likely to return that by co-sponsoring more of your legislation and those who are able to attract, or we know already, if you attract a bunch of co-sponsors, you probably have a pretty good piece of legislation moving forward, but within those co-sponsors those that have a larger percent from across party lines do a lot better, are more effective and are more likely to move those, those bills forward.

Craig Volden (35:53):

Also in this space you know, things about what expertise do you have, what background do you have that points to data we’re collecting on district characteristics and on member backgrounds and really leads us to think about how do you develop a legislative portfolio in your early days in Congress? Is it really based on your backgrounds and your district? Does it get bent a little bit by those who engage in lobbying or give you a campaign contributions, as we’re seeing throughout the lawmaking process. That idea about entrepreneurial behavior is a tough one to wrap our heads around exactly. There’s some interesting work going on right now by scholars at other institutions who are looking at those who are very active in engaging in oversight because there are some elements to which we might think, well, you’re going to spend your time somehow and inside they’re going to be no oversight or it’s going to be in lawmaking. And so those might be substitutes, but a lot of oversight also is talking about the developing of expertise that’s going to lead to better lawmaking in the future. And so scholars are trying to tease out those who are entrepreneurial in both areas and what makes them unique.

Alan E. Wiseman (37:10):

And then Craig loosely related to some of the points that you raised a second ago, but could probably maybe articulate this a bit clearer: someone else also asked a question or a comment saying how you had mentioned that you provide advice to members. Are there examples of advice that you tend to offer new members or alternately members who want to increase their scores?

Craig Volden (37:31):

Right. So we’ve engaged in a broad effort to talk to all sorts of members of Congress, letting them know if they’re doing well on our scores. And many of them take advantage of that. Letting them know different ways to improve their metrics over time. But we think that there’s a big value of talking to new members of Congress and a variety of organizations set up meetings with them. So, in 2017, we were part of the new member seminars that were put on by the Congressional Research Service in 2019. We worked with the Library of Congress in their new member activities. And we’ve produced and now updated, are working on fully updating a new member guide, that people can visit on our website, but that’ll we’ll also put in the hands of every new member of Congress.

Craig Volden (38:23):

And so we want to make sure that that information is really valuable to them. And it’s a little of everything. So, just a couple examples. I mentioned the, you know, crucially, who are your staff members going to be? Those who have served in the Congress for a long time previously with other members are going to know the ins and outs and really help people hit the ground running there. But also early decisions are what legislative portfolio are you going to have? And we talked about developing that as sort of a sweet spot between three things. One is kind of, what does your district care about? That’s going to allow you to run on that, on your legislative accomplishments back in your district. It kind of aligns the time that people often feel is split between their campaign activities and their lawmaking activities.

Craig Volden (39:13):

And second, what do you know about, or are passionate about, or what’s your past job been? And then the third category is your committee assignments or subcommittee assignments. And really if somebody finds that sweet spot between what they care deeply about, what their district cares deeply about, and what they have some influence over, because they’re on the committee or subcommittee, that’s where they should dedicate their legislative proposals. We’ve found a lot of evidence of that but also a lot of evidence that members of Congress don’t do that. Often they’re much more scattered and don’t gain up the expertise that they need. There’s a lot of evidence that suggests that members of Congress that are experts and are considered experts certainly get a seat at the table, certainly have other members come to them, certainly get their proposals moving forward more critically. And so we’re engaged in a big project now about the development of that expertise.

Alan E. Wiseman (40:02):

Yeah. And building on some of those points and this fits in very naturally with Craig description with Building Better Congress project, a lot of our efforts right now, research efforts that are focused to split the.. Pardon… Especially around the cultivation theme, and really trying to understand what members can do once they actually get into the chamber to become more effective lawmakers really serves as a foundation for the advice that we offer to any members and their staffs. So as Craig was noting, a big part of our advice focuses on issues of specialization or trying to cultivate one’s legislative portfolio to reflect to their own specific needs or the comparative advantage or alternative that their district needs. But moving beyond that, and this is closely related to some other projects that Craig made reference to earlier, we’re also trying to ask or engage with very explicit questions regarding the scope of specialization.

Alan E. Wiseman (40:55):

To what degree should members actually be focusing on a handful of topics rather than spreading themselves reasonably thin? And different members and different staff members are gonna have different views on this. But the research that we’re working on right now, and we’re actually in the process of finalizing for presentation at a national meeting, the American Political Science Association next month suggests quite directly that there is a very much a sweet spot regarding the scope of one’s legislative portfolios if someone wants to be a highly effective lawmaker. And us trying to get this message to newly elected or relatively junior members of the House or Senate provides them with hard data driven insights that suggest the conditions under which they’re gonna be more or less effective at advancing their legislative agenda, depending how far they decide to spread themselves so to speak.

Alan E. Wiseman (41:45):

And as Craig noted a big part of what influences the relative effectiveness of lawmaking, especially among relatively junior members, is going to be the way in which they populate their offices. And even though some of the lessons that we’re pointing to or speaking to definitely have, or is the source of quite a bit of anecdotal support, we’re really among the first scholars that could provide large sample evidence to guide people on their employment decisions as to who they should be bringing into their office, what type of work experience these people ideally should have had on the House or in Congress prior to coming into our offices, or alternatively, the potential consequences associated with hiring a relatively less experienced staff. Beyond that, there’s a question that just came up a second ago…

Craig Volden (42:31):

Let me build on yours before we go to that question, because, you know, we mentioned that we’ve scored members of the House and we’ve scored members of the Senate and we scored state legislators. Some of the work that we’ve been talking about is also based on some new metrics that we have coming out where we’re scoring every member of the House and every member of the Senate on 19 different issue areas. So who is really good on health, who’s really good on education, w ho’s really good on what a variety of people care about because they have policy interests or policy expertise there? And out of that, we’re able to find who are the experts and many times they’re who we would expect to share – The chair or the subcommittee on health does really well on health. But in other cases there are people who kind of work behind the scenes or who might not be as high profile that we’re able to detect are really high performers in and out.

Craig Volden (43:21):

There are people on the agriculture committee that do really well and others that do less well. As such, there’s a lot of information that we’re able to provide there about maybe who you would want to contact if you’re interested in proposals or what’s going on in particular policy areas, to get that information into a variety of people’s hands, we’re going to make that publicly available. But our first presentation of that is actually coming up on August 14th, as part of the Batten X program. So people who are interested in knowing a lot about specific issue areas and who are the movers and shakers in those issue areas, that’s an opportunity to find out more about… We put up the link to that event in the chat section but you can also find about that on thelawmakers.org, our website, as well as the Batten school’s website. So that’s another opportunity for an in depth dive specifically on who gets things done in which issue areas.

Alan E. Wiseman (44:23):

Great. So there’s another question that is taking slightly different direction, someone’s asking us whether or not we’ve looked at correlations between campaigning and legislating, and more specifically, this person asked issues regarding say, campaign style or resources, election returns such as margin or alternative of this status of incumbency. Um, so,

Craig Volden (44:43):

Well, it’s kind of the flip side of what we were talking about before, right? We were saying members who are effective might be more likely to be reelected, now it’s what about their election status leads them to be effective or not.

Alan E. Wiseman (44:53):

Right. Right. And related to that point that speaks broadly, well, I guess we could think about that on both sides of the equation, it could be a function of identification. Is there something about the ways in which people campaign that perhaps then maps into how they ultimately become effective lawmakers? And thus far, we don’t have research findings to-date to point to, but that’s an ongoing project that we actually have a team of research assistants engaging with in which they’re coding up essentially campaign ads or the ways in which individuals present themselves during the campaign. Are they presenting themselves as dealmakers or people that are committed to the lawmaking or public policymaking process? Or alternatively, are they embracing more partisan or mano partisan perspectives? And really not thinking, not articulating anything with regards to the lawmaking process, more broadly considered. How does that relate to how they ultimately work once they get into office and the subsequent outcomes? With respect to the relationship between electoral outcomes and lawmaker effectiveness:

Alan E. Wiseman (45:56):

We do have some findings to point to on that. With respect to general election outcomes, what we’ve demonstrated as some of our earlier research, those published in our 2014 book, and it’s continued to reemerge as we’ve expanded the sample to include both the Senate as well as longer periods of time, is there appears to be a nonlinear relationship between one’s electoral security and how effective they are in lawmaking, at least in our scale. And bluntly stated those individuals who are extremely electorally vulnerable, I mean, they’re all, they’re basically running scared, so to speak, and likewise, those who are super safe, they’re either running in uncontested elections, or they’re still getting 75 to 80% of the vote share or above, these people tend to have lower legislative effectiveness scores controlling for everything else than those who are essentially in this electoral sweet spot, these people who are receiving vote shares somewhere in the mid sixties.

Alan E. Wiseman (46:54):

So this raises a lot of questions as to why might that be? Is the case that those who are electorally insecure and those who are very electorally insecure, do different things that contribute to, or alternately detract from, the relative legislative effectiveness compared to those who are in the sweet spot? And these are some of the questions that Craig and I have been exploring. One finding that has, or a handful of findings that have emerged from new research that we currently have under review speaks to the relationship between legislative effectiveness and electoral outcomes and primaries. And there we do uncover some very meaningful and Craig, would you like to fill in some of the details on those?

Craig Volden (47:33):

So it seems like the reason why effectiveness actually affects outcomes in primaries is kind of twofold. One is that voters seem to be a little bit more aware of who their members are and what they’re doing. And the other is that that kind of partisanship is left at the door because these are primaries within each party. You’re not solely going to vote for members based on “do I like their party? Am I a Republican? Am I a Democrat?” But you’re going to base it on other characteristics. And what we found is that highly effective lawmakers are more likely to scare off challengers. So that’s a big first finding, that those who are more effective are getting reelected at a higher rate on average and that by limiting the field, in some ways, those effective lawmakers are able to make their case more dramatically. In larger fields there are so many different issues being discussed that effectiveness is just one of them but in smaller fields, the effective lawmakers can really raise that as a consideration. So pretty strong evidence emerging that primary challengers and primary voters are taking effectiveness into consideration by the general election time it tends to be a much more partisan show based on people’s party alignments. The other thing I’d say in this space was kind of an interesting finding that you were able to dig up Alan when we were looking at members of the Senate and those in there because they’re serving six year terms those who are in their first two years and second two years are different in some ways than their third, final term.

Alan E. Wiseman (49:16):

No, that’s very true. And more specifically, the point that Craig was raising is that those Senators who are actually up for reelection, so those who are in the last two years of their Senate term are actually more notably, more effective lawmakers, holding everything else constant than those who are earlier in the electoral cycle, which suggests that Senators are engaging in different legislative strategies, or at least advancing different types of legislative agendas when they’re coming up for reelection and they’re going to potentially face judgment by the voters. Which if you think about it, then suggests that are is consistent with the argument that Senators at least believe that their law-making effectiveness matters … that their lawmaking effectiveness record matters in some ways in the eyes of the voters, at least to be consistent with that argument.

Craig Volden (49:59):

Right. I mean, because that’s the time when they otherwise might turn away from lawmaking to electioneering. But no, they’re more dedicated to lawmaking at that point in time, presumably because they’re thinking it’ll be fresh on voters’ minds

Alan E. Wiseman (50:12):

Very much. There’s a question that asks, have we gone back historically and how far to look at prior generations of legislative effectiveness? Are the trends in causes consistent over time, any notable trends? So that’s a great question and let me answer that in a couple of ways. One is we have explored the entire data set to look, to see if there’s any notable trends across some of our variables of interest and more specifically, the value of majority party status, for example, in the chamber, or alternatively, the value of being a committee chair or subcommittee chair over time, or whether or not certain groups were perhaps enabled or alternatively inhibited in advancing the lawmaking effectiveness.

Craig Volden (50:56):

And again, that, and go back to the 1970s and up to the present, right?

Alan E. Wiseman (51:00):

That’s exactly where I was going to go next. So, for good or ill we’re unfortunately constrained in our ability to get data before 1973, because it’s not easily accessible in electronic form from the Library of Congress. So even though many people ourselves included, would be really interested to think about how the contemporary Congress either today in 2020, or even going back to the aughts so the 1990s compares to say the 1940s and the new deal congresses, we’re just really not in a position to engage with that from an educated perspective. That said, if we think about 1973 to current day, I mean, we’re looking at it almost 50 years of data. So we are in a position to assess how, if at all devalued majority party status has changed, and that in itself has remained pretty constant. Members of the majority party on average have always been more effective than members of the minority party.

Alan E. Wiseman (51:49):

One very interesting finding that we can demonstrate very cleanly, especially in the House, is that the relative value of being a committee chair in advancing one’s legislative agenda has clearly changed from say the seventies, eighties, and even early nineties up until today. And, Craig made reference to this point a little bit earlier, what we’re able to demonstrate or find in the data, and it’s a very robust pattern, is that the marginal value of being a committee chair and advancing and subcommittee chair, at advancing one’s legislative agenda starts to decrease pretty significantly in the late nineties. And those of you who follow Congress have studied it, or just observe it’s, especially those of you that were thinking about Congress back in the late nineties, will remember that beginning with a 104th Congress, which that’s 1995, for those of us who don’t speak Congress-ese when Republican speaker of the House Newt Gingrich was elected, one of the first reforms that he pushed for is the implementation of term limits for committee chairs. And more specifically members of the Republican caucus could only hold onto their committee chairs for three congresses, for six years. What we find in our data is by implementing those term limits, or at least as a correspondence to the implementation of those term limits is that the relative of lawmaking effectiveness as a committee chair has started to decrease pretty significantly the time thereafter. And it also corresponds to the ways in which the Republican party leadership, at least early in the mid 1990s, tried to really reorient the lawmaking process in the House so that what we think of today as authorizing committees, so these are the committees that handle substantive legislations, such as agriculture, rivers and harbors broadly defined or other broad substantive topics that we’d all think about is what a bill that provides for something, the Republican leadership actually tried to shift lawmaking from those committees to the Appropriations committees, the bill to the committees that really handle the cutting of checks, so to speak in a way to try to centralize control. And what that also did is influence both the relative attractiveness of being the leaders on these committees, as well as their ability to navigate the lawmaking process. And to some degree we’re able to demonstrate that the imposition of term limits on committee chairs, which is a policy that Republican Congresses still endorsed over time led to a pretty systematic decrease in the scope of expertise that these committee chairs had, which probably compromise their lawmaking effectiveness. And it also clearly changed the intra-chamber dynamics as to how people chose or sought to rise through the party leadership to ultimately obtain committee chairs. Craig, are there any other points you’d like to add?

Craig Volden (54:22):

I mean, there’s so much that came up that was just so consistent from Congress to Congress that gave us comfort that we were finding the same things over and over: women in the minority party being more effective and then findings from the House, the Senate of the same patterns were showing there and so we know a lot about lawmaking from these and in very consistent ways. I was drawn towards seeing some of the coalitions that change over time as Southern Democrats faded and they were kind of pushed aside by the rest of the Democratic party and then the rise of Republicans in the South, all of those patterns are reflected in our effectiveness scores.

Alan E. Wiseman (55:02):

Very much so. There was a question: as a direct follow-up to the point about the Senate asking might late in their term, senators began disproportionate assistance from Senate leadership during that period? Just as a followup, remember, a few minutes ago, I had pointed to this empirical finding that we uncovered which was quite robust that those senators who were in their last two years, meaning they’re up for reelection, pardon me, essentially were more effective in their last two years than they had been in the previous four years. Craig, do you want to flush out?

Craig Volden (55:37):

Yeah. I mean, this is a broader question of are people gaming the system in some ways? If you introduce a thousand bills, do you do better on our score, or if you’re a first term House member who’s coming from a vulnerable district, do you get the support of your party leadership? And I have no doubt that there’s some of that going on, lawmaking is nothing that an individual can do by herself. This is something that’s a collective enterprise and working with the committees and working with the party leadership, and so on, is necessary to move legislation forward. And as such, we know that the parties have an interest in keeping their members reelected and building their coalitions and staying in the majority party.

Craig Volden (56:22):

And we were a little fearful that, if our scores go out there and people are hearing about them, people are going to try to game the system. We haven’t seen any of that, but we are able to detect some of these sorts of, patterns that were already going on. And so majority party freshmen in the House who are from vulnerable districts do seem to get a little bit more of a leg up in moving their legislation forward from party leaders, seems like that pretty natural, and possible in the Senate as well. When we were actually looking at the last round of data, we saw the kind of opposite version of that where a number of, pieces of legislation that came from Democrats in the Senate were held on the floor or held off the floor essentially of the House until after the election, because they didn’t want those Senators to get any credit for that legislation. Eventually many of those pieces of legislation were valuable and move forward and found their way into our scores. But not until after the election, they didn’t want members of the other party to get credit for that, finding those sort of games being played around elections are true regardless of whether or not we have these scores out there.

Craig Volden (57:51):

And that brings us to the end of our time. So I did want to thank Alan for spending this time with me talking about the legislative effectiveness scores and the work of the Center. But thanks to all of you who are watching, either as a recording or visiting with us live today. Thanks for all the questions. And as always, if you want to learn more, go to our website, thelawmakers.org.

Alan E. Wiseman (58:13):

Thank you, Craig. And we look forward to hearing any questions or comments you have as follow ups. We’d love to continue to engage with all of you. So thanks a lot. Thanks in advance for your thoughts.

Craig Volden (58:24):

Thanks so much.