I’ve actually been a captain in Ultimate Frisbee in a league (in Boston) that did a draft like this. Can you explain in more detail about the baggage thing?
This was 12 years ago, so my memory is fuzzy…but I think it’s possible that in a situation where for example a 13 baggaged with a 17 - I would pick the 17 in an early round and then when we got to a round where 13s were being picked, I wouldn’t get a pick that round (because I’d already have the 13 that came as baggage). Does that seem right? Or am I making stuff up?
My understanding is that when choosing either of a pair of baggaged players, you get both players at that point and skip your next pick. Also, the pick order every round is from the team with the fewest total points at that point to the one with the most points. So, picking a high-high pair may mean you get a later draw even after the skip, while a high-low pair might yield you first pick after the skipped round. There’s a limit on the number of players that can baggage together (2 or rarely 3), and they must be mutual (i.e. no chaining).
Boston might have a different system, but the way Pittsburgh traditionally has done it, you lose your next round’s pick regardless of rankings. So if you pick up a 17 and a 13, you don’t get a pick next round even if the players are still 17s. This means that sometimes baggaged pairs are picked in later rounds than their top skill would indicate.
The team totals for skill are figured at the end of each round and that affects the order the captains pick players in the next round (assuming they have a pick).
For ultimate rec league teams, it’s important to have the same number of people (and consistent numbers of women and men) on each team as well as to foster skill parity. I’m not sure number of players would matter in a pinball league, although each team should probably have a minimum.
For the Pittsburgh Ultimate draft system, we initially used self-rankings. Later we added oversight to make sure rankings were reasonable, and when the community grew enough that we didn’t all know each other, we started recruiting people to help revise rankings before big drafts.
I haven’t captained for a few years, but this is about how it worked when I did.
Thanks @ErinK, that does sound similar. I still think an A and a B division would be helpful…even if the team as a whole is equal, having a wide disparity of skills on a pinball team could mean that many matches are between two very unequal players. In past season, captains have often sacrificed weaker players against the strongest opponents. If their #1 is better than your #1, then don’t waste your best player on what will likely be a loss anyway.
Do players ever lose levels in this sort of system, over time? I wonder if that is upsetting for a player. If pinball had something similar I’d guess players would gain and lose levels frequently.
My feelings on team leagues, as a Seattle Monday Night Pinball player, is that the point of the team league is to have a good time with your friends. Our team league has grown massive and a big part of that is people bringing friends who don’t normally play in tournaments to come join their team and hang out. The idea of a draft for the league or any tight restriction of who can be on what teams is a huge turn away from any team league to me. This is why Monday Night Pinball has its restriction set up, and why we have the “grandfather” rule allowing teams to continue to stay together as players improve. Does this lead to teams being better than others? Yes of course, but you are never going to have a perfectly even distribution of skill across teams, and I find that home vs away matches does an all right job of balancing out. Even the best teams in our league still lose and most often they lose away games. No matter how you restrict team selection you will still end up with some players having to go up against better players and getting crushed. You will still end up with teams who aren’t as strong losing more than they win. That is pinball and competition. Might as well let people have fun with their friends. Almost every match I see pretty much everyone having fun win or lose in Seattle’s league. Also, the fact that some teams form up with friends who aren’t as skilled come in and get crushed really only seems to foster an environment of those people wanting to go out and play more to get better. Every single team in Monday Night Pinball has risen to the challenge and grown in playing skill massively over the last few years. I think you might find that imposing a draft with all these restrictions would shy people away from the league if they can’t play with their closer friends, and it would lead to people being less inspired to go get better because all the teams are already even, and if you get better, you might be booted to another team. I for one would quit MNP if my team was not allowed to stay together just because we are good.
I’m not excited about a draft anymore. But I will probably still do a salary cap.
We currently use your exact system, with the same restrictions.
This is not true in our league. Last season everyone played every other team both home and away, and every series was a 2-0 sweep.
So, it sounds like you’re on one of the best teams? I’m getting that in my league…players from the top two teams say: “it’s fine, don’t change anything”…meanwhile the other four teams don’t seem nearly as excited. We dropped from 7 teams, to 6, and an are in danger of losing 2 more for the upcoming season.
I want people to be able to play with friends, but, just as you described, I want something that is welcoming to new players. We’re not getting new teams or new players…just thinning down to the existing group of dedicated players (who already play in all of the other local events).
I’m hoping that a B Division, and smaller teams will help with that.
Lose levels over time in ultimate? Not usually (and not automatically). They review the rankings every season/league but it’s a pretty small scale (I think there are maybe four or five qualities and a 5-point scale, so your skills and experience might still be the same but your athleticism would go up and down.)
Oh, and the player rankings we use aren’t public. They’re only used for the draft, which is private. So you don’t get to obsessively compare yourself against the other players if you’re not a captain or league administrator. And it’s not public knowledge who gets picked in which round.
In addition to the previously mentioned arguments against a draft based system (not a chance in Hades I’d be willing to drive to Renton or Edmonds every other week), I think it would kill off a lot of the fun traditions that form when teams persist across multiple seasons.
That said, along with the traditions does come a subtle but definite cliqueness. I’d be interested in seeing a mechanism that kept team legacy while still encouraging teams to rotate through players each season, and encouraged players have more interaction with other teams.
Just brainstorming, but maybe something to the effect of requiring a player to sit-out, transfer, or sub one out of every n seasons could be interesting. Combined with a salary cap, the churn should keep teams fairly balanced while not killing off the chance to build a team legacy with your friends.
Our ultimate group also runs some leagues where you specify a core group and then draft about half the team. That wouldn’t work super well for a 5-person team.
There have been teams that stayed together for multiple seasons during the draft days, but it required a lot of finagling. I didn’t mention that trades are also allowed, as long as they happen immediately after the draft. So I remember one season where another team specifically drafted a player to mess with the team who wanted her, then her team tried to pick good players to trade for her. (I think they worked it out.)
Bumping this as I’m going to start a new season next month. (This league hasn’t been run since my original post).
Refreshing my stats, I think Rating currently fits this group of players more closely than Eff %.
Now, the goal is to figure out the math on how to use Rating for a salary cap. If anyone has thoughts on how one might translate Rating into a different scale that quantifies the difference in ability in a less abstract way, please do chime in.
(For example I do think Eff % was close in this regard…someone with a 20% would win about twice as often when playing someone with a 10%).
Seattle’s Monday Night Pinball put together a neat rating system (IPR) that uses a composite of IFPA and Match Play Ratings: http://www.wapinball.net/mnp/ratings.htm
The general gist is that a team is capped at a total of 40 IPR.
I don’t think Michael is pulling in Match Play Rating data for all players on the website. At least the MP column is blank for many players. My server thanks him
Caps for team leagues is a great example of where combining IFPA and MPR data makes sense. Since IFPA doesn’t endorse team leagues there are often team league players who have low IFPA ranking despite their skill level.
I have to stress that the MNP system works because Michael is doing a ton of work to ensure that every Seattle tournament is submitted to the Match Play Ratings. If you go to https://matchplay.events/live/ratings/submissions you will see literally hundreds of Seattle brackelope tournaments added by Michael. It’s mostly automated at this point, but it’s been a ton of work to normalize names and making sure the correct IFPA numbers are attached.
A side point to that is that it’s equally important that the Monday Night Pinball team results are also submitted to Match Play Ratings. It’s been a good collaboration between Michael, Dave and myself to get the MNP results data massaged into the results submissions for Match Play Ratings.
If you have your team league results in a digital format and have someone who can do a bit of programming I’m happy to help you get the results formatted so that Match Play Ratings can consume them.
While I appreciate your enthusiasm Jared, and your interest @ryanwanger, there are some important things to keep in mind. Up until this post, I was only assuming the Seattle Monday Night Pinball players knew about and used this tool, and it is definitely built with that in mind. First it was for captains to build out their rosters and choose subs, and then it widened to Seattle players as a whole. @dbs (Dave Stewart) opened the door further when he used it to create skill divisions for the Northwest Pinball and Arcade Show tourneys in early June. And now I guess it has been “leaked” to the rest of the pinball world online.
Last season we started using these ratings as an experiment, and while it was successful (yay!), these are still early days and the tool I built was not created with other leagues in mind (yet). I think it’s cool that you want to use it, but I want to make sure you know what you’re getting into, especially since it isn’t documented online (again, I only assumed we’d know about it or use it!)
Important things:
It is NOT “live” data. In an implementation effort to be kind to IFPA and Matchplay servers, names, IDs, rank, and ratings are all cached locally. We wipe, refresh, and lock the data on our own league schedule. Last season (February-May) we refreshed at the start of the season, mid-season, and at the end. That worked out to be about once a month. We also refreshed before the NWPAS since Dave was using it to define amateur/rookie levels in his tourneys at the show. The last update was on June 5th, 2018 (mentioned on the main search page). It’s unlikely we will refresh again until August or September, but I cannot guarantee when it will happen.
Actually, it’s “sort of” live data. Here’s how it works… When we refresh, I have it look up and cache all our known league players and subs. So they get saved on the refresh date and don’t change until the next refresh. Everyone else only gets cached when a search is performed that returns them in the results! This system worked well for us, but is obviously focused on our context of use. If you choose to use the tool, you’ll need to make peace with the idea that the information for players varies, and is as old as the last time they were searched in the system after the last refresh. My recommendation would be to do a search for every name in your league up front, to sort of lock them in, and then search new folks as necessary. While I do not show the saved date in the search results (maybe I should), it is saved in the system.
Worse than the potential “staleness” or date inconsistencies, the IPR thresholds are generated with Seattle Monday Night Pinball teams in mind. Dave’s whitepaper description goes into the details, but the TL;DR is that WE decided meaningful percentages to use for the size of each bucket, 1 to 6. We probably have more than our fare share of top players in our Seattle league, with folks like Raymond Davidson (#1 currently in IFPA) and a bunch of other top 100 players. So for us, the percentage breakdowns make sense. For IPR to truly be meaningful to you, you’d want to figure out your own definition of how big the buckets are. The rank of 6 is our way to separate out those world-class players (and we have specific league rules dealing with them, like you can’t have two 6-rated players on a team), but if you don’t have that issue then it may make sense to size things differently, or maybe even only use 5 buckets. For us, roughly 3-5% of the players on team rosters are world class players, and the rest of the breakdown works out to be roughly a normal curve. It’s still a work in progress though and we may be tweaking things a bit for next season. These percentages are backed into our tool though, so if you wanted to change them you’d need a new tool.
Searching requires care – with great power comes… well, you know! BE AS PRECISE AS POSSIBLE. When you perform a search in the tool, it’s performing some searches on IFPA and Matchplay. Since Matchplay offers a batch search API, it generally only takes one hit to pull in all the info. IFPA’s API is… less elegant. It’s great for finding info on a specific player if you already know who they are, but it kind of sucks if you’re trying to build a search system on top of their search system and merge results. I’m also very thorough. See, the thing is, I have to match up records across the two systems “automagically”, if I can, and that means a bunch of extra searches. So if Matchplay gives me result set “A” and IFPA gives me result set “B”, I also have to go looking for any “B” results not already grabbed from “A” in Matchplay and vice versa. Then I can dedupe, combine based on IFPA ID, etc. In IFPA, that means doing as many separate (!) searches as required to pull in the new names. That translates into two things: a hopefully sanely implemented but lengthy set of API calls against the IFPA website, and slowness based on size of result set across the systems. The TL;DR is BE SPECIFIC WHEN SEARCHING. If you search for “John”, then your search may time out and you may give the Sharpe’s and my web host reasons to hate me. Don’t do that please. Up until now, searching has been minimal due to it being local to Seattle, but if the genie is out of the bottle, I fear it might all come crashing down one day… Careless searching will also grow the cache file. And since I’m loading it before every search (yeah, it’s not in a database – did I mention this was just experimental?), the more names cached, the slower your search even if it’s all already cached!
Rank meaningfulness requires regular play and a decent amount of data. It’s worth mentioning that the goal of this tool was not to create a new global rating system that is somehow better than IFPA and Matchplay. (It may not mean much in a global context, or even an American context, or even in the context of another city area that isn’t the Seattle area.) It was to give us a meaningful rating system for our own league, and help us solve some existing problems in our league rules and system. IFPA rank alone is what we used before, but we noticed that while it covered the top players pretty well, and (at least through last year when a lot of weekly tourneys were for IFPA points) it sort of covered mid-range players, there was a whole population of regular pinball players of various skill level who never played often enough in IFPA ranked tourneys for it to be meaningful. It was also a way to game the system of the league – if you don’t want to deal with player/team rank restrictions, simply don’t play in IFPA tourneys! People were already starting to do that here in support of team fexibility. When we connected with Andreas, he provided a solution. He wanted data for Matchplay Ratings, and I already had the Brackelope data for the Seattle area indexed (all the way back to 2013, because I am insane!), plus there were plenty of non-IFPA ranked weeklies already in his Matchplay data. The end result is a system that you can’t game unless you never play in online events anywhere. People would be judicious about IFPA-ranked tourneys to keep their rank low, but we doubted they would stop playing pinball tourneys entirely. But this system also works as well as it does because players in Seattle play OFTEN. It’s not unusual for a player to play in 3 tourneys a week, and we’ve now got multiple weeklies going on every day of the week, and monthlies, in addition to superleague, SPL (meeting in people’s homes), and Monday Night Pinball (when in season). Andreas’ point about feeding the MNP data back into the system is important too – even if players don’t really play much in weekly events, by definition, their play is being logged and used for ratings when they play in league. Anyhow the point of all this is to stress that while the system (and this tool) works for us, it may not work as well for you. Andreas can provide more advice on this, but I’d highly recommend reading through his documentation on Matchplay Ratings and taking a look specifically at the ratings deviation value for your players. In general, our players’ RD is quite low because of the wealth of data we’ve submitted, but if it’s quite high for most of your players then it’s going to be less meaningful.
Raw data files for the curious… I wouldn’t normally share this stuff, especially since this wasn’t meant for public use, but if you actually do want to use it, I want to make sure you’re well informed. Here’s a link to the cache file: http://pinballstats.info/search/searchinfo.json
That’s everyone that was included in the last refresh, and everyone searched since, until the present moment. You can see the cache dates in there. UTF-8 characters may be buggy in browser, sorry! I also have a simple search log here, for debugging purposes: http://pinballstats.info/search/iprsearchlog.txt
Sorry for writing a short novel, but I wanted to ensure you knew what you were getting into here! I’m curious to hear how you use it and how things go. I’m open to suggestions too, but I don’t really have the time or desire right now to scale this up and make it globally useful, or try to support leagues other than the one it was created for. Improvements and tweaks though are definitely doable.
I don’t think Michael is pulling in Match Play Rating data for all players on the website. At least the MP column is blank for many players.
I’m curious about that. If the player was cached before they existed in Matchplay Ratings, then they wouldn’t show up (because they didn’t exist then!) But if you’re searching for a player that wasn’t already cached, and they are in MR right now, then they should show up. If you found a bug I’d like to know.
The search tool hits both IFPA and MP separately and then combines the results based on IFPA ID. So in some cases there are players who played in an IFPA ranked tourney and exist there. But if that player’s tourney(s) wasn’t hosted on Matchplay and wasn’t imported into Matchplay Ratings, then they may not exist there. There’s also the edge case of a player who has one name in IFPA and another name in Matchplay, AND their MP profile isn’t linked up to their IFPA ID. I try to safely combine player entities when I can, but it has to be based on something trsutworthy, like IFPA ID.
