I suspect the biggest impact this change would have would be on the “mid-tier” players who only play at a weekly or monthly event at the same location. So rather than being ~1000 to ~4000 in the world, they’ll be ~5000 to ~12,000.
It’s probably easy for some of the top 100 players here to think those rankings are all basically the same, but I know lots of players in this range who pay a lot of attention to this stuff (despite, of course, not wanting to venture too far from home to play).
Clearly what’s needed is a new stat: Geographic Span where you add up all the miles from the events in a person’s top 20 or whatever.
No need to get complex. A simple crow flies from stated home to location address will do.
What about areas that only have one suitable venue for holding events? It hardly seems fair to punish those players because of one recurring event in New York City.
Besides the fact that I personally hate the idea . . . the implementation of this would be insane.
Now the calculation of every player’s ranking has to go through a process of analyzing the location for every tournament they play in, with arbitrary limits on how many feed into their top 20.
This also hurts those smaller towns where maybe only one or two locations actually exist. Now you’re limiting the potential for those players to even fill out a top 20 card even if those locations brought in different organizers to run completely different events.
What will end up happening is those players will simply move around to other locations nearby resulting in the same over-ranking of certain individuals (if the location has enough places to play). I would simply run our Super League at one location for 4 months, pick another one for the next 4 months, and pick another one for the final 4 months of the year. It’s not “problem solved” . . . it’s a not too difficult workaround that would only hurt the locations that are doing things right by supporting competitive events on their equipment.
The ideas of trying to find ways to penalize locations that put in the effort to bring the players more competitive pinball is just not something I want to be a part of. Outside of PAPA HQ running PAPA, they host the Pittsburgh Pinball League, Fight Club nights and other ‘local’ tournaments. I’m all for supporting them in hosting those events, knowing that all players have a chance for those events to be meaningful for them in an unlimited fashion.
What is the objective function for WPPR?
Determine #1 correctly?
Order top 10 correctly?
Classify top 50 correctly?
It is not a reasonable objective to say come up with an accurate total ordering of all players in the world.
Not be mention what is the accepted idea of the better player? Do you need to play across all eras? How do you value the skill of hitting a billion tolls on earthshaker vs being able to do things like the alien star up and under Shatz to spinner, spinner. As a community, we seem to favour certain skill versus others, but it feels like how UFC handicapped gracie jiu jitsu for the audience’s.
I think someone had previously mentioned that we’re only good at ranking the top 2% of players in the world, so that puts us at ranking the top 853 players correctly 
Our objective is the accurately rank players better than any other system on the planet, and we constantly try to improve that through revisions to the system over time.
Similar to golf, IMO the best players manage the course in front of them, regardless of the variables that the course presents (long/open course, short/tight course, links style, tons of water, tons of sand, high winds, crazy slope, fast green, long rough, etc). The best players in golf overcome whatever those factors are, and I find the same applies to pinball. Usually the top 20 resumes of players in the rankings reflect a wide range of tournaments that they’ve played.
Have you looked at expanding the active resume to 25 events? I recall the bump from 15 to 20 and you said you wanted to take some time to see how that worked out before making too big of a change. Is 20 proving to be about right, or would a larger sample paint a better picture?
No strong feelings either way, just curious.
I see everyone loves my idea. 
The idea isn’t first 4 but best 4. Are most people’s profiles comprised of only 1 single event? Are there only 12 events in an area every year at 1 location, and nothing else? Maybe I can’t tell because of the different titles of events on players profiles.
Talking about players above the top 2%, and I think Steve inadvertantly brings up this point that the “mid-tier” is all over the place.
stevevt
I suspect the biggest impact this change would have would be on the “mid-tier” players who only play at a weekly or monthly event at the same location. So rather than being ~1000 to ~4000 in the world, they’ll be ~5000 to ~12,000.
So in Vermont the Mid-tier are players in the range from 1000-4000 but in Portland, Seattle, NYC, or Stockholm it may be much higher, more like 30-800. That’s already a massive difference!
I think the most obvious times where people see that a player is “overranked” is when we see a player in the top 50 or top 100 who clearly “doesn’t belong” there. Player X ranked 782 from Portland, OR doesn’t ever get a chance to play Player Y ranked 2322 from Burlington, VT. When a very good player is “underranked” it’s almost always because they simply aren’t traveling and competing as often as their peers.
[quote=“timballs, post:168, topic:1601”]
Player X ranked 782 from Portland, OR doesn’t ever get a chance to play Player Y ranked 2322 from Burlington, VT. [/quote]
You might be able to play #2322 from Vermont at the next US Nationals, if the current trend continues. 
Edit to add:
Vermont, now with 3 top-1000 players!
This statement can be made for all rankings systems out there. I blame the player . . . not the system if a player isn’t competing as often as their peers to be ranked accurately.
Very good players find a way to show that they are indeed “Very Good” whenever they play - exhibit A:
http://www.ifpapinball.com/player.php?p=146
IMO the real issue is players that THINK they are very good, but in reality have no idea where they actually stand among the world’s best because they haven’t actually danced the dance yet.
My favorite example of this is someone who thought they were very good, showed up to their first big event, and realized what “very good” actually meant:
http://www.ifpapinball.com/tournaments/view.php?t=387
(he finished 43rd, and his comments on his local league message board back then going into the tournament and then his messages after it was over was some of the most entertaining shit I’ve ever read)
I’ll give you a secret . . . anyone that is actually “very good” couldn’t care less about any of the sporadic overranked people in the system
I personally feel that 20 events is providing a pretty accurate result. Expanding to 25 events only exacerbates the problems with respect to players that are overranked (they get to shove more of their questionable results into their resume), while furthering the gap of the “have not’s” of the world that are struggling to even put 20 events together over 3 years.
These seem to contradict, although I cannot really say that you are doing a poor job at this at all. Every ranking system in the world largely fails to rank more than a couple hundred people without a huge drop in accuracy.
I would like to know what the IFPA considers to be the biggest problems in the WPPR system today. The suggestions and proposed solutions here hint at what people believe to be the biggest problems. I have an internal list of the things I see as issues or points of improvement which are mostly influenced by the way the WPPR system affects me, but I also am probably recieving some benefits from the way it is set up that I may not consider a problem that someone else might.
The second statement was simply poking fun at the fact that we’re really the only system out there ranking pinball players, so it’s easy to say we’re by far the most accurate
I know Shepherd agrees because we’ve sent our entire database of results to nearly a dozen different people who thought they could make a more accurate system with that data, and we’re at a 100% success rate at never hearing back after we send that data off. We would LOVE for someone else to come up with a system that does it better (seriously).
The biggest problems with the WPPR system always stem from our push/pull of having these two agendas:
Rank players accurately
Promote people to play and organize more and more competitive events
Many times these two factors are at odds with eachother, and most of the decisions we make that favor one of these objectives hurts the other one. Trying to maintain that balance certainly makes for weaknesses in how the system is designed.
The biggest problem facing people who have “solutions” on how to make the system better often involve turning the system into a ‘NOT positive only’ system, and that’s just one of the founding principles of the WPPR system. We never want someone to be motivated to not play, period.
The efficiency adjustment at the theoretical level sounded intriguing as a way to at least put a leash on the advantage the over-players are getting, but it would still have to be at a level where it wouldn’t lead to players being motivated to not play in a given tournament.
I’ve been reminded that this is not a 100% success rate of never hearing back 
Back in 2008 we were still manually adding to the TVA based on how many top 25 and top 100 players were participating, and awarding points in those huge arbitrary chunks based on whether you qualified, won a round, won two rounds, finished top 4, etc.
Patrik Bodin and the “Swedish Invasion” thought they could do it better, so they took our database of results, created their own dynamic system of point distribution that included giving points to all players using some algorithm along with other things that I had no idea how it worked. They ran simulation after simulation over many months tweaking things where they thought things looked better compared to how we were doing it at the time.
That ended up becoming WPPR v3.0 that launched for the 2009 season and is still the foundation for how things work today
So really ANYTHING can become WPPR v6.0 if there’s someone that can find a way to do it better.
I need to resist the urge to think about this more. I am curious are you talking about achedemics, industry experts in ranking systems or Pinball players with a who feel they should be ranked 50 places higher?
I am not sure what error function is being optimised here. There is no curated ground truth. If you run simulation using any generative model, those assumptions will define the maximum likelihood solution. I assume we are measuring success by predicting ordering finish in events (using 5 fold cross validation or something), and then applying some kind of standard loss function?
I’m not sure of the motivation behind those that have been sent our entire database of results (timballs was one of them lol)
Ultimately the implementation of each version of the WPPR system over time has been based on my approval of passing my ‘smell test’. I’ve been fortunate enough to have been playing competitively for 20+ years, have had a chance to watch and play a ton of players over that time, see the success people have had at the most meaningful events our sport has to offer, and go from there.
After seeing the solution of the Swedish guys, I thought they came up a system that did it better than we were currently doing it, so we jumped on their train and haven’t looked back.
I also have no idea what 5 fold cross validation means
Not an expert, but I’m pretty sure it’s one more than 4 fold cross validation, which apparently, isn’t quite enough. 
People complaining about a silly system that’s fun for most of us, that is done by generous people for no monetary or personal gain in their free time.
Just joking around with ya Tim
100% factory default 1.05 code. Properly registering slimer. Moderate tilt. On location. Top 2 scores on the machine are 1 and 1.2 billion. Game is on location at The Sanctum. Mass hysteria has never been started on this machine either.
@TaylorVA put the carrot flippers and drilled a center post on his GB and it has definitely improved his scores significantly!