Unfortunately, the manufacturers themselves put in a lot of effort towards the home buyers market and not enough towards location play. But the thing is...pinball is not an easy thing to find in public. I have to rely on sources like Pinball Map to find these machines, and these sources are not exactly common knowledge. I attribute that to the near-extinction of arcades in the west--there aren't many places where putting these machines up are viable. Even for someone who only plays machines in public like myself, I know that operators would want to take as few risks as possible, and pinball competes for operators' cash with redemption machines and claw machines, which give out prizes (or at least suggest passers-by that they do so). Hence, there are few machines in public. That's a barrier of entry too.
That's the predicament pinball is in: Pinball is something everyone has heard of but few people have actually SEEN. Hence, it's one of those things that's mainstream in awareness but not much further than that.
I can't say much for hockey as I don't live in a cimate with snow and ice, though I am aware of hockey leagues and such. I didn't know auto racing was big though. Does that include nighttime street racing?
For the youth, pinball would compete with video games (including mobile games) for their attention, and having to pay every game is a huge turn-off to them. That's why arcades died out in the first place: They'll pay more for their initial purchase of a game system and the games they want, but they can then play for an unlimited amount. Online play and some games DO have a subscription service, but that's a pretty major sore point for many.
Also, I think the themes being appealing to a pretty narrow demographic is something of a necessity, unfortunately: Because these tables are so pricey, and the manufacturers concentrate on the home buyers audience, they go for the group that they believe can afford to buy these machines. Fans of AC/DC would be old enough to have jobs with the disposable income for one, but fans of Bruno Mars? Probably not. We're getting a table themed on Adam West Batman, but I don't think we're ever getting one themed on Attack on Titan. That being said, if they could aim more at the public play section (I'm not sure how feasible that is), they could potentially aim at any range. Data East clearly aimed their machines at kids, with themes like TMNT, Jurassic Park, Rocky & Bullwinkle, and Back to the Future, and were pretty successful at this.
I think that if there is a kind of club mentality to it, it can be shattered if large groups of newcomers come in. That is, what TV Tropes refers to as a "Newbie Boom". I can say firsthand that Mario Kart used to be like this, as is what's happening to JoJo's Bizarre Adventure right now. (If you've never heard of JJBA, I guarantee you will in the next five years.) If some machine can become a breakout hit outside of the established audience, the exclusionists will be pushed to the fringes, and with nowhere to go, either integrate themselves into the new crowd (as with Mario Kart), seclude themselves in the fringes (as with JJBA), or just dissolve completely as the old fans move on to something else (as with fans of Homestuck before it became popular).