Connecting pinball resources

General
1

I’ve been talking with @haugstrup about what might be a good way to facilitate cross linking between all those tournament player resources the community has been coming up with lately. IPDB IDs seem like a good idea, but for the purposes of players, as opposed to collectors, there’s too much fragmentation there, with different runs of the same machine having different IDs.

For this reason, I’ve started to maintain a simple API that takes in an IPDB ID and responds with an IPDB group ID, where an IPDB group is just a collection of all the IPDB entries that are basically the same machine.

Example link: http://pinballvideos.com/api/ipdb_group/5753

This will look up the group ID for Transformers LE (IPDB 5753), which you can then use to link to resources that are based on group IDs, e.g. http://pintips.net/ipdb/5709 or http://pinballvideos.com/?m=5709. The other way round, if you’re storing actual IPDB IDs and want to facilitate incoming links from applications that are using the group ID, the response will also give you a list of all IPDB entries belonging to that group.

Disclaimer:

Feel free to use this in any way you like, but please be aware of some pitfalls:

To keep things simple, the IPDB group ID is just the lowest IPDB ID of all the machines belonging to the group. On one hand, this makes things easier, in that every machine automatically comes with a group, on the other hand, this means that the group ID might be subject to change for a couple of reasons:

At the moment, I’ve got all the machines listed on pinballvideos.com covered, but there’s lots of others that are not assigned to their actual groups yet, so those might switch groups in the future.

Also, whether two IPDB entries can be considered “basically the same machine” for players’ purposes can be a difficult decision at times, so this might also be a reason for future group changes.

Basically, this means that if you’re using this API for linking purposes, it would be best to not cache the results for very long, if at all.

Apart from that, I think this should work pretty well to connect different pinball resources – I’m currently using it to accept IPDB ID links to pinballvideos.com myself.

6 Likes
2

This is a very nice project! This will allow me to simplify some things on PinTips for sure. I don’t have a complete overview yet, but this will certainly help.

Ping @romballs – this might be helpful for you as well… In some way.

My first thought is that it’s more important to have a stable, non-changing group_id than to have a group_id that matches an IPDB number. I.e. I would rather you store the group_id so it never changes. So even if a lower IPDB number shows up later the original group_id is maintained. Otherwise the system becomes a bit hard to consume.

2 Likes
3

A common standard amongst our apps does seem wise. I’ll talk with Scott about if/how we could make use of it.

I’m still trying to figure out why you two even use groups. We use groups because some users insist (for good reason) on specifying the edition that they’re adding to a location. But some do not - likely because they aren’t even paying attention to what edition the machine is. And so when someone searches for a machine, we want to display not only that exact edition in the results, but all the variants. Chances are they want to see them all.

But for you guys, you’re only listing one version. For example, you both have The Avengers. No variations. So what’s the purpose of the groups?

@haugstrup, I can imagine that since users are creating machines, one may create “The Avengers (Pro)” and you’d want it to immediately show under “The Avengers” rather than as a new machine entry. I haven’t tried creating a new machine on your site yet. Do you screen new machines?

2 Likes
4

It’s useful for PinTips to know which IPDB listings a given PinTips machine covers. I’d like to be able to link to all of them further down the line.

BUT more importantly it’s really useful to have a common identifier across sites. From PinTips I would be able to link to pinballvideos.com without having to do all the manual work I do now to match up identifiers. Same with the pinballmap integration I would have been able to avoid all that manual work matching up pinballmap ids with pintips ids.

At the same time I don’t want multiple listings on PinTips for the same machines. Since they play the same (or mostly the same) I don’t want tips duplicated between multiple machines. I keep an eye on pintips machines and merge any duplicates (e.g. if someone creates “Stern Trek (LE)” I would just merge it in with “Stern Trek”

1 Like
5

Other things: I think these “group ids” should have their own name. Maybe Unique Pinball ID (UPID)?

1 Like
6

@haugstrup: I’ve been thinking about that issue as well, but the alternative to having group ID changes would be not being able to provide any linkable identifiers for the vast majority of games (and there would probably still be some group changes going on anyway).

Considering the two reasons why a group change might occur, I actually don’t think this is a huge issue:

1.) The machine has not been grouped yet: In this case, you’re out of luck anyway, but using the actual IPDB ID for linking purposes might still work. I’m going to add the timestamp of the last group change to each machine, and if it’s 0, you can consider the group ID not final and ignore it, if you want to persist group IDs and not hit up the API every time.

2.) There has been a mistake:

a) I missed a machine that has a lower ID than all the others of its group. Unlikely, but in that case, I’ll keep the established group ID. As for the one I’ve missed, you’re going to have to do an update anyway.

b) Some established group membership turns out to be wrong, e.g. we bunch together GoT and GoT LE, and upon seeing tournament footage we realize that’s not right. There’s nothing that can be done about that, the group ID has to be updated.

c) I dun goofed. If there’s any stupid mistake on my part, the group IDs will have to be updated no matter what.

To mitigate the impact of group changes for people not wanting to hit up the API every time, the timestamps I mentioned might help, but I could also see a subscription model working – I’ll keep the API URLs of applications wanting to get notified of group changes and tell them every time a change occurs, so there won’t be any limbo links until their cache expires. I think, all in all, this would work better than never changing group IDs, which I’m afraid I can’t guarantee anyway.

@romballs: We had a bit of a collision there, I answered by email just as you replied here. My reply may have been a bit wordy – @haugstrup said it just as well, and way more succinctly.

7

Well, you know, cache invalidation and naming things …

You’re right, there should probably be a better name for that, but I’m not sure about UPID – I mean, like, what’s a “pinball”? For collectors, the distinctions the IPDB makes are probably pretty important, so that UPID would definitely not be unique to them. It’s a sort of players’ UPID. I’ll take suggestions.

8

For anyone wanting to persist the group data requested from the API on their end, I added some stuff to make this more feasible:

1.)

Each request now also gives you the timestamp of the last time anything changed with the IPDB entry’s group.

This could mean that the entry has been assigned to another group, or that it stayed in its default group, but it has been made “official”, or that some other machine has moved into or out of the group it belongs to.

If the timestamp is 0, this means that there has not been a manual group assignment yet (in this case the default group ID will always equal the IPDB ID). You can still use such group IDs for linking purposes, but it might not work if the resource you are linking to chose a different IPDB ID than you did for that machine. It’s probably reasonable not to persist those 0-timestamp responses, unless you are using the stuff outlined in points 2 and 3.

2.)

If you do want to persist group IDs on your end, you have to do periodical updates, so your links won’t become stale. I can’t guarantee that assigned group IDs will never change.

You can hit e.g. http://pinballvideos.com/api/ipdb_group_updates/1442162000, which will give you data on all the changes that have occured since your last update (meaning since the timestamp you’re sending in the last URL segment).

The response is an array of updated objects in the same format requests to /api/ipdb_group/[IPDBID] will produce. What “updated” means is described in point 1.

3.)

To make sure your persisted group data is always up to date, you can give me an URL that I will hit whenever there has been an update. When you get that ping, you can update your stuff as described in point 2.

TL;DR:

If you want to check the API whenever you get an incoming request or need to generate a link, that’s totally fine, but if you want to store that data on your end, there’s stuff you can use to make sure that nothing breaks when I’m updating groups.

9

Does anyone know who handles IPDB from the tech side? It would be cool to get this ‘group’ concept built into their data.

2 Likes
10

Looks like it’s a fellow named Christopher Wolf. I’m emailing him now :slight_smile:

1 Like
11

From my experience, there’s a delay between when a machine is released and when it’s added to IPDB. For example, Full Throttle is on location right now, but it’s not yet on IPDB.

So, there will be a period when we won’t be able to connect to one another via IPDB ID.

1 Like
12

There are three things that suggest that this might not be that much of a problem:

1.) It’s mentioned in several places on ipdb.org that if there’s a machine missing, you can just contact them, and they’ll add it.
2.) There are lots of machine stubs without any detailed information or images, so I suppose getting a game added just to give it an IPDB ID shouldn’t be a problem.
3.) The change log says that there are hundreds of updates each month, so this should probably go pretty fast, too.

I haven’t talked to whoever runs IPDB, but it seems getting something added is fast and easy.

13

I’m not sure it’s that great a fit. The IPDB has this encyclopedia vibe with hard facts all around, and what constitutes “basically the same game to actual pinball players” is very much up to interpretation.

I feel this sort of thing is more at home at a players’ type of resource. That said, if it does get integrated into IPDB, all the better. I’m not sure if they are even providing an API there right now, though.

14

Sadly, they have taken the opposite approach: The Internet Pinball Database FAQ

1 Like
15

Good point. I’ve only taken a passive approach with IPDB. I’ve never contacted them, but instead always just wait until the pin is added (in my experience the delay is usually a couple of weeks after release).

Well then. I’d still be curious to know if they reply to you.

1 Like
16

I’ve read that, but to me it sounded like it’s more of a bandwidth issue, with people scraping the whole thing and embedding remote images. Not sure, though, might also just be a reasonable excuse for just not wanting to make their data available (which is fine, too, of course).

Anyway, if you hear back, let us know.

17

When I previously contacted IPDB to inquire about API access (I wanted to integrate support into my League Manager system, to more easily do things like machine name autofill) Christopher Wolf sent a rather brusque reply to the effect that there was no API, there would be no API, and that they considered their data to be copyrighted. Yikes.

1 Like
18

Well, that’s a disappointing stance. Especially considering that facts collected in a database are not protected by copyright (e.g. the production year and designers names etc). I guess I shouldn’t hold out hope for an answer and if we want interoperability between pinball software it might be best to build something open and free (and in beer and speech) ourselves not based on IPDB. It can only cause problems going forward if IPDB is not willing to work with anyone else.

1 Like
19

That is a disappointing stance considering the public donates all of the information to it. Guess it’s time to start a new ipdb with specific content licensing and actual hi-res pictures (personally, so over all the 640x480 stuff). Who’s starting OPDB (open)?

5 Likes
20

The more I think about this the more annoyed I get. I’ve taken the first step and registered OPDB.org

9 Likes