Every so often Everybody Soccer likes to live up to its name and search out unique contributors to American soccer. Last year I talked to Horse Guy about his interactions with the Revs and most recently I stumbled across Soccer Photogrammetry (@OffsideModeling). Soccer Photogrammetry uses a computer program to give a definite answer on offside, goal, or out of bounds situations. While the products are always interesting and informative, it's also a great example of how there is still plenty of room for the every day fan to contribute to the game. Scroll down to learn more about American soccer's most logical fan.
First question, who exactly runs the anonymous Soccer Photogrammetry account and what makes you qualified to run such an account?
My name is Greg Boege (@Sombraala) and I'm a software engineer. I don't necessarily need to be anonymous, but I don't mind it either. When I'm anonymous I can't be accused of being biased towards/against a team - which I like because I try very hard to not let my particular personal interests interfere with how I model things - but I should be able to justify my work in face of the accusations of bias. The other reason is that it isn't really about 'me'. I don't want people to see the account and think of a person as much as a service. In the future, maybe I open up the account to multiple people, especially as the account gets more and more requests.
As far as qualifications go, I wouldn't say I'm particularly qualified to run the account, it's just that I'm the only one that happens to do so. I do have interests and hobbies that are useful - like an interest in photography which helps with the understanding of how cameras and lenses work and, more importantly, can distort reality and result in an image that isn't a true representation of what it depicts. I had taken classes as a computer scientist about computer graphics and understanding the those concepts (viewport, raytracing, etc.) are useful in understanding what exactly I'm trying to accomplish when I'm doing the work.
And, above all, I'm a scientist. Maybe not by profession, but I'm always looking to coax information out of whatever is available. Ultimately that resulted in me thinking "is it possible to know with some certainty whether so-and-so was offside or not?" and then actually going out to accomplish that.
How did you start Soccer Photogrammetry?
It all started back in 2013. I had been using SketchUp to make some 3D models of something (my house, I believe) and I came across the 'Match Photo' functionality. A contentious (non) offside call resulted in a goal against my team so I wanted to prove to everyone that I was right and he was offside and thought "maybe I could use that photo match thing to do it". The resulting model did agree with my assessment, but nobody much cared at the time. It also was pretty sloppy work. (I was just starting to figure out how to do it, after all.)
My first (not very good) offside model, August 2013, NYRB @ SKC:
My next opportunity came on a pretty big call in the 2013 MLS playoffs on an offside call given against Houston vs SKC. (In retrospect, that call may have allowed SKC to win the 2013 MLS Cup.) I had been practicing on my own and gotten much better so I responded to @MLS with a model agreeing with the call. It got Greg Lalas's attention and ultimately referenced on MLSSoccer.com. That was when I realized that people were possibly interested in the work I had been doing. Up until then I had done it for my own curiosity.
Through 2014 and early 2015 I posted a handful of models on my personal Twitter account. Eventually I figured I had gotten good enough to create a separate Twitter account and pulled the trigger in August 2015. The feedback I got was fantastic, especially from the Columbus fans despite my model saying their goal should have been called back. It was especially encouraging when Simon Borg referenced my account in Instant Replay. I literally danced when I saw that.
Another big call I enjoyed doing and have actually seen referenced across many media outlets was the 2015 MLS Cup missed out-of-bounds call. To be able to give fans some sort of concrete notion the distance the ball was out-of-bounds was really rewarding.
What's the most infuriating mistake a commentator can make, regarding offside?
The worst that they could make is to get the rule wrong, but fortunately that doesn't really happen (anymore?). It can be frustrating if they don't seem to have a great handle on passive/active or when it gets reset, but those aspects have changed a bit recently so it can be somewhat understandable. Not appreciating that it's not the "last field player" but actually "2nd-to-last defender" is also frustrating, but those situations are often easy to forget about, especially when you're on-the-spot and expected to be making constant commentary.
The one I would consider the actual most infuriating comment - which is to berate the officials for missing a call without having any empathy for the difficulty of making the call. Sure, there are big misses that have the capability to change the outcome of a game, but just like it's difficult for a commentator to get everything right on-the-spot, it's that much more difficult for an AR to get everything right.
I suppose another one I could mention is just how badly they estimate distances - they'll say a player was a yard offside when really it was a foot. Not that I could do better without spending time to figure it out, but I still find it humorous just how far off they can be.
What's the margin of error for a player being offside but being too close to tell? Should fans really be upset if a linesman misses a striker being off by an inch?
Any time an offside call is given where the player was not offside it's justifiable for a fan to be upset - because they are instructed to give the benefit of the doubt to the attacker. Likewise, if a player is offside and a call not given then fans should understand that same aspect of favoring the attacker. I think it depends on the situation as to just how far off a call must be to be 'badly missed', but I tend to consider 1 foot of being off but not called as a good threshold for feeling hard-done and within 6 inches fans should consider that it's probably as much, if not more, their defense's fault than a poor non-call.
Still, I strongly encourage empathy towards officials. It's not as if ARs won't make mistakes - everyone makes mistakes. Feel bad about the call, sure, but don't make it personal.
Without completely ruining the mystery of how you do your work, how does it work? Is there a "best" camera angle?
Well, I don't mind ruining the mystery if you are interested in having it ruined. I actually put a video on YouTube showing how some of it works. The gist is that I'm somewhat 'reverse engineering' the reality of the moment the pass is made. I use the (standard) markings on the field to replicate the vanishing points - the 18 yard box is, unsurprisingly, 18 yards from the end-line. The arch is 10 yards from the penalty spot which is 12 yards from the end-line, etc. I have a pre-made model of these markings and line up that model with the photo using SketchUp's 'Match Photo' functionality. When my model lines up with the photo I know that I can proceed to model the other parts of the image and they should line up in my model in the same place they were in reality.
This is easier when dealing with just players feet on the ground. Conceptually each point in an image is a ray from the position of the camera, eventually that ray hits a visible object and so we know that the object is somewhere along that ray/line. When we're dealing with feet on the ground I can use the plane of the ground to bound the ray and easily determine the position.
Shoulders, heads, or balls which are in the air are more difficult because I can no longer rely on the plane of the ground and so the point could be one of many with a margin-of-error of many inches to even multiple feet. If I have two angles of the play, however, I can find the point at which the two rays intersect and use that as the position of that point in 3D. Even if I don't have a second angle I can often use cues from where a player's feet are (since I can model those with just one angle) and extrapolate from there. It won't be precise, but sometimes it doesn't need to be - if the entire range of possible values are all onside or offside then I may not know for sure exactly how far on or off they were, but I can still know whether they were or were not onside.
Sometimes I even go as far to use the video leading to the frame in question to model where the player stepped prior to the particular frame. Sometimes I have to just say I don't know and can't tell.
The best angles are going to be the same that you would want to see as a fan - as straight-on as possible to the play. Having a second angle is wonderful, though, and that second angle doesn't have to be great.
The worst angles are any which have no field markings on them at all. I rely on the known values to make the model accurate, but if the image is from the midfield and there are no field markings and no grass striping then I just can't do it. I also can't model it if they don't actually show the player at the time the ball was played. There are calls where taking the frame where contact is first made vs. when the ball leaves the foot make a difference, so waiting a couple of frames to see the player in question is not really going to prove much of anything.
My biggest problems stem from the type of lenses that they use, which tend to be wide-angle lenses in order to get as much field in the broadcast as possible. These have significant distortion problems and that can make something appear in a spot which is significantly far off from where it was in reality. I sometimes have to use software to try and account for this, and this is where my photography training really comes in to play.
Does it matter if a video is in 1080p or 240p? How long does it take to compute an offside call or not?
The better the resolution the more accurate I can model. If a pixel is the equivalent of an inch in real life then I'll be able to do much better than if that pixel represents a foot instead. I'll usually try to say when I don't feel like my model is very accurate due to limitations in the source data.
I can now do a very straightforward call in probably 5 minutes or less if someone gives me the images to work off of and all I have to do is model them. Often times getting the right still to work off of is time consuming (especially for non MLS games where I need to search around for a video to pull one from). If the image needs distortion correction then that takes time too. I've spent hours on a single call before. When new Instant Replay host Brian Dunseth disagreed with my assessment of one in particular. I was not very happy to be told my hours of work was wrong. I still stand by my analysis.
Most, however, are about 10-15 minutes and only rare exceptions are > 30 minutes.
How do you feel about using robots to supplement - if not overtake - human referees on offside decisions? Is Soccer Photogrammetry a hobby or a statement on how the game is officiated?
They never could take over altogether as the judgement call of whether a player gains an advantage (plays the ball) or if offside is reset by a deliberate action on the part of the defense. That said, my utopian future is one where some sort of Google Glass style augmented reality keeps track of who was in what position when the ball was played. Get the human refs out of the business of doing something that our brains are not very cut out to do (see the Flash Lag Effect). Technology should be used to supplement humans, make the human refs' jobs easier.
Soccer Photogrammetry is absolutely a hobby. I absolutely have the utmost respect for the ARs out there. If it were a statement on anything it would be that the laws, as they are written, are impossible to enforce accurately on a consistent basis. They are simply too difficult to adjudicate.
Is there a place for advance linesman statistics and if so, what would they look like?
Not that I'm aware of, and that's an interesting notion. If anyone has it, PRO does.
What's the end goal with Soccer Photogrammetry?
Just to give people the information that they want to know, be a service to the soccer public, and take away that frustrating feeling when it's not so easy to figure that out. Nothing all that great. I would also say that I am interested in writing some custom software to make the work easier to do - handle distortion, help line-up the view ports for the camera angles, etc. But that's just part of that greater goal in the long run.