Thanks to a form of AI called deep learning, computers are now really good at telling the difference between a dog and a cat. But Facebook’s Artificial Intelligence Research (FAIR) lab wants to make machine vision far more useful, going well beyond digital parlor tricks.
FAIR research scientist Piotr Dollar says the first step lies in helping machines not just recognize that a particular thing appears in a photo---say, a cat or a chair or a gun---but spot each individual detail in a photo and understand where it sites in relation to everything else. His team has built a set of tools that does just that.
These tools could provide building blocks that Facebook needs to fashion more sophisticated machine vision systems. For example, the company eventually build software that could fully describe photos to blind users (it's already part of the way there). Plus, it could use these tools in building augmented reality systems that display images in the real world with fine-grained precision.
But this isn't just about Facebook. The company has these tools to the public---open sourced them---so that any developer can play with the code and use it in their own projects.
FAIR's tools aren't the first pieces of computer vision software to isolate individual objects. For example, Microsoft's Kinect tracks the location of your hands and face, and self-driving cars keep tabs on the location of cars and pedestrians around them. But, Dollar explains, those systems rely on depth sensors to help isolate images. FAIR's software can work from two dimensional images. And instead of simply putting a box or rectangle around the images it sees, it's able to create more accurate outlines. That should help Facebook's AIs "see" not just the largest objects in an image, but all the tiny details in a photo.
But Facebook isn't actually using these particular machine vision tools yet. As with FastText, a set of tools that could be used for spotting spam and clickbait that the company recently open source, the FAIR team opted to release its work to the public early, before it's found a particular application at the company. This is a slightly different approach to open source than we've seen from big tech companies.
While Facebook, Google, Microsoft, and many others have openly shared several open source projects---including AI systems---most of those are pieces of software that the companies developed for their own internal use. FAIR, on the other hand, is doing research that may or may not ever be used by Facebook. The idea is that by doing this sort of foundational development, and sharing it with other researchers, the FAIR team can advance the state of AI in ways that wouldn't be possible if its staff were working in secret on projects that solve short-term problems for the company.
This isn't purely altruistic. As the overall state of AI gets better, Facebook will be able to use those developments to its advantage. And it might just get a few ideas about how to actually use the stuff it creates.
