Green screens have been around for years, allowing everyone from big-time professional filmmakers to amateur YouTubers to replace their background for something more unique. However, they typically require a studio-like setup to be in place which can be off-limits or cumbersome to many.
YouTube now has a solution to this dilemma. The company has been working on a technique which uses machine learning to solve a semantic segmentation task using convolutional neural networks. In simpler words, it’s leveraging AI smarts to intelligently identify the foreground and background, separate them into 2 layers, and replace the background with something digital.
This is pretty groundbreaking, since chroma keying AKA green screening and rotoscoping are typically the only two ways to do this. YouTube is only making this available in limited beta for now. It’s further restricting things to creators who are using its new stories format.
YouTube stories looks just like Snapchat and Instagram’s video tools. Given this inescapable comparison, it’s a good thing the brand is trying to differentiate itself by offering something a bit more advanced for its patrons. The feature has been developed specifically for mobile, with the company making sure it’s lightweight and able to perform its magic in real-time at 30fps.
Based on the sample GIFs in Google’s blog post, the company’s technology is still a bit rough around the edges. This is to be expected given its experimental nature, but it may not catch on if everything isn’t up to scratch upon an official release.
It’s not clear if YouTube will eventually allow custom background replacements since it’s currently only offering a limited number of options like Good Night, Studio Light, and Dance Party. We’ll keep an eye out for when this feature goes live for everyone.