Better, faster, cheaper: these were the watchwords of 2016 when it came to new technology. From drones to phones, from VR to AI, pretty much everything got more advanced, and less expensive.
But bigger changes are on the horizon. No matter what device you're using, there's a good chance that you're consuming some type of media content. Soon, that content will respond dynamically to your attention and engagement levels. Getting bored with a movie? You'll find the plotline abruptly branches off in a new direction. Welcome to the dawn of Responsive Media.
We've all seen "interactive media" - video games and websites where the story can be tailored to the actions of its viewer. But these efforts have failed to gain traction, for two reasons. From a video production standpoint, generating narratives that branch off in different directions is time-consuming and labour-intensive. From the consumer's point of view, when the plotline comes to a fork in the road, interactive media asks users how they want the story to develop. This pulls the user out of the experience, and breaks any spell the story may have cast.
Responsive media solves these issues. Instead of asking audiences what they want, new technologies will give them experiences that respond to preferences automatically.
This year saw the arrival of several new technologies that will sharply reduce the time and work needed to produce responsive media. First, there are cameras that generate 360-degree panoramic video, such as Nikon's KeyMission 360, and some even in stereo, like Vuze.
These devices are more manageably sized, and more affordable, than the large, expensive multi-camera rigs released in 2015 by Google/GoPro, Lytro, Jaunt, and Nokia last year. This opens up a wider potential market for 360-degree video production. Once taped, the video is also becoming easier to edit. Web-based systems like Interlude's Treehouse, an interactive video authoring site, make it easier for producers to specify branch points at which the audience can navigate multi-view environments.
In addition to making it easier to produce responsive media content, new technologies are making responsive media more fun to consume. The ultimate goal is to keep viewers immersed in the experience by relieving them of the need to choose a narrative path. Sensors can track user's gaze, pulse, pupil dilation, laughter and other cues of a user's engagement in a media experience. For example, Apple recently acquired Emotient, a company that uses advanced computer vision to recognize how people are feeling by reading their facial expressions. One can imagine a VR movie where such cues drive the storyline.
The human-machine interface is becoming more comfortable. Motion sensors detect the user's natural turn of head, making it easier for them to visually track the story elements they find most interesting. Sensors like Google Tango can detect the position of a smartphone in a room so users can more easily navigate around 3D virtual objects. Technologies with this kind of "situational awareness" allow consumers to move their heads, or their devices, naturally, rather than having to click buttons to see different viewpoints.
In addition to entertainment media, these technologies will be embedded into the kinds of future computing systems that are not controlled by keyboards and touchscreens like VR goggles, robots, autonomous vehicles and smart environments. Rather than responding to commands, future systems will monitor a user's engagement, trying to anticipate what the user wants and prevent her from becoming disengaged by dynamically accessing information services and changing content to suit her preferences, emotional state and current situation. Imagine:
•Home robots can detect when a child is frustrated during homework and offer clues and assistance that keep the child engaged.
•Movies on smart TVs, phones and VR headsets can adapt the story to be more dramatic, action or thought-provoking depending on how the individual responds to the scene.
•Semi-autonomous vehicles can sense the driving conditions in real time and predict when conditions indicate that the driver's attention needs to return to the road.
... and many more responsive experiences.
For users, responsive media will be more like an engrossing conversation than a passive experience. There's a lot to look forward to in this area, both in 2017 and beyond.