Can deep learning help solve lip reading?

Lip reading is a tricky business. Test results vary, but on average, most people recognize just one in 10 words when watching someone’s lips, and the accuracy of self-proclaimed experts tends to vary — there are certainly no lip-reading savants. Now, though, some researchers claim that AI techniques like deep learning could help solve this problem. After all, AI methods that focus on crunching large amounts of data to find common patterns have helped improve audio speech recognition to near-human levels of accuracy, so why can’t the same be done for lip reading?

The researchers from the University of Oxford’s AI lab have made a promising — if crucially limited — contribution to the field, creating a new lip-reading program using deep learning. Their software, dubbed LipNet, was able to outperform experienced lip readers to a significant degree, achieving 93.4 percent accuracy in certain tests, compared to 52.3 percent accuracy from human lip readers. And even in its current, early stages, the software is extremely fast — processing silent video into text transcripts in nearly real time.

However, before we get lost in bad dreams of AI-powered surveillance states and HALreading lips in 2001: A Space Odyssey, the research from Oxford has some serious limitations. For a start, the system was trained and tested on a research dataset known asGRID. This is a collection of tens of thousands of short videos of 34 volunteers reading nonsense sentences, as well as captions. Each clip is just three seconds long, and each sentence follows the pattern: command, color, preposition, letter, digit, adverb. Sentences include, for example, "set blue by A four please," and "place red at C zero again." Even the words within these patterns are limited — there are just four different commands and colors used. This has led some researchers in the field to suggest that the paper's findings have been overblown, especially after one viral tweet linking to the researchers’ video (below) made the sensationalist claim that the work meant there would be "no more secrets."

This is certainly not the case. Speaking to The Verge, two of the researchers behind the paper, Yannis Assael and Brendan Shillingford, readily admitted they were working with "restricted vocabulary and grammar," but said this was due to limitations in available data. "The dataset is small but it’s a good indication we could perform just as well with a much bigger dataset," says Assael.

REALLY, THIS WON'T HELP WITH SURVEILLANCE AT ALL

Both Assael and Shillingford are also keen to stress that their work has no application in the world of surveillance, simply because lip reading requires you to see the subject’s tongue — meaning that the video has to be straight on and well-lit to get a good result. "It’s technically impossible or at least very, very difficult," to use any lip-reading software for surveillance says Assael, adding that frame rate is also a factor; and something that is usually neglected with CCTV. He says: "And if you do have frontal video of someone taken with a very good camera, then you probably have a directional microphone [pointed at them] as well!" (On the subject of surveillance, Assael notes that although one of the paper's supervisors also works with Google's AI division DeepMind, Google itself had no involvement with LipNet's development.)

Instead, the two researchers think that lip-reading AI could help people with hearing disabilities, especially in noisy environments where it’s difficult for computers to isolate speech. For example, someone wearing a camera built into a pair of glasses could get clear, frontal footage of someone they're talking to at a party, and an ancestor of LipNet could then transcribe the conversation in real time, feeding it into their ear. "Anywhere you have speech recognition and a camera, we can improve that," says Assael. He also mentions silent dictation to Siri or Google Assistant as a possible use-case. In the future, then, perhaps those of us who don’t like speaking to our computers, can’t just have them read our lips instead.

Why NASA’s Juno mission could last a lot longer than it was supposed to

NASA is preparing for Juno to stick around Jupiter a lot longer than it had originally planned. The probe — which has been orbiting the gas giant since July — is going to stay in its 53-day orbit around the planet for a while, with no definitive plans at the moment to put the vehicle in a shorter orbit. Originally, NASA had hoped to have Juno in a two-week orbit by now, but ongoing engine troubles are delaying that move.

IT’S NOT EXACTLY BAD NEWS

If Juno never goes into its shorter orbit, it’s not exactly bad news. NASA says it won’t diminish the amount of science Juno can do at Jupiter. The biggest difference is that the spacecraft will be swinging by the planet at a much slower rate, so the mission could conceivably last beyond 2019, instead of its previously scheduled end of February 2018.

The trouble for Juno started last month, right before the spacecraft was about to do its second swing by Jupiter. The vehicle doesn’t orbit the planet in a circle but takes a highly elliptical path in order to avoid as much of the radiation-filled environment around Jupiter as possible. Because of this, Juno gets super close to the surface of Jupiter for just a few hours each orbit. These close passes are known as Perijove passes, and they’re the times when Juno can gather the most data.

The most recent Perijove pass occurred on October 19th, but the mission team didn’t intend to do any science on that one. Instead, the plan was to ignite Juno’s main engine, putting the vehicle in the shorter 14-day orbit. The engine burn can only be done during a Perijove pass and none of the science instruments can be on when it happens. Leading up to the pass, NASA engineers found that a few engine valves were taking longer to open than they were supposed to. "That is something that is significant because it can affect how the engine operates," Rick Nybakken, Juno project manager at NASA’s Jet Propulsion Laboratory, tellsThe Verge.

NASA decided to scrap the engine burn for the October 19th pass and take science measurements of Jupiter instead. But then a software glitch put Juno into safe mode. This mode turns off all of the vehicle’s instruments and prompts the probe to turn toward the Sun while it awaits instructions from Earth. NASA was able to resolve the glitch and bring Juno out of safe mode on October 24th, but at that point the Perijove pass was completed and no science data was gathered.

Now, NASA is focusing on the engine troubles. The mission team won’t be performing a burn on Juno’s upcoming Perijove pass, scheduled for December 11th, and they’re preparing for the possibility of never igniting the spacecraft’s engine again. "That’s the thing we’re looking at," says Nybakken. "We’re not going to do it if we can’t do it safely. And so we’re looking at different ways we can do the burn. Right now, it’s too early to say which way it’s going to go."

THE MISSION COULD LAST BEYOND 2019

That means Juno may not de-orbit in February 2018 like NASA had expected. Instead, the mission could last beyond 2019, according to Nybakken. That’s because the spacecraft will be exposed to the worst parts of Jupiter’s radiation less frequently. Eventually, all those charged particles around the planet will slowly damage Juno enough that it can’t function anymore. But Juno only receives the bulk of the planet’s radiation when it does its flybys of Jupiter. So on a 53-day orbit, the vehicle gets pelted with the worst radioactive doses at a much slower rate. "Radiation accumulation is a function of the number of orbits and not a function of time," says Nybakken. "So it’s really hard to gauge impacts of radiation perspective other than to keep in mind, we accumulate on a full orbit basis not on a time basis."

The only real problem? An eclipse in mid-2019 that would put Juno in Jupiter’s shadow for six to 10 hours. During that time, the spacecraft would not get any sunlight on its solar panels and its temperature would drop significantly. Juno has gone through periods of no sunlight before, but never for such an extended period of time. So NASA’s navigators are trying to figure out a way to change Juno’s orbit to avoid the eclipse, in case the probe is still operating by then. "That looks like a significant obstacle that we’d have to overcome," says Nybakken, "but we have very creative navigators."

Soylent blames algal flour for consumer complaints

After halting production of Soylent Powder and Soylent Bars in October due to consumercomplaints of nausea, vomiting, and diarrhea, Soylent thinks it has finally found the culprit: algal flour.

Since its inception in 2013, the Los Angeles-based meal-replacement startup has gained a strong following among Silicon Valley circles. However, just as the company was basking in its early success with the launch of its new coffee drink, reports surfaced about consumers who experienced unpleasant side effects, Bloomberg reports. The company has since been conducting tests to determine what could be making its consumers sick.

ALGAL FLOUR WILL BE REMOVED FROM FUTURE POWDERS AND BARS

Soylent now plans to remove algal flour from future powders and bars when it releases its new formulation early next year. However, Soylent’s algal flour supplier, TerraVia Holdings Inc., said Soylent’s products contain several irritants — like soy protein isolate and glycerin — that can potentially lead to discomfort experienced by consumers. “Our algal flour has been used in more than 20 million servings of products, and we are aware of very few adverse reactions. In no cases was algal flour identified as the cause.”

In addition to its powders and bars, algae can also be found in Soylent’s premade drink as well as its coffee-replacement Coffiest. However, the ingredients are in the form of algal oil, not algal flour. According to Bloomberg, Soylent hasn’t received any complaints with regards to its premade drink, and although there are talks of consumers being sick from drinking Coffiest, Soylent has yet to officially acknowledge this and has not put a pause on production.

Final Fantasy XV is getting a mobile spinoff from the creators of Game of War

Final Fantasy XV is getting perhaps its most surprising spinoff yet. It’s a new game developed by the company behind ubiquitous mobile games Game of War and Mobile Strike.

Few details on the game are available just yet, but Square Enix says that it will be a mobile massively multiplayer online game set in the ever-growing FFXV universe, featuring “the characters, storyline, and soundtrack” of the game. FFXV launches later this month, but the game’s world has already been fleshed out through an anime series, feature-length CG movie, and several mobile games.

The new, untitled game is being created by MZ (formerly Machine Zone), the California-based developer behind the incredibly lucrative Game of War and Mobile Strike. The company is perhaps best-known for its celebrity-filled TV commercials, featuring the likes of Mariah Careyand Arnold Schwarzenegger pitching multiplayer mobile games. MZ has since taken the infrastructure it built to connect millions of gamers online in real time, and expanded its uses to other avenues, most notably transit infrastructure.

No release date has been announced, but the FFXV mobile MMO will be developed by a brand-new internal MZ studio. FFXV, meanwhile, is launching on PS4 and Xbox One on November 29th.
                        

AT&T will give away free Apple TVs to some DirecTV Now subscribers

In an attempt to make a big splash with its upcoming DirecTV Now service, AT&T will reportedly give away Apple TVs to customers who agree to pay for three months of the service, according to a report from Variety. According to customer support documents obtained by the publication, AT&T’s over-the-top offering — which will cost $35 a month — will also give out free Amazon Fire TV sticks to customers who sign up for one month of the service.

The tactic isn’t new, as Sling TV had a similar giveaway last year, but the cost and potential scale of AT&T’s plan far exceeds previous offerings. Sling gave away $50 discounts for the then $99 Roku 3 and Amazon Fire TV if you bought three months of the service; AT&T would be giving away $150 Apple TVs for the same lock-in period.

The documents uncovered by Variety also gave some insight into the channels DirecTV Now may carry at launch, with channels like Fox, CBS, Univision, and the Sony Movie Channel named on a partial list of channels. (CBS would be the big surprise given that the only OTT network it’s agreed to work with so far is YouTube, which is reportedly preparing a TV service for next year.)

DIRECTV NOW WILL REPORTEDLY GIVE AWAY AMAZON FIRE TV STICKS TO NEW CUSTOMERS

Those channels would go along with the previously announced carriage agreements between AT&T and Disney, NBCUniversal, Viacom, Scripps, Discovery, HBO, Starz, and A&E Networks. AT&T says the service will offer over 100 channels.DirecTV Now will feature a VOD library with around 14,000 titles, according to the report. It will also let users take advantage of a 72-hour catch-up window to watch shows you may have missed for up to three days, but according to Variety, every channel won’t be included with the feature (ESPN is notably exempt).

What hasn’t been clarified is whether the service will offer a cloud DVR service like Playstation Vue does, but with the release of DirecTV set to happen sometime this month, we shouldn’t have to wait much longer to find out.

Twitter is reportedly in talks to sell Vine

Vine may live on with a new owner, according to a new report. TechCrunch says Twitter is now evaluating multiple offers for the short-form video service, which it announced it was shutting down last month. Some of the offers are for less than $10 million, according to the report — less than it reportedly cost Twitter to run Vine for a single month. (The company declined to comment.)

Still, a sale could ensure that the huge archive of Vines remains online at a time when their current host, Twitter, is trying to slash costs. It would also allow the product to evolve after years of near-total stagnation. TechCrunch says that acquisition interest in Vine soared after the outpouring of grief that followed Twitter’s shutdown notice.

Samsung runs full-page apology ads over Galaxy Note 7 recall

In an effort to restore some consumer goodwill after the discontinuation of the Galaxy Note 7, Samsung ran full page apology ads in three major US daily newspapers today. The letters, which appeared in Monday editions of The Wall Street Journal, The New York Times, andThe Washington Post, were aimed at English-speaking consumers, according to The Korean Herald. The letter is signed by Gregory Lee, the president and CEO of Samsung Electronics North America.

"An important tenet of our mission is to offer best-in-class safety and quality. Recently, we fell short on this promise. For this we are truly sorry," the ad reads. "We will re-examine every aspect of the device, including all hardware, software, manufacturing and the overall battery structure. We will move as quickly as possible, but will take the time needed to get the right answers.”

The company says it’s continuing to investigate the device’s development and manufacturing processes to fully unearth what exactly went wrong and caused the device to catch fire and combust. The Note 7, which was initially released back in August, suffered from critical flaws in its design that led to overheating. Samsung initially recalled millions of units in early September, but permanently discontinued production a month later after replacement phones began exhibiting the same issues. As of last week, around 85 percent of all devices have been returned, the company says.

All in all, it’s clear Samsung leadership feels the need to mend bridges. "Most importantly, safety remains our top priority," the ad says in conclusion. "We are grateful for your ongoing support and again, we are truly sorry."