National Guardsmen are helping residents affected by the flooding caused by Hurricane Harvey in 2017. … [+]
The National Guard has finished its work. As first responders to fires, floods and other increasingly common disasters, Guardsmen must be deployed quickly where needed.
This presents one of the highest challenges imaginable for infotech as a sector: In the many square miles affected, which school, bridge or neighborhood needs immediate help the most? Often, lives are at stake.
parent company of Google Alphabet has a factory called X that aims to solve some of the world’s most critical and vexing issues, and X has developed a breakthrough solution — one that uses predictive AI in much the same way that many companies do. Here’s how X did it—and what all business professionals want to know about combating uncertainty and risk with machine learning.
The problem: Tagging aerial photos
Both during and after a severe weather event, drones and manned aircraft gather thousands of aerial photographs of affected areas. These images potentially reveal which buildings and other infrastructure have been affected — but only after each one has been tagged as to the exact location it shows. Unfortunately, images generally lack this metadata.
Tagging photos by hand slows down the National Guard response tremendously. After an incident, it usually takes her team about 12 hours to complete the task. Unfortunately, this process has until now remained manual. Automation is a challenge as photos are taken from different heights and at oblique angles.
But that’s exactly the kind of problem for which the X was designed: The stakes couldn’t be higher, yet it requires a technological breakthrough. X’s initiative to undertake this and related challenges is called Bellwetherdescribed as “the first prediction engine for Earth and everything on it”.
Sarah Russell, who has been running Bellwether since she founded it in 2020, makes the case. “We took on this challenge because we realized that if we could solve it, we would reduce response times to climate disasters and multiply the number of lives saved.”
The solution: Photo matching with machine learning
The breakthrough? Match real photos with artificial ones. Bellwether has compiled a database of simulated reference photos to use as templates. When a real photo matches one from the database, it is tagged—the system then knows exactly where and what a photo is. To compose the reference imagery, X leveraged Google’s wealth of unique geospatial resources, the underlying basis for products such as Google Earth and Maps.
It works. Only a few years after Bellwether formed and began working on this solution, the National Guard already is developing it in tests and plans to use it for during this summer’s fires.
With this solution, National Guard team members can immediately study the hardest hit areas and know which locations they are looking at. They can tell which bridges are out. They can ask the monitored areas, “Show me all hospitals.” They can issue informed answers immediately, freeing themselves from the processing delays that have held them back for years.
ML plays a central role in moonprints like this – just as it does in more common enterprise systems. After all, photo matching is exactly the kind of imprecise process that ML handles well. No match is guaranteed, as aerial photos do not match exactly. Each of them comes from a unique distance, zoom and angle, is potentially weather-locked, and the landscape they capture has often been affected, sometimes catastrophically.
ML removes most of the uncertainty by assigning a confidence level to each match. With many photos coming in, it turns out that several of them match with high confidence, so the system can provide graphics to operations personnel that cover almost all affected locations, even after discarding those that didn’t find a sure match.
This approach is scalable. “By expanding our deployment with the National Guard, our goal is to make this type of service substantially easier for a broader group of disaster responders,” says Russell. “It can be applied to rescue and reconstruction responses to various weather-related phenomena, including heat waves and tornadoes, for example.”
The Universality of Predictive AI
Whether shooting for the moon or shooting for more formal business goals, ML’s core ability to create levels of trust is solving operational challenges worldwide, across all industries. Which customers are likely to buy? Marketing targets them. Which transactions are likely to be fraudulent? Banks block them. Which addresses are likely to receive delivery tomorrow? UPS plans for them.
This well-experienced paradigm—driving large-scale enterprises with ML’s predictions—has a name: predictive AI. It is the practice of systematically filtering out the hypotheses that show less confidence and acting on the more confident hypotheses that remain.
So, how sure are sure enough? It depends. Each project must determine the best decision boundary choice based on practical necessity. For example, the National Guard needs photos that have been matched with very high confidence. Instead, marketing and fraud detection can afford to target a lot of cases that don’t pan out—an inevitable part of the numbers games these types of businesses inevitably play.
To put it another way, predictive AI reduces uncertainty by quantifying uncertainty. Bellwether is working to expand this amazing approach in other ways that will also reduce the damage caused by climate disasters, such as by predicting where the most lives could be saved – which affected areas should be the highest priority for evacuation assistance – and by predicting environmental incidents before they happen.
“ML has become the new paradigm for the earth sciences,” says Russell. “Until recently, for example, hydrology mostly predicted floods with site-specific models. Now, with ML, the best models are developed using data taken from a variety of locations – flood behavior on the US East Coast can be used for West Coast Flood Forecasting’.
