Data proves a constant issue for companies applying AI
Data proves a constant issue for companies applying AI
TIM LEEMASTER
Companies seeking to apply AI quickly find the quality of their data can be a major issue.
Beyond the headlines of racist bots and potentially lethal autonomous vehicles companies are grappling with how best to not only generate the results they are looking for - automating the routine parts of their business processes for example - but more importantly seeing AI live up to its much heralded or hyped potential.
“AI is only as smart as the data set it has to work off and the easiest data within AI is to make sure you have an accurate foundational dataset,” says Sky Cassidy, CEO at marketing firm MountainTop Data based in Canoga Park, California.
One fundamental problem stems from that first batch of data a company produces or comes across. No matter how bad or out of date it has become executives will continue to cling to it.
“They should really replace it and start anew but they feel like they’re losing something,” Cassidy says.
Organizational and management issues also prevent executives from just starting anew, particularly when a company officer has to explain why the dataset has shrunk in size significantly after a much-needed cleaning.
“There’s that negative pressure when we toss out data,” Cassidy says.
Low quality data cost the US economy USD 3trn a year and the average business up to 25% of revenue, according to some studies. On top of that, almost a third of companies have no strategy to improve their data.
While many companies are struggling one, focused on niche real estate who declined to be identified as the company is just emerging from stealth, said publicly available sector data, including documentation from government records is solid enough to be used in as little as a few days.
Public and widely available, however, doesn’t always necessarily mean fast for some firms applying AI.
Iulian Serban cofounder and CEO of Montreal-based education technology company Korbit Technologies says processing the textbooks they use as a base set of data can take months. “There’s a lot of preprocessing and filtering out bad data,” he says. “It’s definitely a challenge.”
A lot of that stems from finding the best examples, those that explain teaching material in the clearest most efficient way possible, among the throngs of textbooks out there.
For marketing companies some clients may not even have the data they need, particularly in retail.
NextOrbit Solutions founder and CEO Kishore Rajapal says the company can get in front of a potential client and agree on the problem they can work at solving and then find they very often fail to have the data the company needs to proceed.
“We need 3-4 years of data and they just don’t have all of that,” Rajapal says.
Or there can be months long gaps in a set of content that misses, say key holidays or seasonal sales information, he adds.
Mahi de Silva, cofounder and CEO of internet marketing firm Amplify.ai, says it can take weeks after deploying its technology before seeing a marked improvement in customer engagement at its clients.
“We will ingest their call scripts…or email systems as we’re trying to build a grammar that’s really unique to that brand,” de Silva says. “Even though there’s a lot of noise it gives us a good place to focus.”
The company uses natural language processing across company websites and social media feeds.
Keeping people in the loops is one key to getting data refined and on the right track, company executives say.
“We bring in human assistance if there’s a miss rate on a certain conversational path,” de Silva says. “Maybe some phrases were not understood.”
The problem is not just an internal corporate issue but threatens the real uptake of the technology.
“AI has gone up and then a bit down in the hype cycle because [people have] got all this great data - they’re drowning in it - but they’re not generating great insights” says James Newell, at venture capital company Voyager Capital.
It makes for an attractive investment thesis for funds like his, he adds. “We’re not only looking at people who are using applied AI but folks who are working to solve the problems that prevent AI and machine learning from living up to the promise that we all know is inherent.”
Read our full conversations with these executives at the links below: