Sunday, July 21, 2024

Intel And Microsoft Work With DARPA To Solve Biggest Analytics and AI At Scale Problem

Datamation content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.

Getting secure global data access to data is a key problem. This problem came up early on as the industry spun up advanced Supercomputers and AI systems like IBM’s Watson to rapidly locate existing medications that could help prevent hospital stays and the loss of life due to the pandemic. The data that could provide the answers the researchers needed was spread worldwide in encrypted databases, uniquely secured, and had little in common with each other.

Tools were created to get around the technical problems associated with each unique database, but regional laws protecting patients’ privacy and that required the data remain in-country remained problematic.

A technology, Homomorphic Encryption, was identified that gets around these restrictions without causing a data breach or violating laws protecting the privacy and prohibiting transport out of the state. Intel and Microsoft have partnered on a DARPA program to bring Homomorphic Encryption into the mainstream, and it should make AIs, particularly medical AIs, far more capable as a result.

Let’s talk about that this week.

Homomorphic Encryption

Protecting information, particularly in the medical field and in various other areas, is critical, and much of that critical information is encrypted for that reason.

But encrypted data is pretty worthless unless you can access it. But if you have to decrypt it for access, that provides an opportunity for a broad breach because unencrypted data isn’t protected. If you transport encrypted data, you also have to transport the keys to access it, and if you are out of the country, you have to do this legally, which, for medical information, is problematic. The world needs to analyze the data where it resides without having to decrypt it or expose the keys that would allow a hostile actor to decrypt it.

What makes Homomorphic Encryption unique is that you can query the encrypted data without decrypting it. The analysis results can then be sent to the analyst, or increasingly the AI system, for analysis without surfacing details on the people who contributed this information. Validating the data, which would likely require access to client information, can be done in-state and entirely in compliance with the law. In contrast, State analysis doesn’t require access or transport of this information to analyze it.

By using always encrypted data, there is no opportunity for an attacker to gain access to the secured data repositories. In effect, the data is being analyzed because that data is never decrypted. But there is a downsize; currently, if you move to a fully Homomorphic Encryption process, you massively increase the system’s performance requirements and introduce significant latency issues.

The process, while safe, also massively resources intensive.

The Intel Microsoft Fix

Intel, working with Microsoft, has announced the creation of Homomorphic Encryption-focused ASICs (application-specific integrated circuits).

These focused parts, once created, should be able to significantly reduce the cost, in performance and dollars, of a Homomorphic Encryption solution. The result, when fully realized, should deliver a significant improvement in FHE (Fully Homomorphic Encryption) workloads over existing systems using off-the-shelf CPUs or GPUs.

Expectations are that the resulting targeted solutions can improve processing time on this always encrypted data by five orders of magnitude. This improvement will take the solution that solves one problem, security, while creating another performance, to a far smaller performance and price tradeoff.

While Intel mainly focuses on the ASIC, Microsoft will use its expertise in cloud infrastructure, software stacks, and Homomorphic Encryption to complete the solution enabling far broader data sharing across borders and potentially a far more rapid response to Pandemic events.

Wrapping Up: This Will Take A While

The industry needs a secure way to access data worldwide if analytics systems, particularly AI deployments, will grow to their full potential. Intel and Microsoft working with DARPA (called DARPA Drive), are beginning a process that will likely take several years to create an ASIC-based solution that will work across borders to enable the secure analysis of data wherever it resides in the world.

These companies, in parallel, will work for international standards to assure that when this technology is ready, it can be widely deployed all over the world so that the next time the world needs a global solution, that solution won’t be delayed as much by security concerns and restrictions on access.

This effort is the kind of thing we’ll need to solve big global problems like pandemics and assure we’ll never again have to go through months of shutdowns and lockdowns while researchers work to get access to the data they need to create a remedy or cure.

Finally, this will also be critical for AI use cases that can then better anticipate problems and provide far better results because they can access data all over the world like Global Warming. This effort, as a result, maybe one of the most critical efforts to assure the analytics and artificial intelligence of the future are the true solutions we believed they always could be.

Subscribe to Data Insider

Learn the latest news and best practices about data science, big data analytics, artificial intelligence, data security, and more.

Similar articles

Get the Free Newsletter!

Subscribe to Data Insider for top news, trends & analysis

Latest Articles