Monday, June 17, 2024

Microsoft Azure: Big Data, IoT and AI in the Cloud

Datamation content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.

By now, you’re probably well acquainted with Microsoft Azure’s infrastructure as a service (IaaS) and platform as a service (PaaS) catalog. However, your IT managers and developers may now be clamoring for solutions that can handle more advanced workloads, like big data analytics, complex IoT deployments and AI-enabled applications.

Cloud Storage and Backup Benefits

Protecting your company’s data is critical. Cloud storage with automated backup is scalable, flexible and provides peace of mind. Cobalt Iron’s enterprise-grade backup and recovery solution is known for its hands-free automation and reliability, at a lower cost. Cloud backup that just works.


Like all major public cloud providers, Microsoft is going beyond the basics and catering to the growing market of cloud-based intelligent services. The tech titan’s sprawling cloud ecosystem can sometimes make it tough to explore one’s options, so here’s a guide to help you get started.

Azure HDInsight: Cloud-based Foundation Big Data Analytics

In terms of IT priorities for 2018, business intelligence and data analytics came in second only to cloud services in Gartner’s recent survey of government CIOs. When it comes to IT spending, they came in third behind the cloud and cybersecurity.

All types of organizations, from multinational corporations to local governments, are looking ways to derive more value out of the mountains of data they’re already collecting. That’s where Azure HDInsight comes in.

With Azure HDInsight as a foundation, enterprises can build big data analytics and business intelligence applications based on leading open-source frameworks, including Hive, LLAP, Kafka, Spark, Storm, and of course, Apache’s ever-popular Hadoop.

There are no shortage of cluster deployment and configuration options, along with third-party integrations. For example, Microsoft just announced the ability to run Spark on a GPU-enabled clusters, allowing users to make quick work out of their big data workloads.

HDInsight isn’t Microsoft’s only cloud-based big data offering.

Based on HDFS (Hadoop Distributed File System), Azure Data Lake Store enables petabyte-scale storage of practically any type of data (structured, unstructured and semi-structured) for massively parallel analytics. Azure Data Lake Analytics, is a companion service that enables data transformation and processing.

In terms of pricing, a good rule of thumb is to refer to each service’s pricing page. For starters, prices can vary from region to region. Microsoft is also known to slash prices, a popular tactic in the competitive enterprise cloud-computing market.

For example, Microsoft recently slashed prices by up to 52 percent on HDInsight. Across the U.S., a base HDInsight General Purpose node with a single CPU and 1.75GB of RAM costs 7.7 per hour to operate.

It’s also a good idea to bookmark Azure’s status page for potential outages and service interruptions and the regional availability page to see if a desired service, big data or otherwise, is available in your market.

Azure IoT: Deploying and Managing Large-scale Internet of Things Deployments

As businesses soon discover, implementing, securing and managing enterprise IoT solutions is easier said than done. Microsoft offers cloud services that serve as the IT backbone of expansive IoT deployments encompassing millions of devices.

A good place to start is Azure IoT Hub. As its name suggests, it enables businesses to connect, manage and update their ‘things.’ Once configured, it can be used to gather telemetry, issue commands to remove devices, register devices with the its Device Provisioning Service, and more.

Another key aspect is Microsoft’s approach to IoT security, often considered the biggest roadblock to enterprise adoption. To help ensure that an organization’s devices don’t end up part of an attacker’s botnet, or worse, endanger user safety, Microsoft employs a flexible, per-device authentication scheme.

“Set up individual identities and credentials for each of your connected devices—and help retain the confidentiality of both cloud-to-device and device-to-cloud messages,” states Microsoft on its website. “To maintain the integrity of your system, selectively revoke access rights for specific devices as needed.”

Once a business establishes its IoT environment, they can use Azure IoT Suite to capture and analyze the data produced by their devices, particularly their Industrial Internet of Things (IIoT) equipment.

It supports various third-party integrations, including SAP, Salesforce and Oracle, allowing organizations to incorporate device data into their businesses processes and workflows. Azure IoT Suite can serve as the basis of connected factory, remote monitoring and predictive maintenance solutions, among several other use cases.

Azure Stream Analytics enables organizations to capture and analyze IoT data in real-time. Companies that already use Power BI, Microsoft’s cloud-based business intelligence (BI) and analytics platform, can use the service to quickly spin up interactive dashboards based on the data it produces.

For organizations wishing to offload a significant portion of their IoT processing to edge devices, Microsoft is currently running a beta of its aptly-named Azure IoT Edge service. Many devices, including inexpensive maker boards like the Raspberry Pi, are no slouches in the data processing and storage departments.

By capitalizing on this processing power and onboard storage capacity, business can extend AI capabilities (more on that later) and other services to their devices. They can also be selective about which cloud resources they devote to IoT data processing, reducing the bandwidth and storage costs associated with transferring those workloads to the cloud.

Finally, Microsoft recently launched a fully-managed IoT software as a service (SaaS) product called Microsoft IoT Central. It borrows many elements from the services listed above but dispenses with much of the backend work, allowing organizations that may be lacking in cloud development skills to quickly connect, manage and draw insights from their devices.

Microsoft Cognitive Services: AI-enabled Apps and Services

The Microsoft AI Platform is where to go to tap into the company’s growing ecosystem of artificial intelligence, machine learning and deep learning technologies.

It contains several cloud services that organizations can use to add intelligence to their business applications or launch entirely new intelligent applications. The most attention-grabbing of these is the company’s Cognitive Services suite.

Microsoft Cognitive Services features APIs that enable developers to add human-like attributes to their solutions. They include image processing and recognition, natural language recognition, text-to-speech, intelligent search and knowledge mapping for semantic search and intelligent recommendations.

For businesses seeking to engage their customers in a conversation way, Microsoft makes available its Azure Bot Service. Developers can create bots that that link to Cognitive Services to light up services that can deliver targeted information, drive sales or improve business productivity with question-and-answer experiences that come as naturally as striking up a friendly chat.

AI developers and data scientists can use the company’s Machine Learning services to build, deploy and iterate on their AI models. It includes various tools, including Docker-based model-management capabilities, a downloadable Workbench application for Windows and Mac machines and Experimentation service that uses scalable Spark clusters to help users fine-tune their models.

Also available are Visual Studio Code Tools for AI that help users of Microsoft’s popular IDE get a running start on AI software development. Finally, for organizations wishing to extend AI functionality to their IoT Edge devices as mentioned earlier, Microsoft makes available an AI Toolkit for Azure IoT Edge for just that purpose.

Bottom line:Microsoft Azure has plenty of options for businesses pursuing AI, IoT and big data analytics technologies as a means to drive digital transformation and create additional value from the data that they collect. Although it trails behind Amazon Web Services (AWS) in terms of market presence and overall popularity, Microsoft’s developer focus and global data center footprint, along its underlying enterprise-grade cloud infrastructure, are some of the reasons Azure is worth a look.

Pedro Hernandez is a contributing editor at Datamation. Follow him on Twitter @ecoINSITE.

Subscribe to Data Insider

Learn the latest news and best practices about data science, big data analytics, artificial intelligence, data security, and more.

Similar articles

Get the Free Newsletter!

Subscribe to Data Insider for top news, trends & analysis

Latest Articles