Google’s Cloud-Based Big Data: a Competitive Edge

hi I'm James McGuire and our topic today is managed big data services and Google's data analytics tools to talk about that I'm joined by Fausto Ibarra product director product management at Google Cloud I felt so how are you gonna die hello James great dozen to you good then and you are there in the Sunnyvale campus correct we're in Sunnyvale yeah at Google cloud pack headquarters fantastic so thinking about you know big data manage services obviously it's a growth sector because a lot of companies don't want to deal with building their own big data infrastructure a lot of complexity involved there's expense involved there's your headaches doing things like spinning that peg goop clusters so clearly you know big data as a managed service is growing I'm wondering what you think is basically happening with the secretary of these days and what impact it's having out on the analytics community you know to what extent is Big Data managed services changing the relationship between companies and their data what's going on there yeah sector is going to an incredible transformation something that we haven't seen any time in our history we companies are moving from having to think about the infrastructure to just completely focusing on their application so it used to be the case that companies would have to buy a lot of expensive hardware if they wanted to have a data warehouse or buy many servers they wanted to deploy a Hadoop cluster now they can come to the cloud for example we put a platform they can use services like bigquery which allows you to have a data warehouse without having to worry about the infrastructure at all all you have to do is log your data and start clearing it so that's basically frees up a lot of resources so that companies can focus just on their data just on their applications and just on empowering their different users to ask questions of their data do you think it's a it's a pretty Universal move among companies to go ahead and say hey I want to I want to do a big data manage service offering or there's still plenty of companies to go oh no we're going to we're going to build this ourselves in-house and then do it ourselves in house yeah I definitely a majority of companies today I think companies are definitely thinking twice before making any kinds of investments we have customers for example like uh Spotify so Spotify operate world-leading music service they actually starts building their own data centers alright okay a lot of servers but quickly they realize that where the value was was basically their data understanding their customers what music do they like creating a playlist for them new product and so on so basically they've embarked on a project to mobile all of that to Google cloud platform and the way they think about it is they have limited engineering resources and they'd rather have those engineering resources working on applications working on machine learning working on analytics rather than managing Hadoop clusters or their data warehouses so like Spotify we see many customers in financial services healthcare media and entertainment and factoring public sector and so on who are moving to the cloud directly without having to worry about the infrastructure anymore I'm going to talk about some of Google's data analytics tools themselves I know there's a there's a few come to the tip of the tongue there's a you know cloud data pockets a spark and Hadoop service bigquery you mentioned that that's pretty well known enterprise data warehouse enables people to to analyze large you know vests or Hussle data dataflow is a tool for developers correct me if I'm wrong you know what am i leaving out there I decide how our company is using some of these tools in the real world yes in a interesting way so for example with a child at appraoch it allows companies to be blown you how to four star clusters literally in a minute so it's not to analyze your data you know in 60 seconds they can do a cluster whether it is you know 10 nodes under stones or even a thousand nodes and they can start analyzing their data right away so we see companies who are migrating from existing Hadoop clusters that they had on-premises to the cloud we also do customers where it's going new applications using how to a new scene tell data persuaded at one of a very popular service being where you have to mentioned a Cal data warehouse used by a lot of our customers from small startups all the way it's very large enterprises we have companies who are migrating you know existing data warehouses using legacy technology they're moving them to the cloud we also do companies who are using bigquery to do analytics on their marketing data their advertising data financial services doing reconciliation and so on and to climb any of these products together that's work at cloud dataflow campaign today the flow allows you to do both a bass and stream processing pipelines so here when I get data very easily from multiple sources into multiple destinations all you have to do we define what your data pipeline looks like and we'll take care of all the infrastructure will scale it will make sure that if there aren't really well we'll make sure that it's secure so you can worry just about your application and nothing else and a data flow is is really it's a developer tool correct me if I'm wrong there whereas bigquery you might not need to have developer skills to be really interfacing with it true or false sir yeah so data flow is actually both for developers and for IT ops people so we actually see a lot of ideas people who are using need to implement data pipelines for example getting data from your on-premise data sources to a data warehouse and bigquery is used by similarly by IC by developers and also by business analysts right now for example bigquery can easily integrate with all of the popular data visualization tools like tableau or take view or MicroStrategy so developers can be to get access to their insights writing the tools that they know they're for being able to make smarter decisions and being able to get access to all of their analytics now a new class of users that we're also finding are using cloud if data scientists hmm it isn't like all of these tools because companies today are looking to make their applications smarter they're looking to build intelligence and machine learning into their apps directly and we offer a full set of machine learning API and a service report opportunity learning engine to require custom machine learning models we can it's really nicely with all these other products is to do things like product recommendation or protection or you know image recognition speech recognition a lot of really exciting new applications that are comfortable now with the cloud well that's somewhere there is tensorflow correct the machine learning the library absolutely so Google has been doing machine learning for many many years we used it in Google search Google Maps YouTube etc so about the year and a half ago we open-source tensorflow it has become the most popular machine learning framework in the community Ron we offer the ability to run tensorflow in google cloud platform and same principle as with the other products you don't have to worry about any infrastructure you don't have to worry about provisioning clusters you get access to our specialized hardware we announce what we call the tensor of protein units or GPUs that are basically designed for machine learning for training and prediction so you can employ any kind of machine learning model no matter how large it is without having to worry about the infrastructure we also make everything so that companies can focus on their applications right you're essentially you mentioned you know data scientists as a new class of users that are coming on board is there anything that you're finding the data scientists are looking for or wanting but sort of the earlier users weren't weren't as savvy to or what what effect is this new this new community of growing community of data scientist Evan we are given the most important change that data scientists are looking for speed of iteration that they want to be able very quickly go from I have a lot of data - I have insects from the data - I have a model for applications that can do that so that plays really nicely with stuff from because we offer you a very strong performance again you don't have to worry about any kind of infrastructure so it used to be the case that ideas that scientists would create a machine learning model and maybe they would have to wait 24 hours for that model to be trained to be able to resort now they can do that in a minute so just like you can iterate multiple times even in a single day and that has given rise to an exponential increase in number of data scientists in fact as you may know we acquired kaggle right companies address we have the biggest community of data scientists in the world a caracal b acquisition of calgary was in the spring of a fire remember Christ in the spring and we just announce that we cross 1 million developers and it is basically part of that community and they're basically all taking advantage of new technologies to be able to yeah get in fact that were impossible before and to acquire new skills as well that help their companies you know deploy more sophisticated applications that's a fantastic community of knowledge knowledge folks that's fantastic interesting think about the future I mean obviously there the need for big data and big data managed services is exploding if there's you know data from mobile there's data coming in you know from Internet of Things as this is really growing rapidly think about billions or trillions of devices each giving information I mean where you see big data big data in the cloud going in the next several years if you can put yourself in the mind of a hypothetical business owner who needs to you know track where this is going say one three five years is outing what kind of advice would you give to that person in terms of keeping up with big data in the years ahead yeah definitely want to make keep transmission learning and artificial intelligence and like being is that you know ml and AI will be part of any every application right have to think about it that's a separate category so if you are building a data warehouse the data warehouse will have AI built into it yes you'll run predictions on your data right there another big trend that we're going to see is the trend towards real-time proximate data yeah IOP we're a company for example our manufacturing customers may have you know millions of devices connected and they wanted to analytics based on that so they want to live in real-time like one of our customers is coca-cola European partners but they provide an application using beacons that basically allows them to to promotions to their customers as they get close to a retail location Location Georgia country fridge we coca-cola product and then you can automatically get a promotion but in certain mobile app plus IOT and then coca-cola uses our big data tow services like bigquery to be able to do analytics based on that so can you imagine you can actually understand your customers where they are what they bought and then you can target your promotions to them so we're going to see more and more applications like that which bring together IOT with big data and with machine learning to derive the insights that were impossible before fascinating it seems like it's going to be a real competitive edge companies will absolutely need to be leveraging artificial intelligence and machine learning to stay competitive in the years ahead clearly this it's not optional absolutely it's definitely going to become one of the biggest sources of competitive advantage and another very important thing is that every business entity every developer will get access to these tools for example I was going to one of our enterprise a retail customers recently I'm going to do that they said is that they want every one of the business analysts to be using machine learning and getting analytics without even having to learn anything new so we're going to see more and more to essentially build build that intelligence into the tools so that everyone can benefit from them whether it is to do product recommendation to analyze you know in a call center situation what the conversation was all about to be able to proactively do maintenance and so on any applications are possible do you think that it seems like one of the increases is nature of the user interface in the optimistic data scientists will be able to understand a far more complex user interface and say a sales rep in the field perhaps is there a move towards creating a simple user interface or is it more about a deeper feature set yeah I think there's most definitely source having a different types of simple user interfaces but they have to be adapted to the specific customer needs if you're a data scientist you may be interested in using something like cattle you know cattle we offer a way to like download data set to a very quickly start interacting with your data to publish and share your results with your colleagues can be able to do all of that within a secure environment if you're a business analyst - may be interesting something more like a data visualization tool in both cases the tools are simple but they cater to the needs of specific users for developers in developers like interacting with products using their popular languages so it supports all the popular languages that they may be interested in so that they can build that those B capabilities and and machine learning into their applications so there's definitely a move towards simplification of these interfaces but they have to be catered to the users needs may foster I think you said it's absolutely fine it's really fascinating it really interesting stuff I'm sure that you know obviously the markets going to grow usually in the years ahead I appreciate you sharing your expertise with us this afternoon I will send you the link when it's all done and thank you very much well thank you for the opportunity James always the pleasure

This transcript was generated automatically from the video's captions and may contain errors.