Big Data Trends and Best Practices

hi I'm James Blair and our topic today is Big Data we'll talk about best practices for Big Data and we'll talk about the future of data analytics to discuss that I'm joined by Steve Stein chief data officer for AT&T hi Steve how are you doing today good afternoon James nice to be with you thanks for the opportunity to discuss big data and any other topics you have on your mind yeah well you are there in in sunny Texas tell me the city today I'm in Dallas Dallas okay fantastic I assume it's a sunny day today and in Dallas well actually it's cloudy but that's a rarity so we'll take the clouds in cool weather so essentially your job tail is actually chief data officer that's a pretty new job title I mean I think the officer the job with CBO did not exist several years back what's going on with that so in August of this year when John Donovan was named CEO of ATT communications Melissa Rinaldi took over as president of ATT Technical Operations so um in running that role she introduced the CTO and I was given the opportunity to lead the CDO organization I'd previously worked for John around a group that we had set up to do automation solutions really aimed at more of the hyper automation you know how to bring machine learning AI together but one of the things that we always found that was interesting it was you know we needed data to drive where we were going and to make sure that we were doing the right things as we looked at automation for the business and so the Big Data organization existed we worked very closely with them and then the other piece of that of course is you know the really the data engineering and you know getting the data into the lakes into the warehouses or wherever you need to do analytics for your scientists so Melissa asked me to take those three organization the data supply chain the Big Data and as well as the automation solutions group bring those together to form the sea so that happened on August 15 of this year all right it's a brand new gig and you're you are a pioneer in this new new job title well is a new gig and I'm really excited about working with all of the not only the remarkable team that I have within the CDO but with our business partners to ensure that they have what they need to do to make the business run successfully great yeah I'm interested to get your thoughts in terms of what advice you might give to companies about handling big data because I think a lot of companies have gotten onto onboard with big data in the last few years and they thought to themselves great we can put all the numbers into the system we're gonna get insights back we're gonna not gonna have to like you know fly by the seat of our pants as we did in the old days but it hasn't always exactly worked out that way I wonder what advice you might give and maybe a couple of best practices for big data leveraging yeah I think as simple as it may sound the biggest piece of advice I give is make sure that you know what you want from your data so what it is that you're trying to get out of it so we spend a lot of time with our business units working real closely with making sure that they have what data they require in order to make the decisions that can you know help our customer experience cost structure revenue etc and so just really staying close to your partners in making sure that you understand their needs prioritizing those in order to make sure that it's timely and it's cost efficient for them to get their hands on it you know we as you probably are aware we handle a large amount of data I think the last number I saw was a hundred and sixty eight petabytes traversing our network on a daily basis going on there and then behind that you've got operational data within the network and other components so I think making sure that you know what are the elements that you need for the business to be equipped to be successful is job one of ours so that's if I say my simple find that down correctly by saying you've got to know what question to ask the system is that but bottom line there are not necessarily yeah it is but I want to emphasize it's not really about us in the CDO going to the business units with technology or you know the or the latest algorithms and asking where the business can apply them it's more saying what would they like to accomplish and then helping engineer for them what they would require whether that be you know datasets whether it be automation solutions or anything else that'll make them successful yeah what challenges do you see out there for handling big data I think it can be you know remarkably confusing and you know some businesses that have data scientists on staff some don't even the folks who have data scientists on staff I think they're ready to pretty much confusion it are there are there challenges you see out there that are sort of like the default challenges well I I'd say it begins with you know as I mentioned earlier does your client have the data that they need in order to make decisions and so you know what we try to do is once we know what that is put it in in in those that needs to knows hands but let their analyst work on it as well we like to to utilize a distributed model I would hate to think that we just as a group of data scientists were the queue for everybody waiting for the next big brain idea to come from the you know the remarkable team that I have but I really need to push this out equip the business with the self-serve self enablement type of model so you know the business really needs to to also work on what their analytic capabilities are once they understand you know the things that they're trying to do to impact positively like I said experience cost structure etc so you're saying that folks you know in the cubicles of the folks out in the field can also get insights from the system is not just the folks in the c-suite yeah I would tell you that if that's not happening and you're not enabling people to do that then you're gonna miss out on a lot of opportunities even with the you know the advances in technology where you can get into machine learning so you can start to process larger quantities of data and get finding from that data you still want those that are closest to whether its operation sales care that are closest to action they're seeing things that are going on every day they should be able to take that data that they know is being generated by performance or you know the things that our customers need and really try to work on on understanding themselves on on what it is that they may want to take play you know they may want to do to have a positive impact right what about the cloud and data analytics seems like for a lot of companies the cloud is their first choice they're accessing their solutions through the cloud is should the cloud be the automatic choice it seems to be these days well needless to say there's a lot going on in that space and it's very exciting and you know our approach is that we need to consider all elements as in and all opportunities frankly so you know we have warehouses we have lakes you know that we're managing on half behalf of the business will we will have some of those you know long into the future you know we want sure use the high cloud appropriately as we move forward so I see a hybrid mix for you know what we're trying to do it's not going to be one-size-fits-all particularly because of the you know the legacy information that we have available you know just moving it into the cloud alone and it would be a challenge in itself and the other thing that we want to make sure of you know we keep really close eye on this is all activity and you know the safeguards around data we want to be confident that wherever we are working with data it has all of the you know the things that we would expect you know to safeguard privacy and the things that we you know care for on behalf of our customers in the business right what do you think about the future looking a few years ahead when we get back together and have this conversation a couple years from now what are we going to be talking about in terms of big data in other words as companies ramp up their own systems what should they be thinking about to get ready for 18 months from now yeah I would tell you the first thing we'll be drastically different we'll be the technology and I only say that because of the rate that technology is being developed in the space now whether it's analytics whether it's automation including AI you know all of those components mean there's a lot of investment a lot of energy going in into this space we know that there's you know some very high yield coming out of it as well so I don't think that pace will it's certainly not going to slow I think it will accelerate so you know understanding you know that evolution of technology and how you apply that to the environments that we're trying to operate are gonna be absolutely critical so that would probably be the one thing that you know I would say would change the most and then the other is I do believe that we're gonna learn a lot more as we go along you know not all data is created equal and nor does it need to be and so you know being able to determine very quickly what you need from raw data and how far how far along you need to mature it in order to give value for the business that you know you may not need entirely business ready data sets for people to be successful you know and take care of our customers and so I think what we will see a big opportunity to really understand you know how much are your data you actually need to be shaping in order to get the most out of it I'm not sure I understand that you're saying that you can work with a partial data set you don't need to be perfect before you start the process yeah absolutely I think that they're depending upon what you're trying to resolve you know you you may not need to have a complete data set in order to do that and so I think we're going to learn a lot more of that as we go along I mean needless to say there are some areas where you you you want to have a complete data set when it comes to things like network operations obviously security items you know intrusion I mean we those are mission critical things that you know we want to be very very certain around there may be other things that we say you know we have enough to get going let's go ahead and see what we can do with this before we have to spend a lot of time and energy and cost you know the the investment in order to mature some of the data right what about the AI piece that to me is fascinating if some of the things going on where the Machine develops something of an independent entity machine can to a certain extent can can teach itself do you see how do you see that shaping corporations these days well I think it will will have an application in I I can't imagine any area where you wouldn't benefit particularly when you're handling large amounts of data from the machine insights now you know in order to get there you have to get the data to the Machine you have to give the Machine the opportunity to learn you know you still need people involved with it to make sure that it's learning as you would expect it and then ongoing you know overseeing the learning to ensure that you know the outcomes that you're expecting continue hopefully to improve and so you know the I think that the key is going to be finding you know when it should you just programmatically put machine learning against data sets once again not all data is required to do that and would benefit from that but where would it apply so I think that you'll see you know a continued emphasis investment in this area the technology will continue to move along very quickly I think more algorithms will be available you know be interesting to see what would be proprietary in this versus open-source right as we go forward that will really accelerate this in and let many people take advantage of of findings as they happen state I would you started is there anything you'd like to add before we wrap up today now I appreciate the opportunity to visit with James you know appreciate the work that you do the insights that you share you know with all of us that follow you so thank you for your time and we'll look forward to talking in the future Steve thank you very much I'll send you the link when we're done with you know law or tweet about it okay thank you

This transcript was generated automatically from the video's captions and may contain errors.