Episode: 548 |
Adam Braff:
Business Analytics Diagnostic:


Adam Braff

Business Analytics Diagnostic

Show Notes

Show Notes:

The Umbrex Business Analytics Diagnostic Guide that is discussed in this episode can be downloaded at no cost here: https://umbrex.com/resources/business-analytics-diagnostic/


In this episode of Unleashed, Will Bachman and Adam Braff discuss the creation of a data analytics diagnostic guide. Adam, a former partner at McKinsey and a consultant on data analytics, discusses the importance of data analytics in solving business problems in any company or investment firm. He explains that a business analytics diagnostic is designed for organizations with multiple people, computers, and analytics processes. The goal of this diagnostic is to determine the performance and alignment of the data science or analytics function with the overall mission of the company. He explains the size and type of company that uses this and who would monitor and manage the data analytics of a company


The Diagnostic Guide Format Explained

The diagnostic guides follow a format with scorecards for individual pieces of an area, typically 15 to 25 different scorecards, and within each one, objective criteria ranging from nascent to optimized. These guides are divided into categories and subcategories, such as analytics strategy, data management, advanced analytics, AI, talent, decision-making process, tools, and infrastructure. Adam explains the format of the diagnostic guide, beginning with top level categories including analytics strategy, strategic alignment, performance measurement, and future roadmap. Analytic strategy involves understanding the business objectives and problems to be solved, such as growth, customer retention, risk management, and problem-solving. Strategic alignment also involves determining the location of analytics people, whether centrally located in a Center of Excellence or distributed across different functions. Performance measurement involves tracking key performance indicators for the analytics function, such as cross-sell, revenue, pricing, and marketing ROI teams. Benchmarking this number against competitors can help determine if the company is on track and if it is underinvesting in analytics. Performance measurement also includes ROI, which is the understanding of specific goals and projects that the analytics team is working on. By tracking these metrics and reporting the total impact analytics has on the business each year, the analytics strategy part can be evaluated.


A Roadmap for the Analytics Strategy

Adam emphasizes the importance of having analytical people distributed throughout the business and dedicated resources for analytics initiatives. To round out the analytic strategy, it is crucial to have a roadmap of the next eight quarters, such as tackling Net Promoter Score analysis, customer satisfaction drivers, or adopting a new data management tool. This roadmap should include hiring and development strategies, cutting-edge innovation, and research, which can be revisited and changed strategies as needed. This helps ensure the analytics team is effectively working towards achieving their goals.


Data Management: Warehousing, Sourcing and Integration

Adam goes on to talk about the importance of warehousing, data sourcing and integration involving sourcing data from internal systems or external sources, such as customer satisfaction surveys or third-party surveys. This is crucial for asset managers who need to acquire data for investment analysis and decision-making. Automating data loading processes is also important, as it allows for efficient data flow. Business intelligence is another important aspect of data management, which involves creating interactive dashboards and alerts for all stakeholders. Data quality is a critical aspect of data management, involving conscious decisions on the quality of data. More mature businesses have higher standards for accuracy, timeliness, and completeness of data, with constant profiling and monitoring to ensure data meets these standards. Data governance encompasses coordination across different parts of the business, ensuring consistency in data definitions, appointment and training of stewards, and governing data for regulatory and compliance purposes.


Advanced Analytics and AI-Driven Decision Making

Adam discusses the importance of analytics in a company’s operations, particularly in areas like operational analytics and revenue. He highlights the need for centralized, advanced analytics functions that focus on predictive modeling, machine learning, and AI-driven decision making. These functions should be evaluated for their maturity and effectiveness. Another area of focus is AI-driven decision making, which involves how a company uses AI to improve operations.  He goes on to talk about talent management and three main areas: people, performance, and technology and how these tools can be used in this area. Training and development are crucial aspects of analytics talent management. This includes understanding skill gaps within the team, designing a curriculum to fill them, and providing continuous learning opportunities. Internal or external certifications and specializations can also be beneficial. Lastly, community engagement and collaboration are essential aspects of analytics talent management. This involves sharing knowledge with the organization, building collaboration, and engaging with external partnerships and networks. Adam explains how innovation and co-creation initiatives can help spur creativity and innovation within the analytics team. These efforts can be internal or external, pushing the envelope on innovation and ensuring the success of the business. Overall, analytics talent management is a critical aspect of a company’s operations.


The Decision-making Processes in a Data-driven Culture

The decision making process involves three buckets: data driven culture, analytical decision making, and predictive decision making. A data-driven culture focuses on controlled testing of experiments and measuring things rather than relying solely on intuition. This includes tracking demand for analytics use cases, managing cultural change, and ensuring data accessibility and democratization. Analytical decision making starts with analytical frameworks and tools, such as customer lifetime value frameworks and CLV calculations. It also involves decision-making process integration, ensuring checks are in place before recurring functions occur to ensure data analysis is involved. Performance tracking and feedback are essential for comparing individual decisions made with data to the overall function. Adam explains how and why analytical decision making is used, and how predictive decision making involves planning out budgets for next year, understanding macroeconomic impacts, weather, and operational and financial budgets. Predictive analytics can help manage various risks, such as customer numbers, macroeconomic impacts, and weather. Predictive data is used for strategic planning questions, forecasting sales, and risk assessment. He explains how infrastructure scalability involves capacity planning and management, disaster recovery, and business continuity. Analytics diagnostic guides can help organizations prioritize their future state and decide what they want to invest in. Consulting firms should consider the bigger picture strategic choices, such as whether they are a data-driven company or if it’s not important to spend time and effort on data and analytics. Companies may also want to focus on specific examples of demand in the business that they don’t know about today, which can help them make better decisions.


Data Analytics: Tools and Infrastructure

Adam talks about the various platforms that can be used, and how choosing a point along the continuum of low maturity, intuitive, data-driven, and algorithmic can help companies determine if they want to be more analytical or not. By understanding the needs and preferences of their clients and identifying areas for improvement, businesses can make informed decisions about their future state and investment in analytics. He talks about the importance of being able to integrate tools, scalability, fitting the needs of the business and customers, and the ability to customize the tools. Adam discusses the concept of a company’s approach to building capabilities and whether they want to be an analytical firm or not, and which analytics will help the business. He suggests that companies should make strategic choices about centralized or distributed analytics functions, monetizing external data, and maintaining a high level of customer consent. He also suggests that companies should build these capabilities aggressively, gradually improving over time, and that companies should start with quick wins on important use cases and gradually build on more complex ones, such as marketing ROI models. 

For listeners interested in learning more about his practice, Adam recommends visiting braff.co, which offers resources such as a blog, an annual forecasting contest, and programming course. He also mentions that he has taught this content in graduate programs at Brown and NYU and has started teaching a corporate version of the analytics intensive course.



01:18 Setting up data analytics function in a company

07:02 Analytics Strategy and Measurement

12:52 Data management categories and sourcing

16:12 Data management, analytics, and AI in businesses

22:06 Managing and developing analytics talent

26:50 Data-driven decision making and analytics in business

29:13 Data-driven decision making and analytics tools

34:44 Data analytics maturity and strategic prioritization

40:17 Building a data analytics function for a business



Website: https://braff.co


One weekly email with bonus materials and summaries of each new episode:


  1. Adam Braff


Adam Braff, Will Bachman


Will Bachman  00:02

Hello, and welcome to Unleashed. I’m your host will Bachman and I’m so excited to welcome today Adam Braff, former partner at McKinsey, who is a consultant on data analytics. Today, we’re doing something exciting. We haven’t done this before. Umbrex has published a series of diagnostic guides, perhaps you’ve seen one or two of them. We have one, supply chain diagnostic guide, sales operations, diagnostic guide, one for talent management, one for organizational design, one for branding. And today, we’re going to create one with Adam, He’s put some thoughts together on what a data analytics diagnostic guide ought to look like. And after our conversation, we’re gonna go ahead and produce the whole thing. So you can download it. And today, we’re going to talk through it. Adam, welcome to the show.


Adam Braff  00:53

Thanks. Well, it’s great to be back.


Will Bachman  00:56

So you’ve put some thoughts into tell me what you what you’ve come up with, I think I see in front of me, we’re talking through a structure for the categories and subcategories, take it away.


Adam Braff  01:08

If we’re talking about a business analytics diagnostic, I think we have to put some terms into play here, just so we understand what we’re talking about. This is really the function that involves data and analyzing data to solve business problems in any kind of company or investment firm. I was thinking if there are any businesses that are too small to benefit from this, and I think there are some, I think if it’s like a, like a one person bakery or something that they’re not going to really need a data analytics diagnostic. But anything that’s got multiple people, it’s got computers, it’s got analytics going on, there is at least something going on where people are looking at trends in performance trends in their financials, trying to understand what’s happening, what’s going to happen in the future. And this diagnostic is for those organizations. Of course, most of the people listening to this are either working at big companies, or they consult to larger companies. So they’ll be they’ll be familiar with what we’re talking about. And in those cases, there would be something like a VP of analytics inside of a company or a data science organization, or something that is analytical and artificial intelligence and digital are sometimes in the name, but it’s something that’s analytical there. And the goal of this diagnostic is to figure out how well that function is performing and how well aligned it is with the overall mission of the company and what you can do better. Yeah,


Will Bachman  02:34

so the kind of paradigm for what this is designed for is, well, smaller companies should do some sort of analytics, we’re primarily at least base case, we’re talking about a larger corporation that actually has a data analytics function, someone who’s a VP or director or SVP, or something of data and analytics. And just tell us sort of that, you know, fortune 500 company with that function? What’s that look like? Who do they typically report to they report to the head of strategy or someone else? And what will how big is that function.


Adam Braff  03:12

So typically, the data science or analytics function is going to report either into the head of a business unit, that was the first to have the idea of doing this, or to some function like the chief revenue officer or chief marketing officer, if the types of analytics they’re doing are revenue focused, sometimes it springs out of strategy, like you pointed out or out of the finance organization, it can really attach in any of those places, a lot of the ones that I’ve worked on, because they’ve been a little more revenue focused a little more consumer focused, have ultimately gone up into the chief revenue officer. But that’s not necessarily the case. Also, we should keep in mind, financial services firms and investment management firms are also in scope here. And in those cases, there might be a centralized analytics team, that’s gonna report up into some head of back office or some head of operations for that firm. So it can really it can really go anywhere, but you’ll sort of know it when you see it, like if you are that person, or if you are thinking of starting this function, or if you’re looking at this function, you’ll probably know where it is. All


Will Bachman  04:16

right. So just a preface is that for these other diagnostic guides, and you follow the same kind of format, we’ve we have a set of scorecards for individual pieces of that area, typically 15 to 25 or so different scorecards, and within each one of them some objective criteria, ranging from nascent to optimized, so that just sets it up and and we group those 20 or so because we are consultants into several groupings each for the, you know, four or five, and you’ve done the same thing with categories and subcategories. So with that preface, go ahead and walk us through how you’re thinking about setting up these different aspects. Look at, let


Adam Braff  05:01

me give you the top level categories first, just so you can count on your fingers. And keep me honest here. So the top level categories are analytics strategy, data management category called applied analytics, sorry, advanced analytics and AI. So put that together into one blob, analytics, talent, decision making process and tools and infrastructure. They’re like Jeopardy categories. So for the first one, if you’ll take analytics strategy for 200, or 200. So the three subcategories, there would be strategic alignment, performance measurement, and future roadmap. So what are we talking about here in this big category, called analytic strategy? It’s a little dry and abstract to start with this one, because you think you’d want to start with like, what are the analyses that make the most money for this company, but you do want to make sure that everything that the company is doing in analytics is tied to some business objective. So that’s what this first subset, this first sub part is about strategic alignment, you have to understand what the goals of the business? Are they trying to grow the top line? Are they trying to prevent customers from leaving? Are they trying to manage risks? What are the problems they’re trying to solve? And are there analytical resources arrayed against those problems? In some way? That feels proportionate? Like do you have the right number of nerds working on acquisition versus retention? Also, within strategic alignment? Is there a kind of a business owner, a stakeholder, someone who’s paying for this thing? Who’s the sponsor? And are they really feeling that ownership? Are you? Is the analytics leader meeting with that person? Are there readouts? Is there some kind of engagement with that stakeholder, and then the other stakeholders who have to be involved for this thing to be successful? And then, finally, within this subcategory of strategic alignment? Where are the analytics people located? Are they centrally located in some Center of Excellence? Or are they distributed far and wide? Across all of the different functions of the business and business units that can benefit from it? There’s no one right answer here, right? So if you’re thinking of it as a maturity model, to say, Well, what does optimized look like? The answer is that these choices have been made sensibly with respect to this business. So to take the example of Federation of analytic, if the business has extremely different kinds of sub businesses inside of it, like, you know, Berkshire Hathaway, you wouldn’t want one center of analytics in Omaha, right, doing all the analytics for all these different disparate businesses for Berkshire Hathaway. But if you’ve got lots of different regions of a company that historically were run fairly separately, but they’re selling the same products, and there are a lot of overlaps, and the kinds of data and analytics will do, you would want something more centralized there. So the learnings can be shared more. So that’s the strategic alignment subcategory.


Will Bachman  07:59

Okay, that makes sense. Yeah, great.


Adam Braff  08:02

Performance measurement is also part of this analytic strategy question. And here, it’s about what are the key performance indicators for the analytics function. You could have these people doing analyses all day long, but you want to be able to measure if they’re doing it? Well, in some analytics teams, what they’re doing is so closely mapped onto the businesses, they’re supporting that their KPIs can be kind of the same thing as the business’s KPIs. So if there is a cross sell team and a revenue team and a pricing team and a marketing ROI team, the analytics folks who support them can probably be measured with roughly those same KPIs. But in other cases, the analytics team has a life of its own. And it’s got a set of objectives and projects it’s working on, like a generative AI Initiative, or some kind of analytics training rollout. And you want to make sure that you’re also measuring that group on whether those those things are happening to so thoughtfully selecting the KPIs of the analytics team. And tracking those is important. You also want to make sure that you’re benchmarking that number where possible to competitors, it’s hard to do, because the analytics team can be somewhat buried inside of a company. But if you count the number of people, for example, who are working on analytics for your company versus your competitors, you might get a sense of whether you are way off base and then you want to understand why is it because you are severely under investing in Analytics, which is possibly the case, or is there a reason? And then also within performance measurement is ROI. So the sense of understanding specific things you set out to do like I am going to improve my customer targeting model to improve conversion on the website by 5%. And therefore it’s going to be worth this much money. Are you actually tracking those things and reporting out the the total amount of impact analytics analytics is out on the business each year. So that’s the analytics strategy part.


Will Bachman  10:08

Okay, that you’re showing, like, if the analytics group had never existed versus what we actually did, do, you know, was it worth the investment in the analytics team to, to do it and making your case to keep your funding or maybe even grow the team to measure measure the impact?


Adam Braff  10:26

Yeah, that’s right. And, you know, it can obviously be a little bit of a slippery slope here, there are analytical people distributed throughout the business. And they might have been able to do some version of this without the analytics, folks. But if you have people stacking hands on a set of analytics initiatives, and you’ve dedicated analytical resources toward building those models, and answering those ad hoc questions and building those dashboards, then you should fairly be able to allocate ROI to that team. And of course, you want some margin of safety like with any call it like, call this like an uncertain internal consulting team with any consulting project, you’d want there to be like a TEDx return, so there wouldn’t be a lot of fighting on the margins about whether this team actually is pulling their weight, you would hope that they’re really, you know, shooting the lights out. Yep. Okay. And then to round out the analytic strategy, you don’t want it to just exist as a an a formless void, like at a point in time, you want there to be some sense of the future and a roadmap of where this thing is going. So even if you’re not going to adhere doggedly to this roadmap, you want there to be a roadmap. So you have at least a hypothesis of what the next eight quarters look like, for the analytics team. Is this the quarter when we’re going to tackle Net Promoter Score analysis and the drivers of customer satisfaction and the relationship between satisfaction and profit? Is this the quarter when we’re going to implement chat GBT with code interpreter or is this the quarter when we’re going to adopt a new data management tool. So having all these things mapped out over the next eight quarters, in the technology adoption plan, some sense of how you’re going to be building the skills of the team over time through hiring and development, and some sense of where you’re doing the most cutting edge innovation and research that has to be written down somewhere so that you know where the puck is going, even again, if you’re going to revisit it and change strategies a little bit as the future goes on.


Will Bachman  12:21

So it’s okay to change your plan. But if you do not have a plan, that would be nascent category at the nice yeah,


Adam Braff  12:28

I put that. I put that nascent Yeah, I you know, it’s a little bit hard for us to justify a person being completely unplanned in this in this category, but maybe if it’s like an improvisational theater troupe or something, and they fly by the seat of their pants, but everyone else, you’d probably have something written down on what they want to do with analytics and how to get better over the next eight quarters.


Will Bachman  12:47

Okay, and I think that rounds out our first thing category of analytic strategy.


Adam Braff  12:51

Yes, sir. And on to data management. So within within data management, we’ve got four subcategories data warehousing, data sourcing and integration, data quality, and data governance. Data Warehousing, is what it sounds like, it’s where is the data stored? Is it data lakes? Is it a data warehouse, this is in particular data to be used for analytical purposes. So this is not really about your operational data store, which is a separate category, perhaps for a different one of your friends diagnostics. This is about where do we put the data that we’re going to be using for analytics, ad hoc analytics and reporting? And it includes questions like, How is the data warehouse set up? Is it a bunch of them strung together? Is this where the unstructured data goes? Is this where the structured data goes? What is the lifecycle of data in that warehouse? And does it expire after some point in order to comply with, you know, privacy and other regulations? And is this thing scalable, so that as the business grows, and as we decide there are new categories of data, we want to bring in even at the same size of business? Will the data warehouse be able to accommodate that?


Will Bachman  14:05

Okay, so let me see if I got it. There would be one massive data warehouse for the company for all the data. And are we saying here that there’s a, like a separate working kind of data warehouse for the analytics team that you would PORT stuff over to when you’re doing your analytics on it? Or are you saying something different than I understand? Yeah,


Adam Braff  14:25

no, it’s in a simplified model. That’s basically what’s going on is that big industrial strength businesses have some system that is like basically taking in customer you know, onboarding, new customers, fulfilling orders, you know, like running the actual function of the of the business. And there are extracts that come out of that thing into a place where people who want to look at the data, analyze it, understand it, that’s what they’re hitting, so they’re not the data scientists aren’t pulling data directly out of the core systems of the business. That’s basically what’s going on. Yep. Okay, got


Will Bachman  14:58

  1. All right, data warehousing. All right, data sourcing and integration is next. Yes.


Adam Braff  15:03

So a lot of the data for doing this analysis is internal to the business, it’s going to be like the, if you want to understand how many customers you have, and what the lifetime value of each customer is, you’re going to be drawing on data that comes out of those operating systems into this warehouse. But you also might be sourcing data from outside from third parties. For example, if your big questions are about market share, you’re not really going to know what the answer is there. If you just look at your own internal data. Or if your questions are around customer satisfaction, you have to make sure you’re pulling in the data from customer satisfaction surveys, which might be yours, or they might be some third party. So you have to be good at data sourcing. This is especially important for asset managers, because they have to acquire data in order to make their investment analysis and investment decisions. Also, within this category, are the processes for automating that. So you don’t want to just be hand loading data into this data warehouse, you don’t want to have to have like somebody like shoveling coal into an engine, you want the machine to do what’s called ETL, Extract, Transform, and load operations automatically. And then on the way out. Another important category of how data is flowing through these pipes is business intelligence, which is to say, dashboards, how you make sure that information that is critical for running the business is being pushed out into interactive dashboards and alerts and visualizations that everyone in the business can see and use. Okay, great. On to data quality. So this is largely about making conscious decisions on how good the data has to be. Is this a caveat emptor thing? Or are you stamping and certifying as the central data team, hey, these numbers are right, and they’re consistent across the entire business, I think we can agree that the more mature businesses, you’re gonna have more guarantees around data quality, they’ll have higher standards for accuracy and timeliness and completeness of data. And there will be a constant profiling of the data and monitoring to make sure that the data is actually meeting those standards, and a set of initiatives to improve that and to run down any problems and trace the root cause where the data is breaking. So that’s the data quality function. Okay. And then finally, you’ve got a sort of broader category that wraps around data quality, which is data governance, and that is to say, how are we coordinating across the different parts of the business to make sure we’re all using the same data definitions that we have a counsel or somebody we can appeal to have two different parts of the business are defining profit margin differently? And it’s annoying the CFO? Who are the stewards who are in charge of different data elements? And how are they appointed? And how do we make sure that they’re well trained? And that they’re doing their jobs? And how are we governing data for regulatory purposes, for compliance purposes and audit? So data governance encompasses all these things, and the more mature businesses have got a very robust data governance function.


Will Bachman  18:15

Okay, so that, that finishes up data management, you walk me through data warehousing, that’s kind of where we’re temporarily storing the data, data sourcing and integration. How do we, where do we get data? And how do we automatically flow it in there data quality? Making sure the data is good, and data governance? How do we set rules around it


Adam Braff  18:35

correctly now? Well, I will take a brief pause here to do an interlude, let’s have a little interlude, which is to say that a lot of businesses that are going to be analyzed and diagnose this way are big and complex. So at some point, you can loop back and repeat analytic strategy and data management diagnostics for the different parts of the business, you’re gonna get different answers for different countries, different regions, different business units, different functions. In each of these cases, one of the things you’re looking at is how is data being used to answer the important questions for this region for this business. So it kind of it’s a little hard to plot on a diagnostic because it’s so specific to each particular business and each part of it. But if you if you take the example of a of a subscription, consumer subscription business, the natural deep dives you’d be doing at this point would be around things like well, customer acquisition, and you know, cross sell and retention and win back and customer satisfaction. So there would naturally be exploration of strategy and data as it applies to each of those questions. Right? So so this is just to say, like, I’m not forgetting all of the important business value drivers that are out there, but this is happening on a sort of different level, right? This is a diagnostic that is meant to be broadly applicable across a very wide range of businesses. outstrips,


Will Bachman  20:00

right? We’re not getting into, like, what analyses you should be doing. This is a bit more high level than that. And if it was a, you know, industrial pump manufacturer, you’d be less excited about customer acquisition and maybe more about tolerances and failure rates and stuff.


Adam Braff  20:17

Yeah, exactly. Exactly. Tons of operational analytics. In that case, it might be some, you know, wholesaler purchases, you know, analytics around revenue, but you yes, you’re going to be different types of analysis you’re doing. So this is more of the, how is the analytics function doing at solving whatever the problem is that important for that


Will Bachman  20:35

business? It’s like the infrastructure of the function. Yeah, yes, you got it. Alright, cool. interlude complete. Let’s go on to advanced analytics and AI. So


Adam Braff  20:48

here this is worth calling out is its own kind of discipline within the analytics function, because it’s generally going to be centralized within a company, like most companies that are not like digital natives and not born doing AI as their core core business function. To the extent they have an advanced analytics function, it’s going to be probably clustered together. And that’s great. And that’s as it should be. So they’re going to be doing things like predictive modeling, building models to predict what is the next best, you know, product to offer here, which customers are most likely to churn? What is the likelihood of someone making it all the way through this flow. So developing those models and validating them and, and deploying them and maintaining them is a whole discipline of its own, and you want to diagnose how mature that function is. You’re also going to have machine learning in many, but not all businesses. So you’re going to have some function that is automatically gonna have labeling and reviewing data and generating algorithms, and basically producing outcomes that are not necessarily hand driven in a deterministic way, by data scientists, it’s just sort of like setup to run right? In those cases, the diagnostic diagnostics going to be around, well, which algorithms are you using? How have you trained them up? How is the data prepared and processed? And how are we evaluating this model and making it better over time? Okay. And then finally, there’s gonna be a category around AI driven decision making this is both categorizing AI and the new thing as of the last year, which is generative AI? So how, if at all, is this company using artificial intelligence to do things like make those industrial heat pumps better or improve the efficiency of its operations? Even if it’s service operations in a call center? Is there is there a customer facing Chatbot? That’s AI driven? And how is that experience? And how is the engagement? And to what extent are we internally using AI for strategic insights and foresight? The demo that we did a few months ago on advanced data analysis, aka code interpreter would fall into this third category. Are we using generative AI to broaden and accelerate our analyses that we do? Good. So the remaining three categories at the highest level of analytics, talent, decision making process, and tools and infrastructure are the proverbial people process and technology, right. So this is where you get to the deeper level of like, nuts and bolts on how you make these things work. So we’ll dive into analytics talent now. Okay, that’s going to start with talent acquisition and retention. So are you finding the nerds that you need? Like, how are you recruiting them? Where are you going out to get them? Do you have, you know, an entry level job category to bring them in? Are you? Are you effectively developing them over time? Do you have retention programs? So how do you know what makes these people happy and sad? And are you building their skills over time? Are you deliberately rotating them across functions? If that’s possible to do? Are you giving them a way away to have a career path? And a big part of this also is performance evaluation, and incentive? So how rigorous is your performance evaluation process? Are you interviewing the internal clients of these analytics people to understand how good they are, where they can improve in terms of their technical skills and their client hands and their communications and everything else? So that’s talent acquisition and retention.


Will Bachman  24:37

Next, we have training and development. Yeah,


Adam Braff  24:38

so training and development. So here, I’m gonna focus a little bit more on the core team training and development as opposed to the broad one. So understanding where there are skill gaps in the team, if it’s around problem solving, if it’s around, if it’s around synthesis, if it’s around specific types of modeling and analytics, and then designing a curriculum that will fill those gaps So with continuous learning, and and then to the extent it’s helpful internal or external certifications and specializations that people can have, so that there’s some proof that they have gotten this training, but developing people over time is obviously very worthwhile compared to the cost of hiring more people from the outside who are going to be filling those gaps.


Will Bachman  25:21

Okay, so yeah, this is a good kind of talent management aspect of the whole function. How’s that doing? And then this last third piece of analytics talent you have is community engagement and collaboration. Tell me about that one.


Adam Braff  25:34

Yeah. So some of this is about about sharing the knowledge with the rest of the organization and building collaboration. A big part of this is those dashboards we talked about earlier. So if you’ve got a business intelligence initiative, and you’ve pushed out a bunch of dashboards, you want to make sure people know what they are, what the terms on them are defined as and how they can use these tools, and generally drive adoption and have them use consistently so that these things don’t go to waste. You also have a broader community that you’re engaging with external partnerships and networks, maybe you’ve got universities in your area, and you’ve got a network is kind of an ecosystem of vendors that you’re developing products with, and you’re developing analytical capabilities with. And then finally, innovation and co creation initiatives, hackathons, and other efforts to really spur people’s imagination, like a science fair, you know, where different people on the team are able to come up with their own ideas for analytics, it could be inside the analytics team or outside but finding ways to deliberately push the envelope on innovation.


Will Bachman  26:43

Okay, great. I love the wrap on talent management aspect, analytics talent.


Adam Braff  26:49

Excellent. That’s done. So. So then on to process decision making process. So a big part of this is having a data, I’m sorry, the let me give you the three buckets here, data driven culture, analytical decision making and predictive decision making. The data driven culture, I think you would recognize any business you’ve been in, as to whether it’s more intuitively driven, or more data driven. And this is not to say that every single decision must be made and can be made using data, often you’re doing something that’s novel, and you have to use intuition. And these things are used together. But there is an aspect of being a data driven culture that has things like controlled testing of experiments, and, and generally just wanting to measure things, and and to kind of not take for granted that you know, what every answer is always using intuition. So here, it starts with tracking the demand for analytics use cases and how the team is being used, like think of it as like an internal CRM system, for how your requests coming in, and the ad hoc analytics team tackling these things, and making sure that the team is being is being utilized, then you’ve got that you’re pushing this stuff out into the business, is there cultural change happening? So are people sticking with the old metrics? When the data governance people have said, no, no, we’ve agreed that we’re going to use this new metric stop using the old metric, you’re confusing the CEO, right? So cultural change management. And then finally, within culture, there’s, there’s a piece around data accessibility, and democratization. So can you know, are things well labeled, let’s say, right, or is it possible for people in the business who want to use these dashboards and these data warehouses and these tools to get support for either self service or from somebody who can help them? So that’s the data driven culture.


Will Bachman  28:41

Okay. Let’s talk about analytical decision making next piece.


Adam Braff  28:47

Okay, so, analytical decision making is going to start with analytical frameworks and tools. So a big one might be a customer lifetime value framework where you’re looking at all of the different value drivers across the customer lifecycle, and having people kind of stack hands on that and agree that that’s the framework that’s being used, and you know, a CLV calculation that we’re going to use in all aspects of the business to make decisions. Then we’ve got decision making process integration. So making sure that there are checks along the way before some recurring function happens in the business to make sure that analyses are involved. So for example, when I was working at a pay TV company, we were raising prices every year, we made sure we held out a control group so that we can see what the effect was of the price increase on customer attrition, and that was a built in check that we had inside of the process of changing prices there. And then finally, performance tracking and feedback. So we need to make sure that when individual decisions are made that are augmented by data, we put in a new model for our door to door sales team, right. And we compare it to the, you know, the prior model that worked Tracking the results of each of these things individually, so that we can roll them up later into the broader performance measurement of the overall function. So that’s analytical decision making. Yeah,


Will Bachman  30:10

so I guess, part of this one is, well as the previous subcategory data driven culture, it’s, you probably want to do almost an inventory of the types of decisions that are getting made at the company and see which ones are based on data versus intuition. You don’t want to just say, Okay, where is there pull for data and only look at those, but you want to say, what are all the decision types that are being made? Where are we using data? Where are we not using data? Right? Yeah, exactly. Built in there. Okay.


Adam Braff  30:40

Exactly. And you in some sense, you’d want to start even one level up from that, which is which parts of the business are functioning well, and which ones aren’t like, which are our KPIs are we succeeding on right? And you would presumably dive deeper on the ones that aren’t succeeding and discover that those are the data poor ones, but what you might discover actually, is that there’s some push and pull between these different metrics. And actually, what it looks like a customer retention problem is really a customer acquisition problem. And to your point, well, it was the failure to analyze customers on the way in that caused us to acquire customers were churning too much and leaving us. So it is it is fair game to do what you just said, which is to analyze how rigorously data is involved in all the decisions that are made the important decisions that are made in the business. All right. Then finally, within decision making processes predicted decision making. So we’ve done the descriptive stuff, and now we’re doing the predictive stuff. So this is going to be things like we’re planning out our budgets for next year, how rigorous are the predictive analytics around that? Do we know all the forces at work that are going to cause customer numbers to go up or down? And do we understand the macroeconomic impacts and the weather? And? And is this all tied back into our operational and our financial budgets? And do we have data that we can use for operational decisions? So can we predict demand on a week to week or day to day basis? And finally, do we have the data that we need for risk assessment and management? So this is a big part of what an analytics team might do. And like a risk team within a financial services firm, we need to be able to manage a wide range of of risks, you know, different, you know, different factors that will drive performance and say factors in the kind of the financial sense. And therefore, are we using analytics to the extent possible to manage that risk?


Will Bachman  32:32

Okay, so predictive would be this big kind of strategic planning questions, what sales is going to be or what are our competitors going to do? And then there’s more operational real time things and the risk assessment piece. Okay.


Adam Braff  32:47

Yeah, yeah, exactly. They think of the first ones is like, the average score that you’re predicting to happen for kind of revenue for operations. But then the third one is, what’s the variability around that? So we’re ready for anything?


Will Bachman  32:58

Okay, great. And then we’re in the last big category, tools and infrastructure. Yeah,


Adam Braff  33:04

so the analytics team has to have good tools in order to do their jobs. And this has changed obviously, over the last, you know, 1020 years, I think, I would have said, like, you know, 20 years ago, a lot of firms were just using Excel, or they’re just using SAS. And then over time, a lot of businesses have really been developing their own analytical software. And they’re writing their own scripts in Python. So you’ve got a wide range of choices here. And so within tools and infrastructure, you’ve got analytics platforms, infrastructure, scalability and tool, that integration, tool integration. The analytics platforms are really around making good choices on which platforms are going to use. So the mature state isn’t necessarily any particular named platform, or that you have to do it all in the cloud, right? But it’s, it’s, there’s a reason and a justification for the platforms that people use to analyze data and to visualize data, and that there’s training and support for those things. And we’re monitoring the performance of those of those platforms. So this could just be it could be Excel on everyone’s laptop. But odds are, if it’s anything, it’s happening at scale, you’re gonna have some more centralized layers of the infrastructure underneath that. Okay. Then we get to infrastructure scalability, so capacity planning and management, do we have all the room in this data warehouse to do all of the analysis we want to do as it grows over time? And that would include cloud based solutions? And then do we have this that disaster recovery and business continuity so that if something if disaster strikes, do we have a backup for this data warehouse? And then finally, tool integration? So all of these analyses are creating numbers and answers and graphs. And ideally, you want to be able to move them from one platform to another. So to take the investment firm example, you may have a central team that’s doing analytics on alternative data debts that are, you know, credit card data and Clickstream data that are telling you something about the key performance indicators of the businesses that this buyside firm is covering. But then the people out in the business who are to say that portfolio managers and their analysts, they’re using some other set of tools to build their own models of companies and understand, you know, how to how to, you know, put on their orders, and that you have to be able to move the information from one thing to the other, right? So is that going to be automated, or is it going to be hand carried, the more mature business is going to have a way of integrating these things with API’s, and making sure it fits the needs of all the different kinds of portfolio managers out there. And then finally, you need to have the ability to develop custom tools, whether that’s people in the central analytics team or people out, you know, distributed in the business, they have to be able to develop their own custom tools and solutions, it can be as simple as a custom dashboard. Or it could be, you know, you know, some some alert that goes off that tells them when they need to make a decision. But you want to have some flexibility built into that consistent with all of the data governance and the constraints that you put around the system earlier.


Will Bachman  36:12

Okay. So this is amazing. And for those of you listening to this, if you’re listening to this episode, you can go to the show notes and download the Analects diagnostic guide, we’ll have it prepared and ready to go. So, Adam, talk to me a bit about how someone might use the insights from this. So let’s say you’ve gone through the checklists, and you’ve done a diagnostic for your organization, how should someone think about prioritizing designing their future state figuring out what they want to invest in? Upgrading or, or moving along the maturity? process?


Adam Braff  36:56

Well, it kind of takes you back up to the top of the page, which is about strategy. So once you do all this diagnostic, first of all, if you’re if you’re doing this as a consultant, and then you show it to your sponsor, and you say, Look, this is where you guys are on the on the maturity model. Part of it is is going to be it’s going to be intuitive to both you and your client. Right that that Okay, wow, we’re we’re actually, we’re actually, you know, pretty good in terms of like, our data is in pretty good shape. But like, we actually don’t have the people that we need to go after it. Like we have this sort of ad hoc group of people that have just been tagged, like in an Expansion Draft, they just got pulled into this analytics team. And that’s, that’s, that’s weird and suboptimal, like so some of the answers might be kind of intuitive. Coming out of this, right? More likely, what’s going to happen is, if it’s any, if it’s a company of any size, you’re going to want to take a clean sheet of paper and say, Well, what are the bigger picture strategic choices that we need to make here? Like, are we actually a data driven company? Or is that really not what we are, as a company, it’s not that important to be spending a lot of time and effort on data and analytics relative to other things that we could be doing. Like just, you know, just blasting out new features, or being really creative or doing something else. Odds are, I’m talking my book here, but odds are, you will realize that there’s something that you want to do more with data and analytics, if only because people in the business when you go around and talk to them and do the interviews that are necessary to complete the diagnostics that we just talked through. Inevitably, one of the questions in those interviews is going to be, hey, are there things that you, you know, head of ops, Head of Marketing, that you want to know about your business that you don’t know, today? And that’s keeping you from being able to do the best job possible? And like nine times out of 10? They’re gonna say yes, and they’re going to tell you what those things are. Right? So to the extent that you have collected a bunch of these, some more specific examples of demand in the business, that’s going to give you a feeling for whether you want to be a more analytical firm or not. Right. So this so where we were like, before that little digression was like, how do you go put this into into practice? Well, if you then pick a point along the continuum of very, kind of low maturity, kind of ad hoc, intuitive on one end of an extreme and superduper, data driven and algorithmic on the other extreme, you’re somewhere in the middle there, right? And you want to kind of edge probably more to the right, right, you want to try to get more toward the more sophisticate. Right, then there are going to be other strategic choices. Like, do we want to keep this thing more centralized? Or do we want it to be more distributed? And by this thing, I mean, the analytics function, right? Do we want to be distinguished in our ability to harness our own data? Or do we want to be more external facing and just be more aggressive in gobbling up external data, or vice versa, like monetizing our data by selling it to third parties, right? Do we want to be extremely buttoned down? and and you know, very cautious in terms of reputational risk and how we’re using data. Like if you know, we don’t want a single thing to be done without explicit customer consent? Or do we want to be a little more freewheeling on that one, right? And it sounds like I’m sort of telling you the answer on that. But you know, genuinely companies occupied different points along that continuum. So lots of different strategic choices. From that what follows out is, okay, this is this is how aggressively we want to build these capabilities, right, we want to get better, by next year, by two years out by three years out, odds are, you’re going to have something like eight quarters, or 12 quarters in which you want to gradually get better at these things, starting with some kind of quick wins on use cases that really matter. And building over time on the things that take a little more time to build out, like your, you know, you know, like your marketing ROI models or something. Yeah. So that’s the, you’re kind of looping back to the strategy part saying, Okay, let’s make the roadmap, let’s get aligned on our strategy. Let’s make the roadmap, let’s plot out what we want to do and people processes and technology and specific use cases over the next eight quarters and do that in a logical order.


Will Bachman  41:13

What I like about what you’ve laid out here, it’s somewhat meta, but the idea of the data analytics is you’re making things visible instead of using your intuition. And this, this, this really detailed diagnostic that you’ve come up with, would allow someone to take a complete, holistic look at the function and say, oh, yeah, I guess there are some areas where we could invest, and it just helps make it visible, right, as opposed to intuition.


Adam Braff  41:41

Yeah, exactly. Like any framework, like the millions of frameworks that you have helpfully kind of deployed, like it’s a check for completeness. And, you know, there may be things on this list that escape your notice, as you’re building up this analytics function, like you’re super duper focused on AI and having the, you know, making the choice between chat GVT, and Claude or something. And what you’ve forgotten is, oh, wait, we forgot to build a data driven culture, like no one, no one in the business is adopting any of this because they don’t care, right, or we forgot to measure how much lift this initiative is having. Or we forgot to make sure that the data that we need to feed into that beast is actually being stacked up somewhere and ready to go. And it’s high quality, right? So that’s really, that’s really the goal of this diagnostic is like you really want to cover off all of these different dimensions. And in order to have a complete solution.


Will Bachman  42:30

Now, I have a question that someone might raise about this is, you don’t really talk in here about data security? And is that because that would be really you think of that as a separate function in the business more of the, you know, information security officer talk to me about that piece of it?


Adam Braff  42:46

Yeah, I didn’t want to get too deep into the kind of the fractal layers of detail around InfoSec, and kind of data security. And it’s kind of tucked in here under Data governance, and kind of, you know, compliance and all that stuff. But But yeah, I mean, you definitely have to worry about that. I want to distinguish also, between the business analytics function and the IT function, right, the Chief Information Officer, it’s generally quite rare. When you are asking me where this function attaches. It doesn’t usually report up into it. In some sense, it is like the business partner who is enabling these things. So all of this stuff around the category of data management, and tools and infrastructure are like the requirements that this team would set out that some IT team would then go implement on their behalf. So inside of that IT team is going to be a seaso, who’s in charge of, of information security, as well. But yeah, this is more of the what is the business really needed? And how do they know the language of what to ask for so they can do their job better?


Will Bachman  43:48

Excellent. And Adam, for listeners that want to find out more about your practice, where would you like to point them online?


Adam Braff  43:57

They should go to braff.co. That’s B R A, F f.co. And there they will find a whole bunch of other tools and resources. There’s a blog that’s about food and analytics, there is a forecasting Contest, which I think by the time this drops, it’ll probably be underway. It will. But there’s, yeah, there’s an annual forecasting contest. There is just a whole bunch of stuff to go look at. And there is a programming course.


Will Bachman  44:23

Right, like you have a let’s bring that one up. I mentioned that. Yeah,


Adam Braff  44:27

the analytics intensive is what I was about to say. So there is a course that I teach. So I teach this content in graduate programs at Brown and NYU. And by popular demand, I’ve begun teaching a corporate version of that exact same material. And and I do that in something called the analytics intensive, so you can find that on the website as well. Okay,


Will Bachman  44:49

we will include those links. And we’ll also include the link to download the analytics diagnostic guide in the show notes as well. Adam, thank you for putting this together. This is incorrect. ratable And I really appreciate you sharing this with us


Adam Braff  45:03

well thanks for the time I appreciate your curiosity about this and it’s stuff that I love to do so I hope everyone else gets a kick out of doing it too excellent and

Related Episodes


Building a World-class Professional Services Firm

Russell S. Reynolds, Jr.


AI Project Case Study

Paul Gaspar


AI Project Case Study

Astrid Malval-Beharry


AI Project Case Study

Julie Noonan