Hares win races.
Their grit and determination propels them move quickly with agility. Oftentimes that’s not how business intelligence (BI) solutions are stood up. It’s really a common tale: months of conferring in a series of meetings between multiple layers of an organization - a proposal is submitted, mulled over, and questioned. Indecision and stagnation are rampant! Why does it take so long to setup a BI solution to actually analyze your data and propel your company into this century? In this brief guide, we’ll discuss the key features of standing up your BI Tech Stack including some common, not-so-common knowledge about the process, and potential pitfalls that may turn your company into a tortoise.
Why do you need a BI solution again? It can be as simple and intangible as ‘to think in a more data-driven way’ or as specific as ‘to align marketing, sales, and finance to the same goals’. The truth is, there are many reasons that to dive into BI, but the best ones take aim at solving a specific problem or gaining a new level of capabilities. Excel is creating silos? Get a BI solution. Too much manual effort to create reports? Get a BI solution. Data is saved in too many different places? Get a BI solution!
And who is going to run this thing? I can’t tell you how many times a BI project (even small ones) fail because of a lack of executive sponsorship…which is really just ownership. Commitment not only goes with the budget but also paves the way for greater cooperation and coordination between end users, developers, and managers. Typically the CIO or CFO owns an enterprise-wide solution. However, not every company wants commit all at once. In which case, CMO, CRO, or any other Director-level should do. The key is, take ownership of the solution to the problems mentioned above.
Typically there are a three layers in a solution, but you can always get much more granular and sophisticated.
The ‘raw’ layer is the data coming from your source applications (marketing operations platform, customer relation management (CRM), etc.). In addition to this is the ‘extract’ and ‘load’ process of getting said data into a data lake or data warehouse. Think of this as a big ‘copy and paste’ of your data into an accessible place. Common source systems include Salesforce, Hubspot, Marketo, Amplitude, Netsuite, Shopify, and literally thousands others. Extracting and loading data is its own task which can require a specialized coder or a self-service option like Fivetran, AWS Glue, and Informatica.
From there, you have to package the data according to your projects. Not ALL of the data is needed for EVERY project. This can save you time and money through efficient development times and lower compute resources. This ‘curated’ data layer is oftentimes referred to as ‘data modeling’. Common data lakes and warehouses are Snowflake, AWS - Redshift, Azure, and many others.
Once the ‘data modeling’ has been complete, you need a software to make the data readable and accessible to your audience. Tableau, Power BI, and Looker are common enterprise-wide solutions for this. If you have a 365 account, Power BI is awfully tempting to simply tack onto your current subscription.
Every solution costs something. Usually it depends on the number of licenses you need, amount of computation resources, and data storage (we’ll touch on employee costs later). Each software in the above curation and visualization layers have a cost. Most solutions in the curation layer are ‘pay for what you use’ and extract and load processes are less than $1,000 / month unless you’re doing some heavy data lifting. Visualization costs can be $8 - 99 per user. Data lakes are a bit trickier to price out without a specific use case as their costs fluctuate wildly.
This is where we have to remind you - don’t become paralyzed by analyzing the costs of things. Spending $15,000 in labor costs to evaluate a solution that will cost you $2,000 more per year is the definition of a waste of time.
So who is going to develop or manage all of this? There are many labor structures you can use, but we’ll assume that you are taking an enterprise-wide solution for this section:
Data Analysts (DA) - data modeling and data visualization work, adding business context to the analysis.
Data Engineers (DE) - data movers, wranglers, and managers of ELT procedures (optional at first)
Database Administrator (DBA) - the true permissioning and setup arm of the different systems in the BI solution
Data Scientist (DS) - whoaa! don’t get ahead of yourself. These mathematicians build algorithmic models that predict the future (not needed to standup)
The truth is, you won’t know all of the people you need until you software signed up. In sequential order, you’ll need the DBA, DA, and DE to get going.
The last skillset you need is someone to guide and direct these individuals. Project Managers (PM) are key to communicating with stakeholders, setting priorities, and aligning the development efforts with initiatives that you’re team is promising to complete on time! We recommend an Agile methodology and a simple google sheet with a weekly standup cadence for projects.
If this seems overwhelming, then you might be at risk of running more slowly than you could when you stand up a BI solution. Our team delights in helping clients you hit the ground running - to break up data silos, align your sales and marketing teams, or become more data driven!