The real meaty questions about the business such as: “what contributed to the sudden growth in revenues last month?” cannot be answered by a generic dashboard with revenue and traffic numbers. You need to do a deep dive into the problem. You need to unravel the question into specific queries.
The conclusion may be that there was a seasonal effect (people buy more during certain times of the year), there was a successful marketing campaign that boosted traffic into the product, or a cohort of users materially affected the revenues. Whatever it is, there is a deep, explorative nature to finding out the answer to a seemingly simple question. There is also a second facet to this workflow: organizing and sharing these findings. I call this entire process “Deep Analytics” (as opposed to “shallow analytics” that is more focused on building generic metrics and sharing insights via dashboards — a more blunt, if not more all-purpose, tool).
The problem is that the broader business community does not put weight into these deep workflows, even though this is precisely the type of work that drives the most amount of business outcomes. The really great, idiosyncratic insights don’t materialize from staring at dashboards. They come from looking at the business and its data in novel, non-obvious ways. And this by definition cannot be done in a one-size-fits-all fashion.
The truth is that even the analytics community does not put weight into these deep workflows. Lack of care result in lack of standards and tools. A clear example: the SQL IDE has not been redesigned in decades. Sharing and organizing analytical queries is an afterthought. There are no standards around discovering and choosing the different versions of analytical data to be used for analysis.
Great tools for a workflow don’t just make the workflow more efficient. They define the workflow. Github defined version control and collaboration for code. Figma defined collaborative interface design. Ableton Live defined electronic music production. We need a tool that defines Deep Analytics.
Traditional BI tools just don’t cut it. They are built to help the analyst to build self-service data tools (e.g. dashboards) for the general business audience. SQL IDEs don’t cut it either — they were built for transactions, not for explorations (too many tabs!). Jupyter notebooks are great for data science workflows in Python, but not for analytics. Data Discovery tools are great for general observability, but are not integrated into the query-writing workflow. Wiki tools like Notion and Confluence are great general-purpose places to share information, but are not purpose built for sharing analytics insights. It’s a disjoint workflow with tooling that is not focused on getting the job done.
By giving a workflow a name, we bring life to a concept. We can refer to a nuanced concept in a concise way. We have a name for Data Science as a discipline. We have a name for Deep Learning as a transformative class of predictive models. We invented the term Business Intelligence to signify the importance of data-driven decision-making through self-service tools. More recently, we invented an entirely new profession: the Analytics Engineer. We need to give attention to the category of Deep Analytics. And we need tools to help us be more effective at it.
Joseph is the Founder and CEO at Hyperquery, a company that is building a platform for Deep Analytics. Reach out to Joseph at joseph@hyperquery.ai and on Twitter @josephmoon_ai.
