Why the Common Data Model is about to be the keystone of modern data insights architectures on Azure and how to use it with Power BI dataflows together with Azure data services

Common Data Model For Analytics (Image by Microsoft)

Introduction

To begin with, this article isn’t about comparing data warehouses, data lakes, data lakehouses, data oceans, data meshes or whatever approach or naming may exist, it is about the evolution of the Common Data Model (CDM) and why it is about to be the keystone of modern data insights architectures on Azure by filling in some very important missing gaps¹. For the overall understanding it is, however, necessary to mention that many of today’s data warehousing solutions still integrate relational databases. A very common pattern over the last years have been hybrid architectures or so-called Modern Data Warehouses leveraging distributed…


Brief introduction to Databricks including its deployment options and the different ways of integrating data into Delta Lake

Delta Lake at Scale on Azure

Introduction

Before Databricks, Apache Spark quickly replaced Hadoop’s MapReduce programming model in being the number one processing technique when it comes to distributed computing and processing huge amounts of data in real-time. It’s optimized for speed and computational efficiency by storing most of the data in-memory and not on disk, compared to the MapReduce approach which is rather meant for long running batch jobs. Databricks nowadays isn’t just Apache Spark anymore, but it’s a fully managed end-to-end data analytics platform on the cloud with collaboration, analytics, security and machine learning features, data lake, and data integration capabilities. That means, apart from…


The evolution from SQL-based query engines for big data to data lake engines including its impact on data warehouses and data lakes

Photo by NASA on Unsplash

Introduction

From a high-level perspective, most data and analytics solutions have been built up the same way for many years now. In a nutshell, consisting of diverse integration processes to load all data into one central place serving as the single source of the truth for upcoming data modeling and analytical use-cases. While in the older days those central places were mostly expensive and inflexible tightly coupled hardware/software-systems, nowadays there is usually taken advantage of the cloud and distributed architectures including the separation of compute and storage. …


Two Custom Time Intelligence Tricks to Improve the Usability of Your Report

Photo by Markus Winkler on Unsplash

Introduction

The time dimension is the only dimension which is essentially part of every data warehouse or dimensional data model. Without having this time relation most performance indicators would be meaningless. In this article, I will demonstrate two simple but effective ways to make time selections in Power BI reports more flexible and user-friendly. The first one is about setting a default selection using a normal slicer element as opposed to using the rather static relative date filter option. And the second one is about implementing a slider to move back and forth in time as you please or even a…


Introduction to Webooks, an event-driven alternative to Polling

Photo by Hello I'm Nik on Unsplash

Introduction

For years, we have been experiencing the trend towards service-oriented architectures for creating adaptive applications based on self-contained services, often implemented as microservices nowadays. The communication between those independent services happens through a wide range of different API technologies using some sort of lightweight communication protocols. This overall architectural design approach is a fundamental part of today’s cloud and serverless computing where servers including databases tend to completely disappear. Of course, they are still used, but the idea is that developers do not need to be aware of them. …


When it still makes sense to directly rely on visualization libraries rather then using data visualization tools

Photo by Lukas Blazek on Unsplash

Introduction

There basically coexist two ways of visualizing data. The first one is ready-to-use visualization tools and the second one are purpose-built programming libraries for data visualization. Both approaches have continuously been developed further over time, either from their corresponding software vendors or open-source communities. At this point, it has to be mentioned that most visualization tools actually rely on visualization libraries for drawing charts and diagrams generically. …


Why you finally can navigate properly through the Power BI service using a Microsoft developed browser

Photo by Microsoft, edited by Author

Introduction

It has been strange telling companies and users to use Google Chrome for the best user experience in Microsoft’s Power BI online service. Standard and custom visuals work as expected after publishing out of Power BI Desktop. Performance gains in terms of filtering and rendering reports are significant by browsing reports in Google Chrome compared to other browsers. It seems to be the only browser that prints/exports all components of the report correctly.

Chromium and CefSharp

The reason for this, is the Chrome’s underlying source code and rendering engine Chromium which is the open-source browser project behind Google Chrome.

The desktop application “Power…


Recreating Excel table in Power BI having applied conditional formatting on a row-level

Conditional Formatting by Rows Power BI

Introduction

The Conditional Formatting capabilities of Power BI allow you to apply either rule based or dynamic formatting on different kind of visuals. You can change colors, add icons or data bars together with the ability to control various color options such as color gradients and lower and upper thresholds. Though, in spite of these advanced formatting capabilities, there are still cases where you might reach the limit of what is possible out-of-the-box. For instance, this is very likely to arise by shifting Reporting from Excel to Power BI.

Requirement

The requirement in my particular case was to apply dynamic color scaling…

Patrick Pichler

Passionate Data and Analytics Engineer

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store