The evolution from SQL-based query engines for big data to data lake engines including its impact on data warehouses and data lakes

Image for post
Image for post
Photo by NASA on Unsplash

Introduction

From a high-level perspective, most data and analytics solutions have been built up the same way for many years now. In a nutshell, consisting of diverse integration processes to load all data into one central place serving as the single source of the truth for upcoming data modeling and analytical use-cases. While in the older days those central places were mostly expensive and inflexible tightly coupled hardware/software-systems, nowadays there is usually taken advantage of the cloud and distributed architectures including the separation of compute and storage. …


Two Custom Time Intelligence Tricks to Improve the Usability of Your Report

Introduction

The time dimension is the only dimension which is essentially part of every data warehouse or dimensional data model. Without having this time relation most performance indicators would be meaningless. In this article, I will demonstrate two simple but effective ways to make time selections in Power BI reports more flexible and user-friendly. The first one is about setting a default selection using a normal slicer element as opposed to using the rather static relative date filter option. And the second one is about implementing a slider to move back and forth in time as you please or even a combination of both. This is especially useful for the purpose of rolling forecasts or business reviews as it allows an extended view of the expected performance. …


Introduction to Webooks, an event-driven alternative to Polling

Introduction

For years, we have been experiencing the trend towards service-oriented architectures for creating adaptive applications based on self-contained services, often implemented as microservices nowadays. The communication between those independent services happens through a wide range of different API technologies using some sort of lightweight communication protocols. This overall architectural design approach is a fundamental part of today’s cloud and serverless computing where servers including databases tend to completely disappear. Of course, they are still used, but the idea is that developers do not need to be aware of them. …


Introduction

There basically coexist two ways of visualizing data. The first one is ready-to-use visualization tools and the second one are purpose-built programming libraries for data visualization. Both approaches have continuously been developed further over time, either from their corresponding software vendors or open-source communities. At this point, it has to be mentioned that most visualization tools actually rely on visualization libraries for drawing charts and diagrams generically. …


It has been strange telling companies and users to use Google Chrome for the best user experience in Microsoft’s Power BI online service. Standard and custom visuals work as expected after publishing out of Power BI Desktop. Performance gains in terms of filtering and rendering reports are significant by browsing reports in Google Chrome compared to other browsers. It seems to be the only browser that prints/exports all components of the report correctly.

Chromium and CefSharp

The reason for this, is the Chrome’s underlying source code and rendering engine Chromium which is the open-source browser project behind Google Chrome.

The desktop application “Power BI Desktop” also relies on this project for rendering visuals locally by using CefSharp. This is a wrapper that allows you to embed web browsers into your .NET application. If you open up your task manager after starting up Power BI Desktop, you will find several instances of a process named CefSharp.BrowserSubprocess. They can sometimes be even quite memory consuming. …


Recreating Excel table in Power BI having applied conditional formatting on a row-level

The Conditional Formatting capabilities of Power BI allow you to apply either rule based or dynamic formatting on different kind of visuals. You can change colors, add icons or data bars together with the ability to control various color options such as color gradients and lower and upper thresholds. Though, in spite of these advanced formatting capabilities, there are still cases where you might reach the limit of what is possible out-of-the-box. For instance, this is very likely to arise by shifting Reporting from Excel to Power BI.

Requirement

The requirement in my particular case was to apply dynamic color scaling beyond the values of a single column, but the values of different columns in the corresponding row. The following Excel table needed to be transitioned to Power…

About

Patrick Pichler

Passionate Data and Analytics Engineer

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store