Microsoft Fabric: Capacity Cost Management Part 3, Pause Capacity During Christmas with Azure Logic Apps

Microsoft Fabric: Capacity Cost Management Part 3, Pause Capacity During Christmas with Azure Logic Apps

In the first blog post of this series, I explained that we can Pause and Resume a Microsoft Fabric capacity from Azure Portal. In the second blog and its accompanying YouTube video, I showed you how to automate the Pause and Resume actions in Azure LogicApps so the capacity starts at 8:00 AM and stops at 4:00 PM. While I have already mentioned in those posts, it is worthwhile to mention again that these methods only make sense for PAYG (Pay-As-You-Go) capacities and NOT the Reservation capacities. While the method works fine, you may need more fine-tuning.

Managing operational costs becomes crucial for businesses leveraging Microsoft Fabric capacities when the holiday season approaches. This presents a unique challenge of maintaining efficiency while reducing unnecessary expenses, especially during Christmas when business operations might slow down or pause entirely.

In this post and video, I will extend the discussions from my previous blog and demonstrate how to optimise your Azure Logic Apps to manage Microsoft Fabric capacity during the Christmas holidays.

Extending the Logic Apps Workflow

Existing Setup Recap

In earlier discussions, we’ve explored using Azure Logic Apps to manage Fabric capacity effectively from 8:00 AM to 4:00 PM on regular business days and pausing operations afterwards. This setup ensures that we’re not incurring costs when the capacity isn’t needed, particularly from 4:00 PM to 8:00 AM the next morning, and throughout the weekends. I encourage you to check out my previous post for more information. This is how the existing solution looks like in Azure LogicApps:

Automating Microsoft Fabric Capacity with Azure LogicApps
Automating Microsoft Fabric Capacity with Azure LogicApps

Incorporating Holiday Schedules

The key to extending this setup for the Christmas period lies in integrating specific holiday schedules into your existing workflows using Workflow Definition Language which is used in Azure Logic Apps and Microsoft Flow. The following expression determines if the current date (in New Zealand Standard Time) falls within the period from December 25th of the current year to January 2nd of the next year:

and(
    greaterOrEquals(
        int(
            formatDateTime(
                convertFromUtc(
                    utcNow(), 
                    'New Zealand Standard Time'
                ), 
                'yyyyMMdd'
            )
        ), 
        int(
   concat(
    formatDateTime(
     utcNow()
     , 'yyyy'
     )
    , '1225'
    )
   ) 
    ), 
    lessOrEquals(
        int(
            formatDateTime(
                convertFromUtc(
                    utcNow(), 
                    'New Zealand Standard Time'
                ), 
                'yyyyMMdd'
            )
        ), 
        int(
   concat(
    add(
     int(
      formatDateTime(
       utcNow()
       , 'yyyy'
       )
      )
     ,1
     )
    , '0102'
    )
   )
  )
)

The following section explains how the expression works.

Continue reading “Microsoft Fabric: Capacity Cost Management Part 3, Pause Capacity During Christmas with Azure Logic Apps”

Microsoft Fabric: Capacity Cost Management Part 2, Automate Pause/Resume Capacity with Azure Logic Apps

Automate Pause Resume Suspend Fabric Capacity with Azure Logic Apps

In the previous blog post, I explained Microsoft Fabric capacities, shedding light on diverse capacity options and how they influence data projects. We delved into Capacity Units (CUs), pricing nuances, and practical cost control methods, including manually scaling and pausing Fabric capacity. Now, we’re taking the next step in our Microsoft Fabric journey by exploring the possibility of automating the pause and resume process. In this blog post, we’ll unlock the secrets to seamlessly managing your Fabric Capacity with automation that helps us save time and resources while optimising the usage of data and analytics workloads.

Right off the bat, this is a rather long blog, so I added a bonus section at the end for those who are reading from the beginning to the end. With that, let’s dive in!

The Problem

As we have learned in the previous blog post, one way to manage our Fabric capacity costs is to pause the capacity while not in use and resume it again when needed. While this can help with cost management, as it is a manual process, it is prone to human error, which makes it impractical in the long run.

The Solution

A more practical solution is to automate a daily process to pause and resume our Fabric capacity automatically. This can be done by running Azure Management APIs. Depending on our expertise, there are several ways to achieve the goal, such as running APIs on running the APIs via PowerShell (scheduling the runs separately), running the APIs via CloudShell, creating a flow in Power Automate, or creating the workflow in Azure Logic Apps. I prefer the latter, so this blog post explains the method.

I also explain the same scenario on my YouTube channel. Here is the video:

Automating Pause and Resume Fabric Capacity with Azure Logic Apps

Here is the scenario: we are going to create an Azure Logic Apps workflow that automatically does the following:

  • Check the time of the day
  • If it is between 8 am to 4 pm:
  • Check the status of the Fabric capacity
  • If the capacity is paused, then resume it, otherwise do nothing
  • If it is after 4 pm and before 8 am:
  • Check the status of the Fabric capacity
  • If the capacity is resumed, then pause it, otherwise do nothing

Follow these steps to implement the scenario in Azure Logic Apps:

  1. Login to Azure Portal and search for “Logic App
  2. Click the Logic App service
Finding Logic Apps on Azure Portal

This navigates us to the Logic App service. If you currently have existing Logic Apps workflows, they will appear here.

Continue reading “Microsoft Fabric: Capacity Cost Management Part 2, Automate Pause/Resume Capacity with Azure Logic Apps”

Microsoft Fabric: Capacity Options and Cost Management, Part 1; The Basics

Microsoft Fabric: Capacity Options and Cost Management, Part 1

Microsoft Fabric is a SaaS platform that allows users to get, create, share, and visualise data using a wide set of tools. It provides a unified solution for all our data and analytics workloads, from data ingestion and transformation to data engineering, data science, data warehouse, real-time analytics, and data visualisation. In a previous blog post, I explained the basics of the Microsoft Fabric data platform. In a separate blog post, I explained some Microsoft Fabric terminologies and personas where I explained what Tenant and Capacities are.

In this blog post, we will explore the different types of Fabric capacities, how they affect the performance and cost of our Fabric projects, and how you can control the capacity costs by pausing the capacity in Azure when it is not in use.

Fabric capacity types

Fabric capacities are the compute resources that power all the experiences in Fabric. They are available in different sizes and prices, depending on our needs and budget. We can currently obtain Fabric capacities in one of the following options:

If we want to purchase Microsoft Fabric capacities on Azure, they come in SKUs (Stock Keeping Units) sized from F2 – F2048, representing 2 – 2048 CU (Capacity Units). A CU is a unit of measure representing the resource power available for a Fabric capacity. The higher the CU, the more resources we get on our Fabric projects. For example, an F8 capacity has 8 CUs, which means it is four times more powerful than an F2 capacity, which has 2 CUs.

When purchasing Azure SKUs with a pay-as-you-go subscription, we are billed for compute power (which is the size of the capacity we choose) and for OneLake storage, which is charged for the data stored in OneLake per gigabyte per month (approximately $0.043 (New Zealand Dollar) per GB). OneLake is the unified storage layer for all the Fabric workloads. It allows users to store and access our data in a secure, scalable and cost-effective way.

Azure Fabric capacities are priced uniquely across regions. The pay-as-you-go pricing for a Fabric capacity at Australia East region is $0.3605 (NZD) per CU per hour, which translates to a monthly price of $526.217 (NZD) for an F2 ($0.3605 * 2 * 730 hours).

Microsoft Fabric pricing overview
Microsoft Fabric pricing overview

It is important to note that billing is per second with a one-minute minimum. Therefore, we will be billed for when the capacity is not in use. Here is a full list of prices available at the Azure portal by selecting our Fabric capacity region.

Now that we have an indication of the costs of owning Microsoft Fabric capacities let’s explore the methods to control the cost.

Nuances of Fabric’s Cost of Ownership

It is important to note that all the math we have gone through in the previous section is just about the capacity itself. But are there any other costs that may apply? The answer is it depends. If we obtain any SKUs lower than F64, we must buy Power BI Pro licenses per user on top of the capacity costs. For the tiers above F64, we get unlimited free users but, BUT, we still have to purchase Power BI Pro licenses for all developers on top of the cost of the capacity itself.

Another gotcha is that the Fabric experiences are unavailable to either Power BI Premium (PPU) users or the Power BI Embedded capacities. Just be mindful of that.

The good news for organisations owning Power BI Premium capacities is that you do not need to do anything to leverage Fabric capabilities. As a matter of fact, you already own a Fabric capacity, you just need to enable it on your tenant.

Continue reading “Microsoft Fabric: Capacity Options and Cost Management, Part 1; The Basics”

Power BI 101, What is Power BI

Many people talk about Power BI, its benefits and common challenges, and many more want to learn Power BI, which is excellent indeed. But there are many misconceptions and misunderstandings amongst the people who think they know Power BI. In my opinion, it is a significant risk in using tools without knowing them, and using the technology is no different. The situation is even worse when people who must know the technology well don’t know it, but they think they do. These people are potential risks to the businesses that want to adopt Power BI as their primary analytical solution across the organisation. As a part of my day-to-day job, I communicate with many people interacting with Power BI. Amongst many knowledgeable users are some of those who confuse things pretty frequently, which indicates a lack of understanding of the basic concepts.
So I decided to write a series of Power BI 101 to explain the basics of the technology that we all love in simple language. Regardless of your usage of Power BI, I endeavour to help you know what to expect from Power BI. This is the first part of this series.

What is Power BI?

I do not frequently get the “What is Power BI” question from my customers, my website’s comments, or my students within the training courses. It is indeed a question that I often ask people. I usually ask the question to indicate people’s level of understanding on different occasions, such as when a friend wants to know more about Power BI, or in a job interview from a candidate who applied for a Power BI related role, or my students attending a training course. Depending on the context that I ask the question, the responses are often pretty different.

It is the general rule of thumb to know what a “thing” is before using it. The “What is X?” (and X is the name of a “thing”) is a broad question, so the answer is also broad. Therefore we usually need more digging to get a better understanding of the “thing”. 

In our case, the “thing” is Power BI, so the question is “What is Power BI?”. And the answer is:

“Power BI is the Business Analytics platform part of a larger SaaS platform called Power Platform offering from Microsoft.”

Now, let’s dig a bit more with two more questions:

  • What is a Data Platform?
  • What is Saas?

Let’s quickly answer those questions.

Continue reading “Power BI 101, What is Power BI”