UPDATE 16 Sep 2024
I have published a new blog explaining how to resolve the issue of Capacity Admins disappering. Check it out here.
NOTE
This method uses the Create or Update a Resource action. This action uses PUT method which messes up with capacity admin settings on the Fabric Admin portal. This is stated by Kevin and later by Wiroj in the comments section. The REST API to scale or down the Fabric Capacity is as follows:
patch https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroup}/providers/Microsoft.Fabric/capacities/{capacityName}?api-version=2023-11-01If you run the preceding API using PATCH method, you do not need to have your capacity running. In other words, you can run the API even when the capacity is paused. I will post a new blog explaining this but for now, I hope the information provided here is helpful.
Read more in my comment and Michael’s about this topic.
In a previous post I explained how to manage the capacity costs of a Fabric F capacity (under Pay-As-You-Go pricing model) using Logic Apps to Suspend and Resume it.
A customer who read my previous blog asked me “Can we use a similar method to scale up and down before and after specific workloads?”. This blog post is to answer exactly that.
I want to make some important points clear first and before we dig deeper into the solution:
- The method described in this post works with Fabric F SKUs under Pay-As-You-Go pricing model.
- If you have a Power BI Premium capacity, then this method is not valid for your case. But you might be interested in the autoscale option for Power BI Premium capacities.
- Depending on your current workload, scaling down may not work due to resource unavailability.
- Depending on your workload, this method may take a while to go through.
- You need to be either a Capacity Admin or a Fabric Admin to successfully implement this method.
- This method works based on user authentication, however, you may want to use Service Principal or Manage Identity which require more effort but could be a more desirable method in many scenarios.
- This post explains a very basic scenario, you’re welcome to scale it to your specific needs.
- You can consider this post as a continuation of the previous post. So if you are unsure you correctly understand what this blog is trying to explain, then I suggest you read my previous post first where I explain the Logic Apps implementation in more detail.
The Problem
I have an F Fabric capacity and I want to upscale it to an upper tier between the pick-time from 8 AM to 12 PM local time, then downscale it to its original tier.
The Solution
There are many ways to do this including using Azure Resource Manager APIs, Manage Azure Resources in PowerShell, or using Azure Resource Manager connector that can be used on Azure Logic Apps, Power Automate Premium, and Power Apps Premium. This post explores the use of Azure Resource Manager connectors in Azure Logic Apps. With that, let’s begin.
- On Azure Portal, search for Logic apps
- Select the Logic Apps service
- Click the Add button
- Pick a Subscription from the list
- Pick a Resource Group from the list or create a new one
- Enter the Logic App name
- Select the Region from the list
- Select No if you do not require to Enable log analytics
- Select Consumption from the Plan type
- Click the Review + create button
- Click the Create button
- Navigate to the newly created Logic App and click the Edit button
- Select the Recurrence trigger
- Set the Interval to 1 Hour
- Add both Time zone and Start time parameters and set the desired values
- Add a Condition action under the Control operation and use the following expression to check if the time is between 8 AM and 12 PM:
int(formatDateTime(convertFromUtc(utcNow(), 'New Zealand Standard Time'), 'HH'))
- On the True side of the condition, add a Create or update a resource action which could be found by searching for Azure resource manager operation
- After passing your credentials and signing in select the Subscription from the list
- Select the Resource Group from the list
- Select the Microsoft.Fabric from the Resource Provider list
- Enter capacities/your_capacity_name on the Short Resource Id (replace the “your_capacity_name” with your actual capacity name)
- Enter 2023-11-01 on the Client Api Version
- Add the Location and SKU Name parameters, then select the location of your capacity and enter the desired SKU you want to upscale
- Repeat the steps from number 18 and add another Create or update a resource for the False side of the condition to downscale the SKU and save the solution
The entire workflow must look like the following image:
That is it. The workflow will automatically run every hour and upscale the Fabric capacity to an F8 SKU if the time is between 8 AM and 12 PM, otherwise it ensures that the capacity SKU remains at F2 SKU.
Note
This Logic App successfully works only when the Fabric capacity is running. So if the capacity is suspended, then the flow will fail with the BadRequest, Service is not ready to be updated error. The reason for that is the Create or update a resource action from the Azure resource manager operation uses a PUT API method which requires the Fabric capacity to be running.
Conclusion
This blog post offers step-by-step guidance on automating the scaling of Microsoft Fabric F capacity using Azure Logic Apps and the Azure Resource Manager connector. I tried to also highlight crucial considerations and limitations.
As always, I invite you to share your thoughts in the comments below. Follow me on X aka Twitter and LinkedIn to get notified of my new posts.
Thank you for this tutorial. I was able to create an auto-scale Logic App based on the information you provided. I encountered (and resolved) issues caused by the “PUT” command. Our capacity administrators list was getting deleted every time the Logic App ran. that was caused by the “PUT.” You can get around this by calling the API via an HTTP action and using a “PATCH” command.
Thank you for the detailed tutorial, I wanted to share that I was having issues with step 16 I was getting the following error, it could have been a miss on my part.
“InvalidTemplate
Unable to process template language expressions for action ‘Condition’ at line ‘0’ and column ‘0’: ‘The template language function ‘greaterOrEquals’ expects two parameter of matching types. The function was invoked with values of type ‘Integer’ and ‘String’ that do not match.’.”
As a workaround after the Recurrence step I added steps to get the Current Time and Converted Time Zone and then I used the Converted Time Zone output as an input for Step 16 and that solved the issue.
Hi Carlos,
I am happy to see that you found this blog helpful.
Regarding the challenge that you have faced, the formatDateTime() function returns a string in ‘HH’ format. You might have missed adding the int() function.
I have also started my YouTube channel, you might find it helpful too.
Cheers.
Thank you so much for providing such a comprehensive tutorial. We diligently followed each step and successfully ran the application. However, we noticed that all administrative permissions related to capacity were unexpectedly removed. Could you kindly provide any suggestions or guidance on this matter? We would greatly appreciate your assistance. Thank you in advance.
Hi there,
Welcome to biinsight.com and thanks for your comment.
I’m sorry to hear about the issue you’re experiencing. Please ensure you used the correct connector (Azure resource manager) and the correct Action Name.
I run the same solution in at least 3 different tenants and they all run as expected and we never experienced something similar.
Please update us if have any progress resolving it.
Good luck!
Hi, I have rechecked the Azure Resource connector and Action Name and ensured that they are set up as suggested. However, I am still encountering the same issue. Regarding the Client API Version, I have been using the version 2023-11-01. I would appreciate if you could confirm whether this is the correct version to use
I’m sorry to hear that you’re encountering this issue. The api version is correct. In fact, the api would not run if the version isn’t right.
Look, I have used exact same method for more than 6 months in multiple occasions with different settings and user management scenarios and none had faced such an issue.
Can you please clarify what exactly happens in your scenario? What administrative permissions are removed and from where?
Thanks
Could you let me know if that works fine with switch between F64 and F16 ? F64 during the day to allow CoPilot. F16 during evening when no one is using the Capacity.
Switching between F64 and F16 capacities can be beneficial, but there are some intricacies to consider. If there are ongoing data refreshes when downscaling from F64 to F16, it might cause issues. Ensure no critical processes are running during the switch.
Even if the capacity is unused in the evening, check for any background data refreshes or other scheduled processes. These could be disrupted by downscaling.
If the capacity is truly unused during the evening, consider automatically suspending it and resuming it during the day, including keeping it paused over weekends. For detailed guidance on automating this with Logic Apps, refer to my blog post and its YouTube video: https://www.biinsight.com/microsoft-fabric-capacity-cost-management-part-2-automate-pause-resume-capacity-with-azure-logic-apps/
Hi, How can I send my presentation with captured screen to you for clarify my issues? Thanks.
If you use OneDrive, then you can put everything in there and share the link here. Please only share the images (no ZIP files). Or you can record your screen and share the video.
Hi,
Here’s link of screen captured
Thanks
Hi there.
Thanks for sharing the screenshots.
You’re correct! I did not have these kinds of settings on my configuration therefore I haven’t faced the problem.
Thank you for bringing this to my attention.
I think I know what the issue is, but I need some time to check and find a way to fix it.
I will post here as soon as I find the solution.
Cheers
Hi Wiroj,
Thanks again for bringing this to my attention. I have spent a couple of hours to fix the current implementation.
The issue is that the ARM (Azure Resource Manager) action used in this solution, Create or Update a Resource, uses PUT API method in the background which messes up with some capacity settings in the Fabric Admin Portal; while the correct method for our scenario is PATCH.
I tried to force the action to use PATCH method but it totally broke the entire solution.
I also talked to a Microsoft programme manager at the Azure Logic Apps team and got the confirmation that this is not possible to force the current ARM (Azure Resource Manager) actions to use PATCH.
However, we can add properties parameter to define the administration members, but it is not practical as we need to mention the admin members, which is not ideal.
Hence, it was time to seek another approach. The good news is that I successfully implemented it in Azure Logic Apps, but it is not as straightforward as using the ARM connections is.
I will post a concise blog about it in the near future, but for now, you can use the following REST API to scale up or down the capacity without needing to have the capacity running:
patch https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroup}/providers/Microsoft.Fabric/capacities/{capacityName}?api-version=2023-11-01Hopefully that helps for now.
Stay tuned for the next blog.
Cheers
Hi,
Thank you for your suggestion. I finally use properties parameter to define the administration members. I know that not a practical, but Microsoft support also suggest me to do that.
Again, thank you very much for your support
Cheers,
That’s awesome to hear that you’re being successful in implementing the solution.
Thanks heaps for sharing your experience with us.
Hi there,
You may find that after the this App run, the Administrator for the capacity will be empty.
I checked the ARM API and find that the properties.administration is a required parameter for the API.
Adding a property to the parameter will resolve this. Raw code for the app looks like this.
“body”: {
“sku”: {
“name”: “Fxx”
},
“location”: “xxxx”,
“properties”: {
“administration”: {
“members”: [
“[email protected]”
]
}
}
},
Michael, thanks for sharing your thoughts.
While this would work, it is not practical in a long run.
If we want to stick to Azure Logic Apps implementation, then I am more inclined to use the HTTP action and run the ARM API using the PATCH method instead of hard coding admin members.
Obviously, when it comes to automation of APIs, Azure Logic Apps is one option of many…