Friday, May 2, 2025

MS Fabric Parquet Files Query Mods Cost Case Studies? Compute and Capacity and Storage billing line items


General Microsoft Fabric Billing Model

Microsoft Fabric uses a capacity-based pricing model. You purchase a Fabric Capacity (an F SKU) with a certain number of Capacity Units (CUs). These CUs are a pool of compute resources shared across all Fabric workloads (Data Factory, Synapse Data Engineering/Data Science, Data Warehouse, Data Lakehouse, Power BI, etc.).  

Your bill will primarily consist of:

  1. Capacity Compute: Based on the CU consumption of all workloads running within your Fabric capacity.
  2. OneLake Storage: Based on the amount of data stored in OneLake. Transaction costs (reads, writes) to OneLake are generally included in the capacity compute.
  3. Optional Add-ons: Such as резервирование capacity for discounted rates.

Impact of Direct Lake vs. Direct Query on Billing

While direct case studies are lacking, we can infer the potential billing implications:

1. Capacity Compute (CU Consumption):

  • Direct Lake:
    • Queries against Direct Lake models are processed by the VertiPaq engine (the same in-memory engine used by Power BI Import mode).  
    • Since the data is directly read from OneLake in Parquet format and loaded into memory on demand for querying, the compute consumption is expected to be efficient and potentially lower compared to Direct Query for analytical workloads.
    • The refresh operation for Direct Lake models is metadata-only, making it a low-cost operation in terms of compute.  
    • However, if a Direct Lake model falls back to Direct Query (e.g., due to unsupported features or row-level security on the SQL endpoint), the compute consumption would be similar to a standard Direct Query.
  • Direct Query:
    • DAX queries are translated into the query language of the underlying data source (e.g., T-SQL for Fabric Data Warehouse or SQL endpoint of Lakehouse) and executed on that source.  
    • The compute cost here depends heavily on the efficiency and scale of the underlying data source. If the source is not optimized for BI-style queries, it can lead to higher CU consumption as Fabric waits for the source to return results.
    • Each query execution against the source will contribute to the overall capacity consumption.

2. OneLake Storage:

  • Direct Lake: The underlying data resides in OneLake in Delta Parquet format. The storage cost will be based on the volume of data stored in OneLake. Direct Lake itself doesn't incur additional storage costs beyond the base OneLake storage.
  • Direct Query: The data also resides in some form of storage (e.g., OneLake, Data Warehouse). Similar to Direct Lake, the storage cost is tied to the underlying data storage.

3. Specific Billing Line Items:

Without specific case studies, it's challenging to provide exact billing line items. However, when you analyze your Fabric capacity consumption in the Microsoft Fabric Capacity Metrics app, you would likely see:

  • For Direct Lake: Operations related to "Semantic Model Queries" or similar, reflecting the CU consumption of the VertiPaq engine processing the queries. OneLake read operations might also be visible.
  • For Direct Query: Operations related to the specific workload being queried (e.g., "Data Warehouse Query", "Lakehouse SQL Query"), indicating the CU consumption while Fabric interacts with that engine.

Key Considerations:

  • Optimization of Underlying Sources: For Direct Query, ensure your Data Warehouse or Lakehouse SQL endpoint is well-optimized with appropriate indexing and partitioning to minimize query execution time and thus CU consumption.
  • Data Volume and Complexity: Larger data volumes and more complex queries will generally consume more CUs regardless of the query mode. However, Direct Lake is designed to handle large volumes efficiently.  
  • Fallback to Direct Query: Be aware of scenarios where Direct Lake might fall back to Direct Query, as this could impact performance and potentially increase CU consumption.
  • Monitoring: Regularly monitor your Fabric capacity consumption using the Metrics app to understand how different workloads and query types contribute to your bill. Filter by workload and operation type to get a better understanding.

In summary, while direct billing case studies for Direct Lake vs. Direct Query are not readily available, the expectation is that Direct Lake, being optimized for large-scale analytics on OneLake data, would generally lead to more efficient compute utilization for BI workloads compared to Direct Query against potentially less optimized data sources. Storage costs are primarily tied to the volume of data in OneLake, irrespective of the query mode used for consumption.

To get a clearer understanding of your specific billing, it's recommended to:

  1. Experiment with both Direct Lake and Direct Query scenarios with your data.
  2. Monitor the CU consumption in the Microsoft Fabric Capacity Metrics app for each scenario.
  3. Analyze the operation-level details to see how each query type contributes to your overall capacity usage.
  4. Review your Azure bill for the Fabric capacity to see the total CU consumption and storage costs.

This hands-on approach with your specific data and workloads will provide the most accurate insights into the billing implications of Direct Lake vs. Direct Query in your environment.

What difference between Microsoft Power BI Desktop (Optimized for Power BI Report Server - September 2024) and Microsoft Power BI Desktop and Microsoft® Power BI Report Builder

  The differences between Microsoft Power BI Desktop (Optimized for Power BI Report Server - September 2024)Microsoft Power BI Desktop, and Microsoft Power BI Report Builder lie in their purpose, functionality, and target environment. Below is a concise comparison:

1. Microsoft Power BI Desktop (Optimized for Power BI Report Server - September 2024)
  • Purpose: A specialized version of Power BI Desktop tailored for creating and publishing reports to Power BI Report Server, an on-premises reporting platform.
  • Key Features:
    • Optimized for compatibility with Power BI Report Server (September 2024 release).
    • Does not include preview features until they are generally available, ensuring stability for on-premises environments.
    • Supports interactive Power BI reports (.pbix files) and connectivity to on-premises data sources without requiring a gateway for local data.
    • Lacks some cloud-specific features (e.g., dashboards, Q&A, shared datasets, advanced AI/ML capabilities) available in the Power BI Service.
    • Can be installed side-by-side with standard Power BI Desktop.
  • Environment: On-premises, designed for organizations needing to keep data and reports within their firewall for security or regulatory reasons.
  • Limitations:
    • Does not support paginated reports (.rdl files).
    • Limited to features supported by the specific Report Server version (e.g., no export to Excel with current layout).
    • Custom security extensions may prevent direct saving to the Report Server; reports must be uploaded manually.
2. Microsoft Power BI Desktop
  • Purpose: The standard version of Power BI Desktop, designed for creating interactive reports and dashboards for the Power BI Service, a cloud-based platform.
  • Key Features:
    • Supports a broader range of features, including preview features, frequent updates, and cloud-specific capabilities like dashboards, shared datasets, Q&A, and AI-driven insights.
    • Creates .pbix files for interactive, screen-optimized reports with advanced visuals and data modeling.
    • Integrates with Microsoft Fabric and other cloud services (e.g., Power Automate, Power Apps).
    • Allows data import, transformation, and visualization from various sources using Power Query and DAX.
  • Environment: Primarily cloud-based (Power BI Service), though reports can be shared locally or published to on-premises servers with limitations.
  • Limitations:
    • Not optimized for Power BI Report Server; reports created may not be fully compatible with on-premises servers due to version differences or unsupported features.
    • Cannot create paginated reports.
3. Microsoft Power BI Report Builder
  • Purpose: A standalone tool for creating paginated reports (.rdl files) optimized for printing or exporting to formats like PDF or Excel.
  • Key Features:
    • Designed for "pixel-perfect" paginated reports with precise control over layout, ideal for printed reports or fixed-format documents.
    • Based on SQL Server Reporting Services (SSRS) technology, supporting Report Definition Language (.rdl) files.
    • Supports accessibility features (e.g., tagging headers, text box properties) and conditional formatting.
    • Can connect to Power BI datasets or other data sources via SQL Server Analysis Services.
    • Compatible with Power BI Report Server and older SSRS versions (2016, 2017, 2019).
  • Environment: Primarily used with Power BI Report Server or SSRS for on-premises deployment, though paginated reports can be published to Power BI Service with a premium capacity.
  • Limitations:
    • Lacks the interactive, exploratory features of Power BI Desktop reports.
    • Not suited for building dashboards or highly interactive visuals.
Key Differences Summary
Aspect
Power BI Desktop (Optimized for Report Server)
Power BI Desktop
Power BI Report Builder
Primary Use
Interactive reports for on-premises Report Server
Interactive reports for cloud (Power BI Service)
Paginated reports for printing/export
File Format
.pbix (interactive reports)
.pbix (interactive reports)
.rdl (paginated reports)
Environment
On-premises (Power BI Report Server)
Cloud (Power BI Service)
On-premises or cloud (premium)
Feature Set
Stable, server-compatible features
Latest features, cloud-focused
Pixel-perfect, print-focused
Preview Features
Excluded until generally available
Included
Not applicable
Data Connectivity
On-premises focus, no gateway for local data
Cloud and local with gateway
SSRS-based, dataset-focused
Interactivity
High (slicers, filters, visuals)
High (dashboards, AI)
Low (static, formatted reports)
Compatibility
Tied to specific Report Server version
Power BI Service
Report Server, SSRS, premium cloud
Which Tool to Use?
  • Use Power BI Desktop (Optimized for Report Server - September 2024) if your organization uses Power BI Report Server and needs interactive reports hosted on-premises.
  • Use Power BI Desktop for cloud-based reporting, collaboration, and access to the latest features in the Power BI Service.
  • Use Power BI Report Builder for paginated reports requiring precise formatting for printing or export, especially in on-premises environments or premium cloud workspaces.
For further details, refer to Microsoft’s official documentation on Power BI Report Server and Power BI Desktop