Microsoft Fabric is an end-to-end analytics and data platform designed for enterprises who require a unified solution.
It encompasses data movement, processing, ingestion, transformation, real-time event routing and report building. It offers a comprehensive suite of services including data engineering, data factory, data science, real-time analytics, data warehouse and databases - and sets a new standard for data platforms.
Microsoft Fabric was released at the end of 2023 and has since generated significant interest. There are good reasons for this. At first glance, it may seem like just a rebranding of existing products, but the more you delve into the platform, the more you realise the change is far more substantial.
It’s fair to say it sets a new standard for data platforms, making life easier and better for everyone in the data chain. This applies whether you are a data engineer, report developer, AI developer, data consumer, or someone with overall responsibility, needing to balance costs and investments.
Drivers for moving towards Fabric
Simplication and performance as a driver:
OneLake is a central concept of the Fabric offer. As the name suggests, it’s a place for all your data. Microsoft often cites OneDrive to help describe what OneLake does. Just as you can manage all your files in OneDrive, you can manage all your data in OneLake. You can even use File Explorer in Windows to get an overview of your data.
This is a good example of a recurring theme in Fabric – simplifying administration and user experience, making previously technical tasks accessible to non-technical users or citizen developers. Ultimately, this means democratisation, – making it easier for more consumers to access data.
Fabric also achieves something entirely new by using the VertiParq engine together with the Delta Parquet format in OneLake. This combination makes it possible to achieve efficient use of storage space along with strong query performance.
Queries can now be run directly on compressed data in OneLake with good performance. For users, this reduces or eliminates the need for data copying and duplication that may have been required for performance reasons in legacy systems. The Direct Lake concept for making data available to Power BI is an example of how this opportunity can be used.
It's fair to say it sets a new standard for data platforms
Another new component is the simplified process of loading data from external systems. With the concept of shortcuts to external systems, you might completely avoid, or at least simplify, ETL processes (Extract, Transform & Load) to collect external data. With the concept of mirroring, you can access an updated and mirrored copy of an external database without writing any special code for this.
Updates are managed automatically using change data capture technology. In these ways, the need for manually constructed transformations decreases, thereby reducing technical debt whilst improving data quality.
Better AI with proven data quality:
We have seen many AI pilots over recent months, often with limited and quite general use cases. The natural next step is to focus on use cases that create real competitive advantages, leveraging data from our own unique business operations. This is where we see Fabric playing a key role.
One might say that if AI is our goal and our data is our fuel, then we need a vehicle to make the journey to that goal. We believe that vehicle is Fabric.
To succeed with AI, we need to simplify the challenging but essential task of acquiring, organising, and refining the data which underpins our AI applications. Fabric's components help us achieve controlled and proven data quality, ensuring that AI processes the right data in the right way. By simplifying this critical task, we can spend more time building unique competitive advantages. For instance, when using Azure AI Studio, it is fully integrated with Fabric.
Ask open questions with Copilot:
Within Fabric, Copilot support for data engineering lowers the barrier to building better and more efficient data pipelines, with less overhead from code reviews and testing iterations. There is also Copilot support for data science, aiding, for instance, in the development and execution of predictive models.
Perhaps the most exciting Copilot feature is the ability to support a conversation with data as part of Power BI. This feature is currently available only in preview and requires a higher license. The use cases that this feature opens, though, are very appealing to analysts and those working in the decision-making process.
Typically, the starting point might be that a problem owner only has access to prefabricated or static reports which can only partially provide the answers needed for a given problem statement. When the fixed format of these reports is too restrictive for idea generation or more specific exploration, the consumer is left without a simple solution.
While this limitation may sometimes be intentional, the ability to ask questions in natural language and to be able to track down more complex root cause analysis significantly enhances the analysis process, leading to faster and better insights. The Copilot capability in Power BI creates freedom for spontaneous questions to be answered instantly, for example during ongoing meetings or discussions.
Easier to control costs:
Customers for data platforms have increasingly demonstrated reduced willingness to pay for storage but continued acceptance of paying for computing. This is a logical consequence of scenarios like data collection from a company's products when used by customers, creating vast amounts of data. This data holds no value at rest but becomes valuable only once activated. Fabric enables working in this manner and obtaining a cost structure which matches the value structure.
Historically, data activation has involved various types of server allocation and data duplication, often accompanied by complex cost calculations and even guesswork about what the costs will be. The licensing model of Fabric is significantly simplified, especially when used with Power BI. Depending on your current data platform situation, this alone can be reason enough to consolidate all data processing under Fabric.
A simpler cloud journey:
If you are currently using on-premises solutions for your data platform but plan to move to the cloud, you’re in luck with your timing!
Transitioning from on-premises to a modern cloud-based data platform is widely acknowledged to come with benefits such as simpler administration, better scalability, security and cost.
However, with Fabric, you can now achieve a new level of simplicity and cost optimisation. This means you can more quickly reach goals like modernised decision support or entirely new AI applications. A more homogeneous and straightforward tech stack reduces technical debt in the upcoming data platform. This will free up resources for the data team to invest more time in building competitive advantages.
Building a solid foundation moving forward:
Innovation in data must be built on a solid foundation. With this in mind, we recommend Fabric. It’s a compelling platform which we believe will inspire and drive development across industry.
