Welcome to Microsoft Fabric Ideas
1-10 of 24 results
John White on 2024-03-19 19:30:24
Use the M365 Audit Log as a source for EventStreams
Currently there is a tremendous amount of value in the M365 Audit log, but it is extremely difficult to access. Retention is low for API access and other methods are batch oriented at best. Apart from some work in Sentinel, there are no ways to react to events in near real time. This seems lik...
Thomas Pagel on 2023-10-07 19:12:33
Event Processing: Allow CUD actions
It would be great if you can define Create/Update/Delete actions from Event Processing. So depending of a field you can say i.e. "if this is field contains "delete" it's a delete operation, use the ID stored in field ID to identify the record to be deleted in your lakehouse". Same with Updates...
Uri Barash (administrator)
This is already feasible with a combination of materialized views and arg_max on the relevant columns. Please see how it is done for Cosmos DB change feeds here: Get the latest versions of Azure Cosmos DB documents. We will consider adding UX support or wizard, based on popuarity of the idea.
Eileen Allan on 2023-05-26 20:18:50
Administrator
Continous ingestion from OneLake is planned and in execution. It has quite a few aspects, including picking up files landing in OneLake files foder, accelerating Delta table caching, and more.
Thomas Pagel on 2023-10-07 19:07:08
Event Processing: Derive value from complex JSON
Please allow to derive a value out of a complex JSON which is stored in a single field in your stream
Uri Barash (administrator)
Complex JSON cells can be processed today using update policies and JSON data mappings in the KQL database. Creating a no-code experience for this will depend on popuarity of the idea.
Sandeep Pawar on 2023-06-30 16:23:59
Install Kqlmaginc extension by default in Fabric
It's not very widely known or shared that Kql queries can be run in Fabric notebooks. Could you please install the Kqlmagic extension in the default Fabric environment so it can be used without installation. Thanks. <a href="https://learn.microsoft.com/en-us/fabric/real-time-...
Administrator
This is on our short-term roadmap
GARIMA WADHWA on 2024-03-30 16:00:09
Support Liquid Clustering on Delta Tables
The following are examples of scenarios that benefit from clustering:Tables often filtered by high cardinality columns.Tables with significant skew in data distribution.Tables that grow quickly and require maintenance and tuning effort.Tables with access ...
Thomas Pagel on 2023-10-07 19:00:22
Clear Events from CustomApp
A Custom App enables you to skip the EventHub and directly ingest data into an Event Stream. While this is very helpful, the features are very limited. Especially when it comes to removing events from the input buffer.This has i.e. the following effect: D365 F&O allows to define "data e...
Eileen Allan on 2023-05-26 20:29:13
Provide UI interface to view and modify key policies for SaaS experience (caching, mirroring, batching)
Provide UI interface to view and modify key policies for SaaS experience (caching, mirroring, batching)
Uri Barash (administrator)
This is definitely the direction and the different elements of the idea are in different design and implementation phases.
Natalie Harrison on 2023-05-26 20:27:56
Provide connectivity to Kakfa and IoT Hub in Ingestion wizard
Provide connectivity to Kakfa and IoT Hub in Ingestion wizard.
Administrator
- IoTHub: Supporting IoTHub in the ingestion wizard is being considered for the upcoming period. The Eventstream item in Fabric is currently undergoing work to support Azure IoT Hub as a source. The wizard may use or configure this functionality as well.
- Kafka ingestion directly to a KQL database is described here: Stream Data from Apache Kafka to Azure Data Explorer (confluent.io) Additionally, the Custom App source of Eventstream can be used to ingest data from a Kafka cluster using its connection string, and from there routed to the KQL database. Accelerating it via the wizard is also being reviewed.
Livia Bijelli on 2023-09-11 21:32:43
Eventstream source with Custom Sample Data
There is a dependency of Partial Data Preview to complete Schema Definition for KQL Database destination.The only way to configure mapping while events aren't being sent to Event Hub source is using KQL commands directly in the database.For a no-code experie...