Skip to main content
Microsoft Idea

Power BI

New

Speed up Power Query by allowing to break the refresh chain

Vote (66) Share
Imke Feldmann's profile image

Imke Feldmann on 20 Aug 2021 08:00:28

Although we can use quite a couple of methods to speed up slow queries (I've collected some of them here: Speed/Performance aspects – The BIccountant ), one current limitation for query refresh speed lies in the fact that PQ will always evaluate all steps back to the source to determine the "query plan".



I've tested quite a couple of scenarios where I staged intermediate results into static csv files or sheets in Excel and the performance improvements have been enormous.

So what would be really great is that instead of having to create these workarounds, we could edit a property of a query step so its result would be evaluated as a first step and then stored in cache before continuing to evaluate the further steps.



Actually, maybe even opt for a "bring your own datalake"-version where we can store to Azure Blob if the data is too large for the cache. But in a way, that refresh is properly synchronized.

Comments (3)
Imke Feldmann's profile image Profile Picture

Andreu fil on 19 Apr 2024 13:23:42

RE: Speed up Power Query by allowing to break the refresh chain

Good information, thankssehackear.top

Imke Feldmann's profile image Profile Picture

J B on 21 Mar 2023 15:17:38

RE: Speed up Power Query by allowing to break the refresh chain

100% agree.I suggest the function be called "Query.Buffer"

Imke Feldmann's profile image Profile Picture

Brent Pearce on 20 Aug 2021 14:50:02

RE: Speed up Power Query by allowing to break the refresh chain

If i could improve any 1 thing in PQ it would be this issue. As PQ spreads to other Msft products this becomes an even more important issue. My preference would be for Table.Buffer() and List.Buffer() to actually do what it seems they should do... buffer the results such that downstream query actions need to look no farther upstream than this buffered data. Read the source only once!