Skip to main content

Real-Time Intelligence

Needs Votes

Alert option for when eventstreams fail

Vote (1) Share
August Asheim Birkeland's profile image

August Asheim Birkeland on 09 Apr 2024 19:04:24

Hello!


TLDR: is there a way of setting up an alert for when an eventstream fails to ingest data? If not, this is an Idea for Fabric.

I know that you can monitor the eventstream incoming and outcoming data, as well as checking the runtime logs. What I want is for instance an automatic alert when a runtime error is observed in the Runtime Log of the EventStream.


The reason why we need this:

Im working at a project were we use eventstreams to ingest incoming data. Several times we have experienced that the eventstream fails, so far it has been due to the following reasons:

  • Deactivating the Fabrics user that was used to create the eventstream
  • Downgrading the privelage of the user that created / OR EDITED the eventstream
  • More specifically: we had created a set of eventstreams with a serviceaccount that has the highest privelages. However, minor details of the eventstream had been edited with our private accounts. Yesterday we lowered our private accounts to READ ONLY in the workspace, and this made the eventstreams fail, presumably since the last account that edited the evenstreams were these private accounts, and they do not have write access to the bronze layer anymore.

We can not accept the data in our bronze layer not being up to date. At the same time we do not have the capactiy to check up on all eventstreams every now and then to check that they work as intended.

Thanks in advance!

Administrator on 24 Jul 2024 10:08:20

Thanks for this great idea. Product team takes this into our backlog and continue collecting more vote for this idea.