Monte Carlo embraces Fivetran, enables end-to-end observability for data pipelin...
source link: https://venturebeat.com/data-infrastructure/monte-carlo-embraces-fivetran-enables-end-to-end-observability-for-data-pipelines%ef%bf%bc/
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
Monte Carlo embraces Fivetran, enables end-to-end observability for data pipelines
Join top executives in San Francisco on July 11-12, to hear how leaders are integrating and optimizing AI investments for success. Learn More
San Francisco-based Monte Carlo Data, a provider of automated data observability capabilities for enterprises, today expanded its footprint by announcing a no-code integration with data ingestion major Fivetran. The move will enable joint customers of the companies to monitor the health of their data pipelines end-to-end.
Monte Carlo has already announced integrations with multiple players in the data ecosystem, including Snowflake and BigQuery. The integration with Fivetran marks another step for Monte Carlo toward complete data observability for the modern data stack, right from the orchestration layer to data stores. The company competes with players like Acceldata and BigEye in the space.
The data game
Enterprises today are bullish on data-driven initiatives. They’re ingesting more and more information from different sources, making pipelines complex and increasing the chances of data downtime — periods when data is inaccurate or erroneous. Bad data can directly harm applications and users, leading to inaccurate decision-making and eventually days’ worth of fire-fighting.
Monte Carlo, founded in 2019, addresses this challenge with its machine learning-powered platform that provides enterprise data analysts with a holistic view of data reliability for critical business and data product use cases in near real time. The platform detects and sends alerts about incidents faster, allowing teams to address data downtime before it has a significant impact on the business.
Event
Transform 2023
Join us in San Francisco on July 11-12, where top executives will share how they have integrated and optimized AI investments for success and avoided common pitfalls.
Fivetran integration for pipeline observability
With its engagement with companies like Snowflake, Databricks and Google BigQuery, Fivetran addressed the challenge of monitoring data where it’s stored, but that did not include the complete ecosystem. So the company started expanding to other layers, plugging into business intelligence tools like Looker, Tableau and Mode as well as ETL offerings such as dbt and Airflow.
“We’ve seen incredible adoption of our dbt integration, which has demonstrated to us that customers require more and more visibility into the orchestration layer. And with Fivetran being one of our customers’ favorite ELT tools, this new integration will be transformative and give them the ability to detect and troubleshoot [data pipeline] issues faster,” Lior Gavish, CEO and co-founder of Monte Carlo, said.
Gavish told Venturebeat that the quick, no-code integration ensures data pipelines built with Fivetran connectors are monitored as soon as they are built. Observability is built right into them, giving a clear view of upstream problems — from issues related to data freshness, volume and distribution to schema and lineage gaps — before they impact downstream products.
Notably, as and when an issue occurs, notifications are surfaced within the Monte Carlo UI and sent as alerts in Slack, Microsoft Teams, etc. This takes the user directly to all the information and context, like Fivetran logs, that they need to address the problem.
“The integration is the latest in a long list of solutions and platforms that Monte Carlo has available … that scale observability across the modern data stack. Our solution has helped companies using Fivetran reduce data incidents by nearly 90% and save over 100 hours per quarter in data engineering time spent detecting and resolving data quality issues,” Gavish added.
The offering has been beta-tested by multiple enterprises, including Backcountry.com, Checkout.com and MasterClass, and can now be added to the stack free of charge by joint customers of the two companies.
VentureBeat's mission is to be a digital town square for technical decision-makers to gain knowledge about transformative enterprise technology and transact. Discover our Briefings.
Recommend
-
35
Let’s say that you have a function and you want to know what the area is under the curve between 0 and pi.
-
16
Introduction I introduced MCHT two weeks ago and presented it as a package for Monte Carlo and boostrap hypothesis testing. Last week, I del...
-
47
Monte Carlo Simulation in R with focus on Option Pricing Ojasvin Sood
-
18
Rayt
-
12
Las Vegas vs. Monte Carlo algorithms yourbasic.org Zener...
-
9
Monte Carlo Simulation Monto Carlo simulation is a technique for approximating future behaviour based on randomly sampled numbers. By sampling from different probability distributions it is possible to use M...
-
7
Monte Carlo integration uses random numbers to approximate the solutions to integrals. While not as sophisticated as some other
-
12
Eliminating Data Downtime with Data Observability - Why We Partnered with Monte Carlo Data Posted on 2021, Feb 09 2 mins read
-
3
Monte Carlo plans to scale data observability platform Skip to main con...
-
3
Fivetran data integration platform embraces private deployments, AWS GovCloud
About Joyk
Aggregate valuable and interesting links.
Joyk means Joy of geeK