Trifacta and Google Cloud deliver new cloud data warehouse optimizations to reduce processing time from hours to seconds, cutting consumption charges by up to 90%
Trifacta, the Data Engineering Cloud company and the makers of Google Cloud Dataprep, announced widespread customer adoption of its BigQuery pushdown functionality. With 57 percent growth during the month of September alone, BigQuery Google Cloud Dataprep customers now process over two trillion rows of data each month. Since its introduction, numerous customers have adopted Trifacta’s BigQuery capabilities, including Amway, Nexstar, C6Bank, SearchKings, OFI Asset Management, Koffie Labs, Realtruck, MyMuscleChef, and more.
Marketing Technology News: DTEN as a Service Makes Professional, Affordable Video Collaboration Available to Everyone
“Our new process runs in 22 seconds, dropping our consumption cost by approximately 90 percent.”
Earlier this year, Trifacta announced that Google Cloud Dataprep could run inside BiqQuery as the engine for data transformation and cleansing. Trifacta has now expanded this popular functionality to address additional use cases. Updates include handling file to BigQuery pushdown, BigQuery Sampling, BigQuery Upsert, and enriched data engineering experience with BigQuery pre/post-processing functions. With the enhancements, customers have been able to accelerate design time and data pipeline speed more than 60 times, saving thousands of dollars on each job.
“Thanks to Google Cloud Dataprep by Trifacta’s self-service data engineering and its BigQuery Pushdown Optimization capabilities, we were able to spend 15 minutes rewriting a legacy process that historically took more than five hours to complete,” said Bekkie Brown, Manager Data Platforms & Operations at Amway. “Our new process runs in 22 seconds, dropping our consumption cost by approximately 90 percent.”
“Trifacta is regularly enhancing Google Cloud Dataprep experiences to further optimize data engineering work, which helps our customers accelerate time-to-value,” said Sean Kandel, Co-founder and CTO at Trifacta. “Thanks to our unique partnership with Google Cloud and integration with BigQuery, it’s never been as easy and fast to deliver results with data pipelines at scale.”
Trifacta’s data engineering platform is natively embedded into Google Cloud as a serverless and native service known as Dataprep by Trifacta. Benefiting from deep integration with BigQuery and the constant flow of BigQuery innovation, Dataprep now offers a unique enriched design experience for analytics initiatives on Google Cloud.
“Trifacta’s data prep capabilities are helping customers process trillions of rows of data, efficiently and cost-effectively,” said Sudhir Hasbe, Director of Product Management, Data Analytics at Google Cloud. “We’re thrilled to deepen our partnership with Trifacta to further extend its support for Google Cloud and BigQuery, including launching new ELT capabilities and accelerating their time to value with data.”
Building on the success of Dataprep by Trifacta’s pushdown to BigQuery data sources, the company has bolstered functionality for “Full Pushdown” for files and data sources. This addition enables users to translate Dataprep jobs into SQL statements executed on BigQuery. Further, Dataprep enables BigQuery Upsert, which is the ability to update or insert data into BigQuery. This functionality allows data professionals to refresh a portion of a data warehouse rather than reload it entirely. Trifacta also brings to market BigQuery Pushdown Sampling, which is the ability to slice data sets in smaller chunks to offer a better experience to quickly transform and validate data visually.
Marketing Technology News: DTEN as a Service Makes Professional, Affordable Video Collaboration Available to Everyone
The latest Dataprep version comes with many other BigQuery-based specific enhancements, such as extending Dataprep jobs with pre-and-post BigQuery SQL, support for BigQuery arrays, or opening a Dataprep output result directly in the BigQuery Editor. In sum, these innovations create an enriched data engineering experience to deliver data projects at unprecedented ease and speed.
Upcoming, Trifacta will introduce further enhancements to support any external table as a source for the pushdown. This will help expand the pushdown use cases from Google Cloud Storage files as a source to Spanner, Cloud SQL, Cloud Bigtable, and other external sources supported by BigQuery. Trifacta will also support customer extensions for pushdown, including custom SQL and Javascript UDFs.
Trifacta will be present at Google Cloud Next, Google Cloud’s 3-day virtual conference focused on global digital experiences in the Cloud.
