Catalog of ETL and EL-T tools
Catalog of ETL and EL-T tools

Catalog of ETL and EL-T tools

As companies collect increasing amounts of data, the latter tends to end up scattered around various different sources. For this reason, there is a need to distribute data from various data sources to the data warehouse.

This explains the recent explosion in the past few years of ETL/ELT tools. (internal, open-source, and SaaS). This new trend is not going to stop, and we'd rather bring visibility and structure soon.

At Castor, we believe the first step to structure the ETL tools market, is more transparency. For that reason, we put up a list of all the ETL/EL-T tools we heard of. More context on this analysis and the state of the ETL tools ecosystem here.

image

💡

This list is still exploratory, may contain errors, or lack information. Please reach out to us, if you notice anything wrong: louise@castordoc.com.

Get started with ETL and EL-T tools

📢

In-depth analysis and evolution Read the full breakdown by generation and market analysis of ETL tools here.

image

🌱

What does Reddit think about ETL tools? You can find the thread here.

image

Dive deeper into ETL and EL-T tools

What does each column in the benchmark below mean?

Processing: Does the solution propose batch processing, stream processing, both?

Number of data sources: From how many applications can the ETL tool export data?

Observability: Does the tool allow you to see how your syncs are performing at large? Can you easily identify when sync fails and why? Can you get an alert when sync fails?

Transformation: Which kind of transformation does the solution offer? SQL, Dbt, both?

Custom connector: Can the solution build and maintain a custom data connector upon customer request?

Modeling: How is the data warehouse queried using the solution? Is it pure SQL, or does the solution have easy mode/ no-code features such as drag and drop?

Incremental synchronization: When synchronizing the sources with the destination, does the solution offer the possibility to only synchronize data that has been modified since the last export, or does it send all the data in the segment each time

Community: Is there an online community around the tool? How helpful and knowledgeable are the responses found in this community?

Features:

Capture deletes: Data deleted in the sources can still be accessed in the destination although it is marked as "deleted" in the destination

Custom data: The solution replicates your custom data. Custom data includes custom objects, tables, and fields that you have configured in the source system to better suit your business needs.

Re-sync: Re-sync all your data from scratch.

Data blocking: Prevent some tables or columns from replicating in your destination. Make sure only relevant data is synced to the destination, and that PII information remains protected.

History: The solution offers the possibility to see how your data changed over time

Import API: Data sent to the Import API is processed and sent to your destination through the solution like data from any other integration. Possibility to pull in data from any rest API.

Automatic scaling: High availability infrastructure which can process billions of records every day

Change data capture: Change data capture (CDC) is a process that captures changes made in a database and ensures that those changes are replicated to a destination such as a data warehouse.

Failed syncs: Do you get a notification when sync fail?

ETL Benchmark and Key Features

NameWebsiteDeploymentPricingProcess usedOptimized forProcessingOut-of the box connectorsTransformationsincremental updatesChange data captureHistoryAutomatic scalingImport APIFailed syncfeaturesSecurityObservabilitySetup timeCommunityNo-code platform
Fivetran
Proprietary Cloud-based
Volume-based
ELT
ExtractionLoading
BatchStream/real-time
150+Create custom/Rest API
DbtSQL
HistoryCapture deletesData blocking
GDPR compliantHIPAA compliant
5min- 2hrs
Stitch
Proprietary
Volume-based
ELTETL
Transformation LoadingExtraction
BatchStream/real-time
130+
SQLPythonJavaGUI
Import APIAutomatic scalingSupport for paying customersExtract data from any source using singerEasy schedulingEnterpriseNo code
GDPR compliantHIPAA compliantSOC2 compliant
Airbyte
Open-source
To be announced
ELT
ExtractionLoading
BatchStream/real-time
110+Create custom/Rest API
Dbt
Historysupport within 2-3hrsBuild a new connector under two hoursNormalized schemas.Easy schedule
Xplenty
Proprietary Cloud-based
Connector-based
ETLELT
ExtractionLoadingTransformation
Stream/real-timeBatch
140+
GUINo-code
Import API
SOC2 compliantGDPR compliantHIPAA compliantCCPA
Rivery
Proprietary SaaS
Volume-based
ETLELT
ExtractionLoadingTransformation
Stream/real-time
150+Create custom/Rest API
SQL
Change data capture
GDPR compliantHIPAA compliantSOC2 compliant
Alooma
Proprietary
Contact only
ETLELT
ExtractionLoadingTransformation
Stream/real-timeBatch
GUINo-code
Automatic scalingData visualization
SOC2 compliantHIPAA compliantGDPR compliantOAuth 2.0
10 min
Improvado
Proprietary
Contact only
ETL
ExtractionLoadingTransformation
200+
GUINo-code
Import APINormalized schemas.
Segment
SaaSProprietary
Company size
ETL
ExtractionLoading
BatchStream/real-time
40+
GUINo-code
Meltano
Open-source
Free
ELT
ExtractionLoading
BatchStream/real-time
30+
Dbt
Minutes
Hevo
Proprietary
Events-based
ELT
ExtractionLoadingTransformation
Stream/real-time
100+
SQL
Automatic scaling24/7 live support
GDPR compliantSOC2 compliantHIPAA compliant
Minutes
Meroxa
PaaSOpen-source
Events-based
ELT
ExtractionLoadingTransformation
Stream/real-time
10+
Change data capture
Panoply
SaaSProprietary
Volume-based
ELT
50+
GDPR compliantSOC2 compliant
Matillion
Proprietary
Contact only
ETL
ExtractionLoadingTransformation
Batch
100+
GUINo-code
Automatic scaling
Few days
Singer
Open-source
Free
ETL
Batch
100+
Apache camel
Open-source
Free
Stream/real-time
Logstache
Open-source
Free
ETLELT
Stream/real-time
200+
Safe
ETL
ExtractionLoadingTransformation
Batch
450+
No-code
IBM Datastage
Proprietary
Contact only
ETL
ExtractionTransformation Loading
Batch
Informatica
Proprietary
Contact only
ETL
ExtractionLoadingTransformation
Batch
HIPAA compliantSOC2 compliantSOC 3 compliant
Talend
Open-sourceOn-premises
Mulesoft
Proprietary
Pipeline wise
Proprietary
Integrations with Singer
Oracle
Proprietary
Pentaho
Proprietary
Skyvia
Proprietary
SAP data services
Proprietary
FlyData
Volume-based
ETL
Stream/real-time
GDPR compliant
30 min

*This is a brief attempt at classifying the tools on the market. If anything seems wrong. The feature list seems off, or if you don't see your data catalog and want to have it placed, please reach out: louise@castordoc.com