tjirab /
tff
Managing data transformation (e.g. dbt) projects at scale can be challenging: DAG violations, that one model that grew to 2K lines, duplicated logic. Inspired by the book 'Building Evolutionary Architectures' I…
About
Managing data transformation (e.g. dbt) projects at scale can be challenging: DAG violations, that one model that grew to 2K lines, duplicated logic. Inspired by the book 'Building Evolutionary Architectures' I built tff to help data transformation projects evolve in a desired direction. It comes with a github bot that you can plugin to your CI pipeline. It aims to let transformation projects evolve in the desired direction by: * ensuring layer boundary integrity; * detect duplicate CTE logic; * catch violations locally and in CI Core concepts: rules apply to models; checks apply to architecture and DAG checks. Curious to read what you think. What can be improved? What's missing?
Discovery history
- Show HN
Discuss this product