← This weekCommunityData engineering & the warehouse/lakehouseDatabricksThe discussion centers on whether conformed lookup tables and enriched materialized views are suitable for the Silver layer in Databricks, particularly when handling data from multiple clients. The datasets, such as Users and Store orders, share similar attributes across clients.
MyDataWork POV — For teams juggling data from multiple clients, conforming dimensional values in the Silver layer isn't just a technical choice—it's a sanity saver. When datasets like Users and Store orders share attributes, standardizing them early streamlines downstream analytics. But if your data sources are wildly diverse, this might be overkill. Concentrate on areas where uniformity will genuinely simplify processes instead of merely increasing complexity. It's about balancing effort with the clarity you gain.
Discussion happens on Reddit — no comments are hosted here.