MD5 or SHA1 (or similar) encoding in Adrenaline Dataflows

jaeW_at_Onyx
Budapest / Portland, OR 🔴
related to https://dojo.domo.com/main/discussion/49054/md5-or-sha1-in-beast-mode
Sometimes users will use MD5 or SHA1 encoding to create a surrogate key for application in dimensional modelling-informed workflows.
in our use case we needed to create a dimensional table by perform a set of operations on a distinct set of values spread across several columns.
the dataset itself was 50M+ rows, the number of distinct values was around 20k. we need MD5 or SHA1 encoding to create a surrogate key between the transactions and the set of distinct values.
Currently this pipeline starts in Magic (generate the surrogate key) -> Adrenaline (perform the ETL -> Reprocess the data in Magic so we can Accumulate history using Partitions
Jae Wilson
Check out my 🎥 Domo Training YouTube Channel 👨💻
**Say "Thanks" by clicking the ❤️ in the post that helped you.
**Please mark the post that solves your problem by clicking on "Accept as Solution"
Check out my 🎥 Domo Training YouTube Channel 👨💻
**Say "Thanks" by clicking the ❤️ in the post that helped you.
**Please mark the post that solves your problem by clicking on "Accept as Solution"
2
Categories
- 7.4K All Categories
- 13 Getting Started in the Community
- 151 Beastmode & Analytics
- 1.8K Data Platform & Data Science
- 54 Domo Everywhere
- 2K Charting
- 1K Ideas Exchange
- 905 Connectors
- 239 Workbench
- 343 APIs
- 77 Apps
- 19 Governance & Productivity
- 237 Use Cases & Best Practices
- 50 News
- 473 Onboarding
- 573 日本支部