Precision Loss Redshift Data Flow

Reply
White Belt

Precision Loss Redshift Data Flow

I am encountering a weird behaviour on using SQL SUM method on decimal values with DOMO Data Flow.

 

Expected result based on query from MySql:

Screenshot 2020-06-05 at 11.06.10.png                    Screenshot 2020-06-05 at 11.12.48.png

Result in DOMO after Redshift Data Flow completes:

Screenshot 2020-06-05 at 11.07.51.png            Screenshot 2020-06-05 at 11.12.58.png

Issue is that precision is lost in this case and final result will be different in doing SUM on this values:

MySql => 100

DOMO => 99.99.

What can I do in this case to fix that part and to obtain same result as MySql?

Black Belt

Hi @user037867 

 

You could utilize a ROUND(number, <decimal places>) function in a mysql dataflow to force it to do the rounding to the precision you want instead of having to truncate your numbers on input. If the percentage is calculated in your initial query you can use ROUND there instead of a MySQL data flow as well.

 

The other option if you must use magic ETL would be to use a calculator tile to multiply your percentage by 100 (shifting the decimal point over 2, use 1000 for 3 decimals of precision etc), set the data type to be a whole number (forces rounding) -> set the data type back to decimal -> calculator to divide by your original number (100 / 1000 etc).



**Was this post helpful? Click the heart icon**

**Did this solve your problem? Accept it as a solution!**
Black Belt

if possible, i would recommend against intentionally rounding metrics in ETL, keep the data as raw/accurate as possible.

 

Also, try downloading a sample of your data, the UI sometimes hides the precision of the data.


Jae Wilson
Check out my Domo Training YouTube Channel

**Say "Thanks" by clicking the heart in the post that helped you.
**Please mark the post that solves your problem by clicking on "Accept as Solution"
Yellow Belt

If you are using Redshift, I was able to fix this using "::decimal(16,8)" the first number is the precision and the second one is the scale or how many numbers right of the decimal. 

Announcements
New & Improved Dojo: Get excited for a new feel and additional features in the online community.

Site will be down February 4-5th for maintenance.

Launch coming February 8th!