Copying dataset pdp settings

Reply
Highlighted
Yellow Belt

Copying dataset pdp settings

Is there a way to copy pdp settings between datasets?


Accepted Solutions
Highlighted
Major Blue Belt
Major Blue Belt

Re: Copying dataset pdp settings

I scripted it using PowerShell and the API... All you need to add is the access token and the dataset ids. there are comments in the code from past projects and it is not perfect but it worked for our purpose. We have thousands of PDP rules and the excel pugin bogs down. There is also a PDP utility but you have to create a dataset of PDP rules... This was easier.

 


function Get-Dataset($datasetId, $Authentication) {
#Write-Host $Authentication
$headers = @{
Accept = 'text/csv'
Authorization = $Authentication.access_token
}
#Write-Host "https://api.domo.com/v1/datasets/$datasetId/data?includeHeader=true&fileName=dump.csv" -Headers $headers
try {
$csv = Invoke-RestMethod -Method Get -Uri "https://api.domo.com/v1/datasets/$datasetId/data?includeHeader=true&fileName=dump.csv" -Headers $headers
$csv = $csv -replace '\\N', $null
$data = ConvertFrom-Csv $csv
#Write-Host "must have Successfully Read CSV File"
}
catch {
Write-Host "Failed to Read Dataset CSV File"
Write-Host $_.ErrorDetails
Write-Host $_.Exception
Write-Host $_.FullyQualifiedErrorId
Start-Sleep -Seconds 10
}
return $data
}

function Get-Dataset-Object($datasetId, $Authentication) {
#Write-Host $Authentication
$NeedData = $true
While( $NeedData -eq $true -and $Cnct_Cntr -lt 100){
try {
$headers = @{
Accept = 'application/json'
Authorization = $Authentication.access_token
}
#Write-Host "https://api.domo.com/v1/datasets/$datasetId" -Headers $headers
$data = Invoke-RestMethod -Method Get -Uri "https://api.domo.com/v1/datasets/$datasetId" -Headers $headers
#Write-Host "must have Successfully Read CSV File"
$Cnct_Cntr++
$NeedData = $false
}
catch {
Write-Host "Failed to get dataset object"
Write-Host $_.ErrorDetails
Write-Host $_.Exception
Write-Host $_.FullyQualifiedErrorId
$Cnct_Cntr++
Write-Host $Cnct_Cntr
Start-Sleep -Seconds 10
}
}
return $data
}

function Get-Dataset-PDP($datasetId, $Authentication) {
#Write-Host $Authentication
$NeedData = $true
While( $NeedData -eq $true -and $Cnct_Cntr -lt 100){
try {
$headers = @{
Accept = 'application/json'
Authorization = $Authentication.access_token
}
#Write-Host "https://api.domo.com/v1/datasets/$datasetId" -Headers $headers
$data = Invoke-RestMethod -Method Get -Uri "https://api.domo.com/v1/datasets/$datasetId/policies" -Headers $headers
#Write-Host "must have Successfully Read CSV File"
$Cnct_Cntr++
$NeedData = $false
}
catch {
Write-Host "Failed to get dataset object"
Write-Host $_.ErrorDetails
Write-Host $_.Exception
Write-Host $_.FullyQualifiedErrorId
$Cnct_Cntr++
Write-Host $Cnct_Cntr
Start-Sleep -Seconds 10
}
}
return $data
}

function Get-Authorization($clientID, $clientSecret) {
$basicAuth = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(("{0}:{1}" -f $clientID, $clientSecret)));
$authenticationHeader = @{
Authorization = ("Basic {0}" -f $basicAuth)
}
$NeedToken = $true
$Cnct_Cntr = 0

#keep trying to get a token
While( $NeedToken -eq $true -and $Cnct_Cntr -lt 100){
try {
$data = Invoke-RestMethod -Method Post -Uri "https://api.domo.com/oauth/token?grant_type=client_credentials&scope=data%20user" -Headers $authenticationHeader
# $data = Invoke-RestMethod -Method Post -Uri "https://api.domo.com/oauth/token?grant_type=client_credentials&scope=full" -Headers $authenticationHeader
$data.access_token = "bearer $($data.access_token)"
# Write-Host "Got Token"
$Cnct_Cntr++
$NeedToken = $false
}
catch {
#$ErrorActionPreference = Stop #this does not recognize "Stop"
Write-Host "Failed to get Token"
Write-Host $_.ErrorDetails
Write-Host $_.Exception
Write-Host $_.FullyQualifiedErrorId
$Cnct_Cntr++
Write-Host $Cnct_Cntr
Start-Sleep -Seconds 10
}
}
return $data
}

function Send-ToEmail([string]$email, [string]$body){

$message = new-object Net.Mail.MailMessage;
$message.From = "Domo@daltile.com";
$message.To.Add($email);
$message.Subject = "Error Running Vistex jobs Script";
$message.Body = $body;
$smtp = new-object Net.Mail.SmtpClient("smtpmail.mohawkind.com");
$smtp.send($message);
write-host "Mail Sent" ;
}


Write-Host "go get auth token"
$Authentication = Get-Authorization -clientID $apiKey -clientSecret $apiToken
Write-Host "have token"
#Write-Host $Authentication
#$MetData = Get-Dataset -datasetId $datasetmetadata -Authentication $Authentication

$output = ''
$haserr = $true
$Have_PDP = $false
#get the list of WB jobs
while(!$Have_PDP){
try{
$Authentication = Get-Authorization -clientID $apiKey -clientSecret $apiToken
$DS_PDP = Get-Dataset-PDP -datasetId $PDPSourceDatasetID -Authentication $Authentication
$Have_PDP = $true
}
catch{
Write-Host "Failed to get dataset list of PDP rules"
Write-Host $_.ErrorDetails
Write-Host $_.Exception
Write-Host $_.FullyQualifiedErrorId
Start-Sleep -Seconds 10
}
}
# Cycle through the list of jobs ensuring they run successfully.

 


$PDPs = $DS_PDP.Where{$_."name" -like "z_rep_*" -or $_."name" -like "z_ssc_*" } #| Sort-Object -Property $_."Execution Order"| Sort-Object -Property $_."Execution Order"
foreach($PDP in $PDPs){
$haserr = $true
$output = ''
$msg = ''
$Count = 0

Try{
$hdrs = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
$hdrs.Add("Content-Type", "application/json")
$hdrs.Add("Accept", 'application/json')
$hdrs.Add("Authorization", $Authentication.access_token)
$filter = $PDP.filters
$name = $PDP.name
$type = $PDP.type
$users = $PDP.users
$groups = $PDP.groups


$i = 0
$pdp_rule = '{ "name": "'+$name+'",'
$fString = '"filters": ['
foreach($f in $PDP.filters){
$fString += '{ "column": "'+$f.column+'", "values": [ "'+$f.values+'" ], "operator": "'+$f.operator+'" }'
$i++
if($i -lt $PDP.filters.count){$fString += ','}
}

$pdp_rule += $fString +' ], "groups": [ "'+$groups+'" ] }'
# {
# "name": "z_ssc_4000",
# "filters": [ {
# "column": "business_area_number",
# "values": [ "4000" ],
# "operator": "EQUALS"
# } ],
# "groups": [ 612003576 ]
#}


$post = Invoke-WebRequest -Method POST -Uri "https://api.domo.com/v1/datasets/$DestinationDatasetId/policies" -UseBasicParsing -Body $pdp_rule -Headers $hdrs -ContentType 'application/json'

Write-Host "updated $name in Proex dataset PDP "
}

catch {
Write-Host "Failed to add $email to Domo"
Write-Host $_.ErrorDetails
Write-Host $_.Exception
Write-Host $_.FullyQualifiedErrorId

}

}#end foreach job


$now = Get-Date -Format g
Write-Host "$now Done."

View solution in original post

Tags (2)

All Replies
Highlighted
Black Belt
Black Belt

Re: Copying dataset pdp settings

There isn't a process currently place through the interface that lets you do this.  I think there's a post on the Ideas Exchange for it, though.  Ideally, I think, there would be a list of PDPs stored independent of datasets, and you could apply a group of policies to datasets at the click of a button.

 

That said, we're using a not-so-official software application that Domo skunkworks wrote that lets us apply policies across different datasets using a config file hosted in Domo via a webform dataset.  It's saved us a ton of time and serves the same purpose.  You should inquire with your success manager or rep about that.

Aaron
MajorDomo @ Merit Medical

**Say "Thanks" by clicking the heart in the post that helped you.
**Please mark the post that solves your problem by clicking on "Accept as Solution"
Highlighted
White Belt

Re: Copying dataset pdp settings

I was trying to search for this ap but couldn't. Is there anyway that I can access and use it to copy over PDP to differnt data set?

Highlighted
Black Belt
Black Belt

Re: Copying dataset pdp settings

I'm sure this application isn't available to download.  You'll have to ask Domo directly.

I'd also watch out in the coming months to see if something comes into the product.  I wouldn't be surprised to see something at Domopalooza, for example.

Aaron
MajorDomo @ Merit Medical

**Say "Thanks" by clicking the heart in the post that helped you.
**Please mark the post that solves your problem by clicking on "Accept as Solution"
Highlighted
Yellow Belt

Re: Copying dataset pdp settings

I'm revisiting this because I've continued to builed datasets that have the exact same PDP settings as previous datasets, and I still don't see a way to copy the settings from one dataset to another.  

 

Does this exist or do I have to continue recreating the wheel on every dataset?

Highlighted
Green Belt

Re: Copying dataset pdp settings

Have you tried the Excel plugin for DOMO, this allows you to apply PDP security to any dataset you want and is a fast and easy way to apply PDP policies that are exact copies to other datasets.

Just change the DatasetID and apply and your good.

This is also very easy to make slight modifications to PDP's that are close but not the same.

It is also an easy way to manage other components such as user accounts etc.

Randy

Highlighted
Yellow Belt

Re: Copying dataset pdp settings

I will request the admin version of that.

 

Thanks.

Highlighted
Green Belt

Re: Copying dataset pdp settings

You should be able to download it from the Admin/Tools Download section.

Randy

Highlighted
Yellow Belt

Re: Copying dataset pdp settings

The admin one requires a special request, but I did get.  It's not quite the elegant solution I was hoping for within DOMO, but it will work for when I have to add bulk permissions.  

 

Highlighted
Major Blue Belt
Major Blue Belt

Re: Copying dataset pdp settings

I scripted it using PowerShell and the API... All you need to add is the access token and the dataset ids. there are comments in the code from past projects and it is not perfect but it worked for our purpose. We have thousands of PDP rules and the excel pugin bogs down. There is also a PDP utility but you have to create a dataset of PDP rules... This was easier.

 


function Get-Dataset($datasetId, $Authentication) {
#Write-Host $Authentication
$headers = @{
Accept = 'text/csv'
Authorization = $Authentication.access_token
}
#Write-Host "https://api.domo.com/v1/datasets/$datasetId/data?includeHeader=true&fileName=dump.csv" -Headers $headers
try {
$csv = Invoke-RestMethod -Method Get -Uri "https://api.domo.com/v1/datasets/$datasetId/data?includeHeader=true&fileName=dump.csv" -Headers $headers
$csv = $csv -replace '\\N', $null
$data = ConvertFrom-Csv $csv
#Write-Host "must have Successfully Read CSV File"
}
catch {
Write-Host "Failed to Read Dataset CSV File"
Write-Host $_.ErrorDetails
Write-Host $_.Exception
Write-Host $_.FullyQualifiedErrorId
Start-Sleep -Seconds 10
}
return $data
}

function Get-Dataset-Object($datasetId, $Authentication) {
#Write-Host $Authentication
$NeedData = $true
While( $NeedData -eq $true -and $Cnct_Cntr -lt 100){
try {
$headers = @{
Accept = 'application/json'
Authorization = $Authentication.access_token
}
#Write-Host "https://api.domo.com/v1/datasets/$datasetId" -Headers $headers
$data = Invoke-RestMethod -Method Get -Uri "https://api.domo.com/v1/datasets/$datasetId" -Headers $headers
#Write-Host "must have Successfully Read CSV File"
$Cnct_Cntr++
$NeedData = $false
}
catch {
Write-Host "Failed to get dataset object"
Write-Host $_.ErrorDetails
Write-Host $_.Exception
Write-Host $_.FullyQualifiedErrorId
$Cnct_Cntr++
Write-Host $Cnct_Cntr
Start-Sleep -Seconds 10
}
}
return $data
}

function Get-Dataset-PDP($datasetId, $Authentication) {
#Write-Host $Authentication
$NeedData = $true
While( $NeedData -eq $true -and $Cnct_Cntr -lt 100){
try {
$headers = @{
Accept = 'application/json'
Authorization = $Authentication.access_token
}
#Write-Host "https://api.domo.com/v1/datasets/$datasetId" -Headers $headers
$data = Invoke-RestMethod -Method Get -Uri "https://api.domo.com/v1/datasets/$datasetId/policies" -Headers $headers
#Write-Host "must have Successfully Read CSV File"
$Cnct_Cntr++
$NeedData = $false
}
catch {
Write-Host "Failed to get dataset object"
Write-Host $_.ErrorDetails
Write-Host $_.Exception
Write-Host $_.FullyQualifiedErrorId
$Cnct_Cntr++
Write-Host $Cnct_Cntr
Start-Sleep -Seconds 10
}
}
return $data
}

function Get-Authorization($clientID, $clientSecret) {
$basicAuth = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(("{0}:{1}" -f $clientID, $clientSecret)));
$authenticationHeader = @{
Authorization = ("Basic {0}" -f $basicAuth)
}
$NeedToken = $true
$Cnct_Cntr = 0

#keep trying to get a token
While( $NeedToken -eq $true -and $Cnct_Cntr -lt 100){
try {
$data = Invoke-RestMethod -Method Post -Uri "https://api.domo.com/oauth/token?grant_type=client_credentials&scope=data%20user" -Headers $authenticationHeader
# $data = Invoke-RestMethod -Method Post -Uri "https://api.domo.com/oauth/token?grant_type=client_credentials&scope=full" -Headers $authenticationHeader
$data.access_token = "bearer $($data.access_token)"
# Write-Host "Got Token"
$Cnct_Cntr++
$NeedToken = $false
}
catch {
#$ErrorActionPreference = Stop #this does not recognize "Stop"
Write-Host "Failed to get Token"
Write-Host $_.ErrorDetails
Write-Host $_.Exception
Write-Host $_.FullyQualifiedErrorId
$Cnct_Cntr++
Write-Host $Cnct_Cntr
Start-Sleep -Seconds 10
}
}
return $data
}

function Send-ToEmail([string]$email, [string]$body){

$message = new-object Net.Mail.MailMessage;
$message.From = "Domo@daltile.com";
$message.To.Add($email);
$message.Subject = "Error Running Vistex jobs Script";
$message.Body = $body;
$smtp = new-object Net.Mail.SmtpClient("smtpmail.mohawkind.com");
$smtp.send($message);
write-host "Mail Sent" ;
}


Write-Host "go get auth token"
$Authentication = Get-Authorization -clientID $apiKey -clientSecret $apiToken
Write-Host "have token"
#Write-Host $Authentication
#$MetData = Get-Dataset -datasetId $datasetmetadata -Authentication $Authentication

$output = ''
$haserr = $true
$Have_PDP = $false
#get the list of WB jobs
while(!$Have_PDP){
try{
$Authentication = Get-Authorization -clientID $apiKey -clientSecret $apiToken
$DS_PDP = Get-Dataset-PDP -datasetId $PDPSourceDatasetID -Authentication $Authentication
$Have_PDP = $true
}
catch{
Write-Host "Failed to get dataset list of PDP rules"
Write-Host $_.ErrorDetails
Write-Host $_.Exception
Write-Host $_.FullyQualifiedErrorId
Start-Sleep -Seconds 10
}
}
# Cycle through the list of jobs ensuring they run successfully.

 


$PDPs = $DS_PDP.Where{$_."name" -like "z_rep_*" -or $_."name" -like "z_ssc_*" } #| Sort-Object -Property $_."Execution Order"| Sort-Object -Property $_."Execution Order"
foreach($PDP in $PDPs){
$haserr = $true
$output = ''
$msg = ''
$Count = 0

Try{
$hdrs = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
$hdrs.Add("Content-Type", "application/json")
$hdrs.Add("Accept", 'application/json')
$hdrs.Add("Authorization", $Authentication.access_token)
$filter = $PDP.filters
$name = $PDP.name
$type = $PDP.type
$users = $PDP.users
$groups = $PDP.groups


$i = 0
$pdp_rule = '{ "name": "'+$name+'",'
$fString = '"filters": ['
foreach($f in $PDP.filters){
$fString += '{ "column": "'+$f.column+'", "values": [ "'+$f.values+'" ], "operator": "'+$f.operator+'" }'
$i++
if($i -lt $PDP.filters.count){$fString += ','}
}

$pdp_rule += $fString +' ], "groups": [ "'+$groups+'" ] }'
# {
# "name": "z_ssc_4000",
# "filters": [ {
# "column": "business_area_number",
# "values": [ "4000" ],
# "operator": "EQUALS"
# } ],
# "groups": [ 612003576 ]
#}


$post = Invoke-WebRequest -Method POST -Uri "https://api.domo.com/v1/datasets/$DestinationDatasetId/policies" -UseBasicParsing -Body $pdp_rule -Headers $hdrs -ContentType 'application/json'

Write-Host "updated $name in Proex dataset PDP "
}

catch {
Write-Host "Failed to add $email to Domo"
Write-Host $_.ErrorDetails
Write-Host $_.Exception
Write-Host $_.FullyQualifiedErrorId

}

}#end foreach job


$now = Get-Date -Format g
Write-Host "$now Done."

View solution in original post

Tags (2)
Announcements
Coming Soon: Weekly Virtual User Meet-ups to answer all your questions! Stay tuned for more details to come this week.