HomeBig DataUse Amazon SageMaker {custom} tags for challenge useful resource governance and value...

Use Amazon SageMaker {custom} tags for challenge useful resource governance and value monitoring


Amazon SageMaker introduced a brand new characteristic that you need to use so as to add {custom} tags to assets created by way of an Amazon SageMaker Unified Studio challenge. This helps you implement tagging requirements that conform to your group’s service management insurance policies (SCPs) and helps allow price monitoring reporting practices on assets created throughout the group.

As a SageMaker administrator, you possibly can configure a challenge profile with tag configurations that might be pushed right down to initiatives that presently use or will use that challenge profile. The challenge profile is ready as much as go both required key and worth tag pairings or go the important thing of the tag with a default worth that may be modified throughout challenge creation. All tags handed to the challenge will consequence within the assets created by that challenge being tagged. This gives you with a governance mechanism that enforces that challenge assets have the anticipated tags throughout all initiatives of the area.

The primary launch of {custom} tags for challenge assets is supported by way of an utility programming interface (API), by way of Amazon DataZone SDKs. On this submit, we take a look at use circumstances for {custom} tags and find out how to use the AWS Command Line Interface (AWS CLI) so as to add tags to challenge assets.

What we hear from prospects

As prospects proceed to construct and collaborate utilizing AWS instruments for mannequin improvement, generative AI, information processing, and SQL analytics, they see the necessity to carry management and visibility into the assets being created. To assist connectivity to those AWS instruments from SageMaker Unified Studio initiatives, many various kinds of assets throughout AWS companies must be created. These assets are created by way of AWS CloudFormation stacks (by way of challenge surroundings deployment) by the Amazon SageMaker service. From prospects we hear the next use circumstances:

  • Clients must implement that tagging practices conform to firm insurance policies by way of using AWS controls, equivalent to SCPs, for useful resource creation. These controls block the creation of assets except particular tags are positioned on the useful resource.
  • Clients may begin with insurance policies to implement that the right tags are positioned when assets are created with the extra purpose of standardizing on useful resource reporting. By inserting identifiable info on assets when created, they implement consistency and completeness when performing price attribution reporting and observability.

Buyer Swiss Life makes use of SageMaker as a single answer for cataloging, discovery, sharing, and governance of their enterprise information throughout enterprise domains. They require all assets have a set of necessary tags for his or her finance group to invoice organizations throughout their firm for the AWS assets created.

“The launch of challenge useful resource tags for Amazon SageMaker permits us to carry visibility to the prices incurred throughout our accounts. With this functionality we’re in a position to meet the useful resource tagging tips of our firm and believe in attributing prices throughout our multi-account setup for the assets created by Amazon SageMaker initiatives.”

– Tim Kopacz, Software program Developer at Swiss Life

Conditions

To get began with {custom} tags, you have to have the next assets:

  • A SageMaker Unified Studio area.
  • An AWS Identification and Entry Administration (IAM) entity with privileges to make AWS CLI calls to the area.
  • An IAM entity licensed to make adjustments to the area IAM provisioning function. If SageMaker created this for you, it is going to be known as AmazonSageMakerProvisioning-. The provisioning function provisions and manages assets outlined within the chosen blueprints in your account.

The right way to arrange challenge useful resource tags

The next steps define how one can configure {custom} tags to your SageMaker Unified Studio challenge assets:

  1. (Elective) Replace the SageMaker provisioning function to allow particular tag keys.
  2. Create a brand new challenge profile with challenge useful resource tags configured.
  3. Create a brand new challenge with challenge useful resource tags.
  4. Replace an present challenge with challenge useful resource tags.
  5. Validate that the assets are tagged.

(Elective) Replace a SageMaker provisioning function to allow tag key values

The AmazonSageMakerProvisioning- function has an AWS managed coverage with situation aws:TagKeys permitting tags to be created by this function provided that the tag key begins with AmazonDataZone. For this instance, we are going to change the tag key to start with completely different strings. Skip to Create a brand new challenge profile with challenge useful resource tags configured when you don’t want tag keys to have a distinct construction (equivalent to begins with, incorporates, and so forth)

  1. Open the AWS Administration Console and go to IAM.
  2. Within the navigation pane, select Roles.
  3. Within the checklist, select AmazonSageMakerProvisioning-.
  4. Select the Permissions tab.
  5. Select Add permissions, after which select Create inline coverage.
  6. Below Coverage editor, choose JSON.
  7. Enter the next coverage. Add the strings beneath the situation aws:TagKeys. On this instance, tag keys starting with ACME or tag keys with the precise match of CostCenter might be created by the function.
    {
        "Model": "2012-10-17",
        "Assertion": [
            {
                "Sid": "CustomTagsUnTagPermissions",
                "Effect": "Allow",
                "Action": [
                    "codecommit:UntagResource",
                    "iam:UntagRole",
                    "logs:UntagResource",
                    "athena:UntagResource",
                    "redshift-serverless:UntagResource",
                    "scheduler:UntagResource",
                    "bedrock:UntagResource",
                    "neptune-graph:UntagResource",
                    "quicksight:UntagResource",
                    "glue:UntagResource",
                    "airflow:UntagResource",
                    "secretsmanager:UntagResource",
                    "lambda:UntagResource",
                    "emr-serverless:UntagResource",
                    "elasticmapreduce:RemoveTags",
                    "sagemaker:DeleteTags",
                    "ec2:DeleteTags"
                ],
                "Useful resource": "*",
                "Situation": {
                    "StringEquals": {
                        "aws:ResourceAccount": "${aws:PrincipalAccount}"
                    },
                    "ForAllValues:StringLike": {
                        "aws:TagKeys": [
                            "AmazonDataZone*",
                            "ACME*",
                            "CostCenter"
                        ]
                    },
                    "Null": {
                        "aws:ResourceTag/AmazonDataZoneProject": "false"
                    }
                }
            },
            {
                "Sid": "CustomTagsTaggingPermissions",
                "Impact": "Permit",
                "Motion": [
                    "cloudformation:TagResource",
                    "codecommit:TagResource",
                    "iam:TagRole",
                    "glue:TagResource",
                    "athena:TagResource",
                    "lambda:TagResource",
                    "redshift-serverless:TagResource",
                    "logs:TagResource",
                    "secretsmanager:TagResource",
                    "sagemaker:AddTags",
                    "emr-serverless:TagResource",
                    "neptune-graph:TagResource",
                    "bedrock:TagResource",
                    "elasticmapreduce:AddTags",
                    "airflow:TagResource",
                    "scheduler:TagResource",
                    "quicksight:TagResource",
                    "emr-containers:TagResource",
                    "logs:CreateLogGroup",
                    "athena:CreateWorkGroup",
                    "scheduler:CreateScheduleGroup",
                    "cloudformation:CreateStack",
                    "ec2:*"
                ],
                "Useful resource": "*",
                "Situation": {
                    "ForAnyValue:StringLike": {
                        "aws:TagKeys": [
                            "AmazonDataZone*",
                            "ACME*",
                            "CostCenter"
                        ]
                    },
                    "StringEquals": {
                        "aws:ResourceAccount": "${aws:PrincipalAccount}"
                    }
                }
            }
        ]
    }

It’s doable to scope down the particular AWS service tag and un-tag permissions primarily based on which blueprints or capabilities are getting used.

Create a brand new challenge profile with challenge useful resource tags configured

Use the next steps to create a brand new SQL Analytics challenge profile with {custom} tags. The instance makes use of AWS CLI instructions.

  1. Open the AWS CloudShell console.
  2. Create a challenge profile utilizing the next CLI command.
    1. The project-resource-tags parameter consists of key (tag key), worth (tag worth), and isValueEditable (boolean indicating if the tag worth may be modified throughout challenge creation or replace).
    2. The allow-custom-project-resource-tags parameter set to true permits the challenge creator to create extra key-value pairs. The important thing wants to adapt to the inline coverage of the AmazonSageMakerProvisioning- function.
    3. The project-resource-tags-description parameter is an outline subject for challenge useful resource tags. The max character restrict is 2,048. The outline must be handed in each time create-project-profile or update-project-profile is named.
    aws datazone create-project-profile 
      --name "SQL Analytics with Venture Useful resource Tags" 
      --description "Analyze your information in SageMaker Lakehouse utilizing SQL" 
      --domain-identifier "$DOMAIN_ID" 
      --region "$REGION" 
      --status ENABLED 
      --project-resource-tags '[
        {
            "key": "ACME-Application",
            "value": "SageMaker",
            "isValueEditable": false
        },
        {
            "key": "CostCenter",
            "value": "123",
            "isValueEditable": true
        }
      ]' 
      --allow-custom-project-resource-tags 
      --environment-configurations '[
        {
            "name": "Tooling",
            "description": "Configuration for the Tooling Environment",
            "environmentBlueprintId": "",
            "deploymentMode": "ON_CREATE",
            "deploymentOrder": 0,
            "awsAccount": {
            "awsAccountId": "$ACCOUNT"
        },
        "awsRegion": {
            "regionName": "$REGION"
        },
            "configurationParameters": {
                "parameterOverrides": [
                    {
                        "name": "enableSpaces",
                        "value": "false",
                        "isEditable": false
                    },
                    {
                        "name": "maxEbsVolumeSize",
                        "isEditable": false
                    },
                    {
                        "name": "idleTimeoutInMinutes",
                        "isEditable": false
                    },
                    {
                        "name": "lifecycleManagement",
                        "isEditable": false
                    },
                    {
                        "name": "enableNetworkIsolation",
                        "isEditable": false
                    }
                ]
            }
        },
        {
            "title": "Lakehouse Database",
            "description": "Creates databases in Amazon SageMaker Lakehouse for storing tables in S3 and Amazon Athena assets to your SQL workloads",
            "environmentBlueprintId": "",
            "deploymentMode": "ON_CREATE",
            "deploymentOrder": 1,
            "awsAccount": {
                "awsAccountId": "$ACCOUNT"
            },
            "awsRegion": {
            "regionName": "$REGION"
            },
            "configurationParameters": {
                "parameterOverrides": [
                    {
                        "name": "glueDbName",
                        "value": "glue_db",
                        "isEditable": true
                    }
                ]
            }
        },
        {
            "title": "OnDemand RedshiftServerless",
            "description": "Allows you to create an extra Amazon Redshift Serverless workgroup to your SQL workloads",
            "environmentBlueprintId": "",
            "deploymentMode": "ON_DEMAND",
            "awsAccount": {
            "awsAccountId": "$ACCOUNT"
            },
            "awsRegion": {
                "regionName": "$REGION"
            },
            "configurationParameters": {
                "parameterOverrides": [
                    {
                        "name": "redshiftDbName",
                        "value": "dev",
                        "isEditable": true
                        },
                        {
                        "name": "redshiftMaxCapacity",
                        "value": "512",
                        "isEditable": true
                        },
                        {
                        "name": "redshiftWorkgroupName",
                        "value": "redshift-serverless-workgroup",
                        "isEditable": true
                        },
                        {
                        "name": "redshiftBaseCapacity",
                        "value": "128",
                        "isEditable": true
                        },
                        {
                        "name": "connectionName",
                        "value": "redshift.serverless",
                        "isEditable": true
                        },
                        {
                        "name": "connectToRMSCatalog",
                        "value": "false",
                        "isEditable": false
                        }
                    ]
                }
            },
            {
                "title": "OnDemand Catalog for Redshift Managed Storage",
                "description": "Allows you to create extra catalogs in Amazon SageMaker Lakehouse for storing information in Redshift Managed Storage",
                "environmentBlueprintId": "",
                "deploymentMode": "ON_DEMAND",
                "awsAccount": {
                "awsAccountId": "$ACCOUNT"
                },
                "awsRegion": {
                    "regionName": "$REGION"
                },
                "configurationParameters": {
                    "parameterOverrides": [
                        {
                            "name": "catalogName",
                            "isEditable": true
                        },
                        {
                            "name": "catalogDescription",
                            "value": "RMS catalog",
                            "isEditable": true
                        }
                    ]
                }
            }
      ]'

This challenge profile can have the tag ACME-Software = SageMaker positioned on all initiatives related to the challenge profile and can’t be modified by the challenge creator. The tag CostCenter = 123 can have the worth modified by the challenge creator as a result of the isValueEditable property is ready to true.

Grant permissions for customers to make use of the challenge profile throughout challenge creation. Within the Authorization part of the challenge profile set both Chosen customers or teams or Permit all customers and teams.

The usage of the allow-custom-project-resource-tags parameter means the challenge creator can add their very own tags (key-value pair). The important thing should conform to the situation examine within the coverage of the provisioning function (AmazonSageMakerProvisioning-). If the allow-custom-project-resource-tagsparameter is modified to false after a challenge created tags, tags created by the challenge might be eliminated throughout the subsequent challenge replace.

Updates to the challenge profile

Updates to challenge useful resource tags are doable by way of the update-project-profile command. The command will substitute all values within the project-resource-tags part so remember to embrace the exhaustive set of tags. Updates to the challenge profile are mirrored in initiatives after operating the update-project command or when a brand new challenge is created utilizing the challenge profile. The next instance provides a brand new tag, ACME-BusinessUnit = Retail.

There are 3 ways to work with the project-resource-tags parameter when updating the challenge profile.

  • Passing a non-empty checklist of challenge useful resource tags will substitute the tags presently configured on the challenge profile.
  • Passing an empty checklist of challenge useful resource tags will filter out all beforehand configured tags:
    • --project-resource-tags '[]'
  • Not together with the challenge useful resource tag parameter will preserve beforehand configured tags as-is.
aws datazone update-project-profile 
  --domain-identifier "$DOMAIN_ID" 
  --identifier "$PROJECT_PROFILE_ID" 
  --region "$REGION" 
  --project-resource-tags '[
    {
        "key": "ACME-Application",
        "value": "SageMaker",
        "isValueEditable": false
    },
    {
        "key": "CostCenter",
        "value": "123",
        "isValueEditable": true
    },
    {
        "key": "ACME-BusinessUnit",
        "value": "Retail",
        "isValueEditable": false
    }
  ]'

Create a brand new challenge with challenge useful resource tags

The next steps stroll you thru creating a brand new challenge that inherits tags from the challenge profile and lets the challenge creator modify one of many tag values.

  1. Create a challenge utilizing the next instance CLI command.
  2. Modify the CostCenter tag worth utilizing the --resource-tags parameter. Tags configured on the challenge profile the place the isValueEditable attribute is false might be pushed to the challenge robotically.
    aws datazone create-project 
      --domain-identifier "$DOMAIN_ID" 
      --region "$REGION" 
      --name "$PROJECT_NAME" 
      --description "New challenge with tags" 
      --project-profile-id "$PROJECT_PROFILE_ID" 
      --resource-tags '{
            "CostCenter": "456"
        }'

Replace present challenge with challenge useful resource tags

For present initiatives related to the challenge profile, you have to replace the challenge for the brand new tags to be utilized.

  1. Replace the challenge utilizing the next instance CLI command.
  2. On this situation, an editable worth must be up to date and a brand new tag added. Tag CostCenter can have its default worth overwritten as “789” and the brand new ACME-Division = Finance tag might be added.
    aws datazone update-project 
      --domain-identifier "$DOMAIN_ID" 
      --identifier "$PROJECT_ID" 
      --project-profile-version "newest" 
      --region "$REGION" 
      --resource-tags '{
            "CostCenter": "789",
            "ACME-Division": "Finance"
        }' 

Venture stage tags (these not configured from the challenge profile) must be handed throughout challenge replace to be preserved. For tags with isValueEditable = true configured from the challenge profile, any override beforehand set must be utilized or the worth will revert to the default from the challenge profile.

Validating assets are tagged

Validate that tags are positioned appropriately. An instance useful resource that’s created by the challenge is the challenge IAM function. Viewing the tags for this function ought to present the tags configured from the challenge profile.

  1. Open SageMaker Unified Studio to get the challenge function from the Venture particulars part of the challenge. The function title begins with datazone_usr_role_.
  2. Open the IAM console.
  3. Within the navigation pane, select Roles.
  4. Seek for the challenge IAM function.
  5. Choose the Tags tab.

Conclusion

On this submit, we mentioned tagging associated use circumstances from prospects and walked by way of getting began with {custom} tags in Amazon SageMaker to position tags on the assets created by the challenge. By giving directors a option to configure challenge profiles with standardized tag configurations, now you can assist guarantee constant tagging practices throughout all SageMaker Unified Studio initiatives whereas sustaining compliance with SCPs. This characteristic addresses two important buyer wants: implementing organizational tagging requirements by way of automated governance mechanisms and enabling correct price attribution reporting throughout multi-service deployments.

To study extra, go to Amazon SageMaker, then get began with Venture useful resource tags.


Concerning the authors

David Victoria

David Victoria

David is a Senior Technical Product Supervisor with Amazon SageMaker at AWS. He focuses on bettering administration and governance capabilities wanted for patrons to assist their analytics techniques. He’s keen about serving to prospects notice probably the most worth from their information in a safe, ruled method.

Rohit Srikanta

Rohit Srikanta

Rohit is a Senior Software program Engineer at AWS. He works on constructing and scaling companies inside Amazon SageMaker. He focuses on growing sturdy and scalable distributed techniques and is keen about fixing complicated engineering challenges to ship most buyer worth.

Ahan Malli

Ahan Malli

Ahan is a Software program Growth Engineer at AWS. He works on the core information and governance layer behind Amazon SageMaker. He’s keen about constructing scalable distributed techniques and streamlining developer workflows. When he’s not coding, you’ll find him touring or mountain climbing Pacific Northwest trails.

RELATED ARTICLES

LEAVE A REPLY

Please enter your comment!
Please enter your name here

- Advertisment -
Google search engine

Most Popular

Recent Comments