Responsys: Audiences as PETs mapping

[seg-leonelsanches]

Actions Responsys today tries to work with existing PETs. The problem with this is: there’s no way to automatically modify a PET to include new columns when a corresponding audience is created. The actual endpoints only allow us to create new PETs.

The idea is a new Action that creates a new PET for each new audience before sending records to Responsys, also validating if users are already part of the same Profile List the PET belongs. This approach is already used with our customers successfully, but implemented as a Destination Function.

This implementation also follows the cadence that Responsys expects, with one event at a time, waiting for each API response, instead of parallel Promises.

Testing

Payload suggestion for /refreshAccessToken:

{
    "baseUrl": "https://cj01qwy-api.responsys.ocs.oraclecloud.com",
    "username": "my responsys username",
    "userPassword": "my password",
    "profileListName": "PROFILE_LIST_NAME",
    "insertOnNoMatch": true,
    "matchColumnName1": "EMAIL_ADDRESS",
    "updateOnMatch": "REPLACE_ALL",
    "defaultPermissionStatus": "OPTOUT"
}

Payload suggestion for /sendAudienceAsPet:

{
    "auth": {
        "accessToken": "Responsys token returned by /refreshAccessToken"
    },
    "settings": {
        "baseUrl": "https://cj01qwy-api.responsys.ocs.oraclecloud.com",
        "profileListName": "PROFILE_LIST_NAME",
        "matchColumnName1": "EMAIL_ADDRESS"
    },
    "payload": {
        "context": {
            "personas": {
                "computation_key": "my_audience_key"
            }
        },
        "traits": {
            "email": "[email protected]"
        },
        "userId": "1234567890"
    },
    "mapping": {
        "folder_name": "POC",
        "pet_name": {
            "@path": "$.context.personas.computation_key"
        },
        "computation_key": {
            "@path": "$.context.personas.computation_key"
        },
        "userData": {
            "EMAIL_ADDRESS_": {
                "@path": "$.traits.email"
            },
            "CUSTOMER_ID_": {
                "@path": "$.userId"
            }
        }
    }
}

• Added unit tests for new functionality
• Tested end-to-end using the local server
• [Segmenters] Tested in the staging environment

[joe-ayoub-segment]

hi @seg-leonelsanches - thanks for raising this PR.

Looks like this is for a new Action. Could you complete the form on this page please so that we can properly evaluate if a new Action is the right way to go please?

https://segment.atlassian.net/wiki/spaces/~62b2d761d337d0b7d09d6455/pages/2873032746/Professional+Services+PRs+on+the+Action+Destinations+repo

I'm not a Responsys SME so I'd like to loop in @mayur-pitale and @longstoryshort as they were involved with writing the Integration.

[seg-leonelsanches]

hi @seg-leonelsanches - thanks for raising this PR.

Looks like this is for a new Action. Could you complete the form on this page please so that we can properly evaluate if a new Action is the right way to go please?

https://segment.atlassian.net/wiki/spaces/~62b2d761d337d0b7d09d6455/pages/2873032746/Professional+Services+PRs+on+the+Action+Destinations+repo

Hi @joe-ayoub-segment. You're welcome.

Here's the supporting document: https://segment.atlassian.net/wiki/spaces/~62b2d761d337d0b7d09d6455/pages/2874376194/Responsys+PET+per+Audience+review+form

[codecov]

Codecov Report

Attention: Patch coverage is 89.51613% with 13 lines in your changes missing coverage. Please review.

Project coverage is 78.09%. Comparing base (2574714) to head (e6e548e).
Report is 30 commits behind head on main.

Files with missing lines	Patch %	Lines
...tinations/responsys/sendAudienceAsPet/functions.ts	90.66%	0 Missing and 7 partials ⚠️
...nation-actions/src/destinations/responsys/index.ts	66.66%	0 Missing and 2 partials ⚠️
.../destinations/responsys/sendAudienceAsPet/index.ts	90.47%	1 Missing and 1 partial ⚠️
...nation-actions/src/destinations/responsys/utils.ts	89.47%	0 Missing and 2 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2398      +/-   ##
==========================================
+ Coverage   77.99%   78.09%   +0.10%     
==========================================
  Files         991      993       +2     
  Lines       17383    17482      +99     
  Branches     3281     3297      +16     
==========================================
+ Hits        13558    13653      +95     
+ Misses       2737     2734       -3     
- Partials     1088     1095       +7     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[joe-ayoub-segment]

Discussed with @seg-leonelsanches . Agreed for Leonel that he will fix the refreshToken code, then we'll deploy to Stage.

We need proof of testing before this can go to prod.

[seg-leonelsanches]

Discussed with @seg-leonelsanches . Agreed for Leonel that he will fix the refreshToken code, then we'll deploy to Stage.

We need proof of testing before this can go to prod.

@joe-ayoub-segment This can be deployed in Stage now.

[github-actions]

New required fields detected

Warning

Your PR adds new required fields to an existing destination. Adding new required settings/mappings for a destination already in production requires updating existing customer destination configuration. Ignore this warning if this PR is for a new destination with no active customers in production.

The following required fields were added in this PR:

• Destination: Responsys (Actions), Action Field(s):userData,folder_name,pet_name,computation_key,traits_or_props

Add these new fields as optional instead and assume default values in perform or performBatch block.

[joe-ayoub-segment]

OK thanks @seg-leonelsanches I'll do this now. Will notify when done.

[joe-ayoub-segment]

hi @seg-leonelsanches this is deployed to Staging now ...

edit: actually there is an issue. The push command didn't work - I need to go debug it.

[seg-leonelsanches]

Tested in Responsys successfully through Stage (https://app.segment.build/nick-test/destinations/actions-responsys/sources/personas_nick-test-dev/instances/67216bbb86a7770306d97b08/event-delivery?period=past-day):

This can be merged now.

[joe-ayoub-segment]

Thanks @seg-leonelsanches - I'll deploy this to Prod on Tuesday.

[joe-ayoub-segment]

hi @seg-leonelsanches my mistake - we can't deploy this week due to the election. Next scheduled deploy is next Tuesday.

[joe-ayoub-segment]

Hi @seg-leonelsanches PR deployed