# Configuring A/B Testing A/B testing helps marketers test treatments (for example, different marketing Calls to Action in an email) to determine which treatments are performing better. With Treasure Data's A/B Testing, you can compare one to eight versions of something to determine which performs better. To accomplish this test, at a minimum, you need one test case and a control group to validate the results. A control group is important because it provides a benchmark to measure performance against. A control group highlights what works as much as what doesn't. INFO The steps to configure A/B Testing apply only to draft journeys. This topic contains: - [Prerequisites](#prerequisites) - [Limitations](#limitations) - [Verify the Status of a Journey](#verify-the-status-of-a-journey) - [Create A/B Tests in a Journey](#create-ab-tests-in-a-journey) - [Modify an A/B Test](#modify-an-ab-test) - [Delete an A/B Test](#delete-an-ab-test) - [A/B Testing Statistics](#ab-testing-statistics) ## Prerequisites - A parent segment. - A new draft journey or an existing draft journey. - Basic understanding of Treasure Data. ## Limitations - Each stage of a journey can only have two layers of an A/B test added. ## Verify the Status of a Journey You can verify the current status of a draft journey to which you want to add or update A/B tests. The following example does not include an A/B test because we are going to add one in this tutorial. In the case the journey included an existing A/B test, the request retrieves the name, variant, and allocation information for the specified journey. This example uses the following values | | | | --- | --- | | Method | GET | | Endpoint | [ https://{BaseUrl}/entities/journeys/:journeyId ] | | Header | `Authorization: TD1 1/456789xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx` | | Header | `Content-Type: application/json` | ```json Sample Request curl --location --request GET 'https:{{baseUrl}}/entities/journeys/:journeyId' \ --header 'Content-Type: application/json' \ --header 'Accept: application/json' \ --header 'Authorization: TD1 123/456789xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx' \ ``` ```json Sample Response { "data": { "id": "117385", "type": "journey", "attributes": { "audienceId": "57350", "name": "Purchase journey", "description": "", "state": "draft", "goal": null, "createdAt": "2023-09-11T20:47:29.086Z", "updatedAt": "2023-09-11T20:47:29.086Z", "launchedAt": null, "allowReentry": false, "paused": false, "pausedAt": null, "journeyStages": [ { "id": "149348", "name": "Awareness", "description": null, "steps": { "4e7dcb3e-4f3a-4a6c-be3f-a37b09acbac7": { "type": "Activation", "next": "2903c0a0-e43c-4ed6-8e0c-1b682be1cc03", "description": null, "name": "act_1", "journeyActivationStepId": "153668" }, "2903c0a0-e43c-4ed6-8e0c-1b682be1cc03": { "type": "WaitStep", "next": null, "waitStep": 1, "waitStepUnit": "day" } }, "rootStep": "4e7dcb3e-4f3a-4a6c-be3f-a37b09acbac7", "entryCriteria": { "name": "browse", "segmentId": "912056", "description": "" }, "milestone": null, "exitCriterias": null } ] }, "relationships": { "parentFolder": { "data": { "id": "183985", "type": "folder-segment" } }, "jumpFrom": { "data": [] }, "jumpTo": { "data": [] } } } } ``` About this request: - Replace {baseUrl} with the appropriate endpoint for your region. - Replace {journeyId} with the draft journey to which you want to add or update A/B tests. - Replace {TD1 123/456789xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx} with your personal API key. About this response object: - The name of the A/B test and its variants are contained within the journey stage. ## Create A/B Tests in a Journey You add an A/B test to any decision point; there is no limit to the number of A/B tests within a journey. The following example sets up a test name, number of variants, percentage allocation to each variable, and a stable ID key. ```json Sample Request curl --location --request POST 'https:{{baseUrl}}/entities/journeys/:journeyId' \ --header 'Content-Type: application/json' \ --header 'Accept: application/json' \ --header 'Authorization: TD1 123/456789xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx' \ data raw '{ "type": "ABTest", "name": , "customizedSplit": true or false "uniqueId": , "variants": [ { "id": , "name": , "percentage": 10, "next": } ] } ``` ```json Sample Response { "data": { "id": "117385", "type": "journey", "attributes": { "audienceId": "57350", "name": "Purchase journey", "description": "", "state": "draft", "goal": null, "createdAt": "2023-09-11T20:47:29.086Z", "updatedAt": "2023-09-11T20:47:29.086Z", "launchedAt": null, "allowReentry": false, "paused": false, "pausedAt": null, "journeyStages": [ { "id": "149348", "name": "Awareness", "description": null, "steps": { "4e7dcb3e-4f3a-4a6c-be3f-a37b09acbac7": { "type": "Activation", "next": "2903c0a0-e43c-4ed6-8e0c-1b682be1cc03", "description": null, "name": "act_1", "journeyActivationStepId": "153668" }, "2903c0a0-e43c-4ed6-8e0c-1b682be1cc03": { "type": "WaitStep", "next": "e48c0c6f-7de2-4d94-9b22-c6c3eb51105e", "waitStep": 1, "waitStepUnit": "day" }, "e48c0c6f-7de2-4d94-9b22-c6c3eb51105e": { "type": "ABTest", "variants": [ { "id": "ee9aa667-60c7-42e0-b8b1-5cee9a2edfb3", "name": "Variant A", "percentage": 45, "next": null }, { "id": "d4179b47-8d9a-495c-bddd-5dcf25c295c6", "name": "Variant B", "percentage": 5, "next": null }, { "id": "bbf9c797-efb0-41ae-83b0-53082d8337a8", "name": "Control", "percentage": 50, "next": null } ], "name": "test", "customizedSplit": true, "uniqueId": "cdp_customer_id" } }, "rootStep": "4e7dcb3e-4f3a-4a6c-be3f-a37b09acbac7", "entryCriteria": { "name": "browse", "segmentId": "912056", "description": "" }, "milestone": null, "exitCriterias": null } ] }, "relationships": { "parentFolder": { "data": { "id": "183985", "type": "folder-segment" } }, "createdBy": { "data": { "id": "1020", "type": "user" } }, "updatedBy": { "data": { "id": "1020", "type": "user" } }, "activations": { "data": [] }, "journeyActivationSteps": { "data": [ { "id": "153668", "type": "journeyActivationStep" } ] }, "jumpFrom": { "data": [] }, "jumpTo": { "data": [] } } } } ``` About this request: - Replace {baseUrl} with the appropriate endpoint for your region. - Replace {journeyId} with the specified journey ID. - Replace {TD1 123/456789xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx} with your personal API key. About this response object: - The name of the A/B test and its variants are adding to the journey stage. ## Modify an A/B Test You can make changes to an A/B test while it is in draft mode. For instance, you can rename the test or its variants, delete variants or the A/B test, change the Unique ID field, or the allocation percentage. ```json Sample Request curl --location --request PATCH 'https:{{baseUrl}}/entities/journeys/:journeyId' \ --header 'Content-Type: application/json' \ --header 'Accept: application/json' \ --header 'Authorization: TD1 123/456789xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx' \ data raw '{ "type": "ABTest", "name": , "customizedSplit": true or false "uniqueId": , "variants": [ { "id": , "name": , "percentage": 10, "next": } ] } ``` ```json Sample Response "data": { "id": "117385", "type": "journey", "attributes": { "audienceId": "57350", "name": "Purchase journey", "description": "", "state": "draft", "goal": null, "createdAt": "2023-09-11T20:47:29.086Z", "updatedAt": "2023-09-11T20:47:29.086Z", "launchedAt": null, "allowReentry": false, "paused": false, "pausedAt": null, "journeyStages": [ { "id": "149348", "name": "Awareness", "description": null, "steps": { "4e7dcb3e-4f3a-4a6c-be3f-a37b09acbac7": { "type": "Activation", "next": "2903c0a0-e43c-4ed6-8e0c-1b682be1cc03", "description": null, "name": "act_1", "journeyActivationStepId": "153668" }, "2903c0a0-e43c-4ed6-8e0c-1b682be1cc03": { "type": "WaitStep", "next": "e48c0c6f-7de2-4d94-9b22-c6c3eb51105e", "waitStep": 1, "waitStepUnit": "day" }, "e48c0c6f-7de2-4d94-9b22-c6c3eb51105e": { "type": "ABTest", "variants": [ { "id": "51527fb6-72c4-4e85-a4bc-f3b7094100a2", "name": "Semi monthly", "percentage": 17, "next": null }, { "id": "d4179b47-8d9a-495c-bddd-5dcf25c295c6", "name": "Semi weekly", "percentage": 17, "next": null }, { "id": "ee9aa667-60c7-42e0-b8b1-5cee9a2edfb3", "name": "Weekly Once", "percentage": 17, "next": null }, { "id": "bbf9c797-efb0-41ae-83b0-53082d8337a8", "name": "Control", "percentage": 49, "next": null } ], "name": "Sending promotion emails", "customizedSplit": false, "uniqueId": "cdp_customer_id" } }, "rootStep": "4e7dcb3e-4f3a-4a6c-be3f-a37b09acbac7", "entryCriteria": { "name": "browse", "segmentId": "912056", "description": "" }, "milestone": null, "exitCriterias": null } ] }, "relationships": { "parentFolder": { "data": { "id": "183985", "type": "folder-segment" } }, "createdBy": { "data": { "id": "1020", "type": "user" } }, "updatedBy": { "data": { "id": "1020", "type": "user" } }, "activations": { "data": [] }, "journeyActivationSteps": { "data": [ { "id": "153668", "type": "journeyActivationStep" } ] }, "jumpFrom": { "data": [] }, "jumpTo": { "data": [] } } } } ``` About this request: - Replace {baseUrl} with the appropriate endpoint for your region. - Replace {journeyId} with journey you want to add or update A/B tests. - Replace {TD1 123/456789xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx} with your personal API key. About this response object: - The name of the A/B test and its variants have been modified. - The number of variants is increased in the response from 2 to 3. - The customized splits is changed to `false`. - The percentage allocated to each variant has also been modified. ## Delete an A/B Test You can delete an A/B test while it is in draft mode. The section contains: [Verify the A/B Test in the Journey Stage](#verify-the-ab-test-in-the-journey-stage) [Delete an A/B test Within a Journey Stage](#delete-an-ab-test-within-a-journey-stage) [Confirming Journey Status](#confirming-journey-status) ### Verify the A/B Test in the Journey Stage To confirm which A/B test to delete, you first need to confirm an A/B Test applied to a stage. In the example that follows, you can view the journey stage and A/B tests for journey `117385`. **Sample Request** ```Json Sample Request Verify A/B Test curl --location --request GET 'https:{{baseUrl}}/entities/journeys/:journeyId' \ --header 'Content-Type: application/json' \ --header 'Accept: application/json' \ --header 'Authorization: TD1 123/456789xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx' \ data raw '{ "data": { "id": "117385", "type": "journey", "attributes": { "audienceId": "57350", "name": "Purchase journey", "description": "", "state": "draft", "goal": null, "createdAt": "2023-09-11T20:47:29.086Z", "updatedAt": "2023-09-11T20:47:29.086Z", "launchedAt": null, "allowReentry": false, "paused": false, "pausedAt": null, "journeyStages": [ { "id": "149348", "name": "Awareness", "description": null, "steps": { "4e7dcb3e-4f3a-4a6c-be3f-a37b09acbac7": { "type": "Activation", "next": "2903c0a0-e43c-4ed6-8e0c-1b682be1cc03", "description": null, "name": "act_1", "journeyActivationStepId": "153668" }, "2903c0a0-e43c-4ed6-8e0c-1b682be1cc03": { "type": "WaitStep", "next": "e48c0c6f-7de2-4d94-9b22-c6c3eb51105e", "waitStep": 1, "waitStepUnit": "day" }, "e48c0c6f-7de2-4d94-9b22-c6c3eb51105e": { "type": "ABTest", "variants": [ { "id": "51527fb6-72c4-4e85-a4bc-f3b7094100a2", "name": "Semi monthly", "percentage": 17, "next": null }, { "id": "d4179b47-8d9a-495c-bddd-5dcf25c295c6", "name": "Semi weekly", "percentage": 17, "next": null }, { "id": "ee9aa667-60c7-42e0-b8b1-5cee9a2edfb3", "name": "Weekly Once", "percentage": 17, "next": null }, { "id": "bbf9c797-efb0-41ae-83b0-53082d8337a8", "name": "Control", "percentage": 49, "next": null } ], "name": "Sending promotion emails", "customizedSplit": false, "uniqueId": "cdp_customer_id" } }, "rootStep": "4e7dcb3e-4f3a-4a6c-be3f-a37b09acbac7", "entryCriteria": { "name": "browse", "segmentId": "912056", "description": "" }, "milestone": null, "exitCriterias": null } ] }, "relationships": { "parentFolder": { "data": { "id": "183985", "type": "folder-segment" } }, "createdBy": { "data": { "id": "1020", "type": "user" } }, "updatedBy": { "data": { "id": "1020", "type": "user" } }, "activations": { "data": [] }, "journeyActivationSteps": { "data": [ { "id": "153668", "type": "journeyActivationStep" } ] }, "jumpFrom": { "data": [] }, "jumpTo": { "data": [] } } } } ``` About this request: An A/B test can be deleted from a journey stage by omitting the test in the same PATCH request. This request confirms the specified journey has an A/B Test in Stage 1. - Replace {baseUrl} with the appropriate endpoint for your region. - Replace {journeyId} with journey you want to add or update A/B tests. - Replace {TD1 123/456789xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx} with your personal API key. ### Delete an A/B Test Within a Journey Stage To delete the A/B test, the information about the A/B test is omitted from the PATCH. In the example that follows, the A/B test and its associated variants are absent from the payload for journey `117385`. ```json Sample Request - Omit the A/B Test curl --location --request PATCH 'https:{{baseUrl}}/entities/journeys/:journeyId' \ --header 'Content-Type: application/json' \ --header 'Accept: application/json' \ --header 'Authorization: TD1 123/456789xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx' \ data raw '{ "id": "117385", "type": "journey", "attributes": { "audienceId": "57350", "name": "Purchase journey", "description": "", "state": "draft", "goal": null, "createdAt": "2023-09-11T20:47:29.086Z", "updatedAt": "2023-09-11T20:47:29.086Z", "launchedAt": null, "allowReentry": false, "paused": false, "pausedAt": null, "journeyStages": [ { "id": "149348", "name": "Awareness", "description": null, "steps": { "4e7dcb3e-4f3a-4a6c-be3f-a37b09acbac7": { "type": "Activation", "next": "2903c0a0-e43c-4ed6-8e0c-1b682be1cc03", "description": null, "name": "act_1", "journeyActivationStepId": "153668" }, "2903c0a0-e43c-4ed6-8e0c-1b682be1cc03": { "type": "WaitStep", "next": null, "waitStep": 1, "waitStepUnit": "day" } }, "rootStep": "4e7dcb3e-4f3a-4a6c-be3f-a37b09acbac7", "entryCriteria": { "name": "browse", "segmentId": "912056", "description": "" }, "milestone": null, "exitCriterias": null } ] }, "relationships": { "parentFolder": { "data": { "id": "183985", "type": "folder-segment" } }, "createdBy": { "data": { "id": "1020", "type": "user" } }, "updatedBy": { "data": { "id": "1020", "type": "user" } }, "activations": { "data": [] }, "journeyActivationSteps": { "data": [ { "id": "153668", "type": "journeyActivationStep" } ] }, "jumpFrom": { "data": [] }, "jumpTo": { "data": [] } } } ``` About this request - The name of the A/B test and its variants have been removed from the PATCH request. - The "next" step is marked as "null" in the object preceding the A/B test. - Replace {baseUrl} with the appropriate endpoint for your region. - Replace {journeyId} with journey from which you want to remove an A/B test. - Replace {TD1 123/456789xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx} with your personal API key. **Sample Response** ```json Sample Response - Confirms absence of A/B Test curl --location --request GET 'https:{{baseUrl}}/entities/journeys/:journeyId' \ --header 'Content-Type: application/json' \ --header 'Accept: application/json' \ --header 'Authorization: TD1 123/456789xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx' \ data raw '{ "data": { "id": "117385", "type": "journey", "attributes": { "audienceId": "57350", "name": "Purchase journey", "description": "", "state": "draft", "goal": null, "createdAt": "2023-09-11T20:47:29.086Z", "updatedAt": "2023-09-11T20:47:29.086Z", "launchedAt": null, "allowReentry": false, "paused": false, "pausedAt": null, "journeyStages": [ { "id": "149348", "name": "Awareness", "description": null, "steps": { "4e7dcb3e-4f3a-4a6c-be3f-a37b09acbac7": { "type": "Activation", "next": "2903c0a0-e43c-4ed6-8e0c-1b682be1cc03", "description": null, "name": "act_1", "journeyActivationStepId": "153668" }, "2903c0a0-e43c-4ed6-8e0c-1b682be1cc03": { "type": "WaitStep", "next": null, "waitStep": 1, "waitStepUnit": "day" } }, "rootStep": "4e7dcb3e-4f3a-4a6c-be3f-a37b09acbac7", "entryCriteria": { "name": "browse", "segmentId": "912056", "description": "" }, "milestone": null, "exitCriterias": null } ] }, "relationships": { "parentFolder": { "data": { "id": "183985", "type": "folder-segment" } }, "createdBy": { "data": { "id": "1020", "type": "user" } }, "updatedBy": { "data": { "id": "1020", "type": "user" } }, "activations": { "data": [] }, "journeyActivationSteps": { "data": [ { "id": "153668", "type": "journeyActivationStep" } ] }, "jumpFrom": { "data": [] }, "jumpTo": { "data": [] } } } } ``` About this response: - The response confirms the A/B test and its variants have been removed from the journey stage. ### Confirming Journey Status If additional confirmation apart from the response is required, you can do a GET to the endpoint `entities/journeys/{journey_id}`. A similar response as the PATCH request's response is displayed when a GET to the same journey in performed. The journey attributes do not display any A/B test information, as it was successfully deleted during the PATCH request. ```json Sample Response - Confirms Journey Status curl --location --request GET 'https:{{baseUrl}}/entities/journeys/:journeyId' \ --header 'Content-Type: application/json' \ --header 'Accept: application/json' \ --header 'Authorization: TD1 123/456789xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx' \ data raw '{ "data": { "id": "117385", "type": "journey", "attributes": { "audienceId": "57350", "name": "Purchase journey", "description": "", "state": "draft", "goal": null, "createdAt": "2023-09-11T20:47:29.086Z", "updatedAt": "2023-09-11T20:47:29.086Z", "launchedAt": null, "allowReentry": false, "paused": false, "pausedAt": null, "journeyStages": [ { "id": "149348", "name": "Awareness", "description": null, "steps": { "4e7dcb3e-4f3a-4a6c-be3f-a37b09acbac7": { "type": "Activation", "next": "2903c0a0-e43c-4ed6-8e0c-1b682be1cc03", "description": null, "name": "act_1", "journeyActivationStepId": "153668" }, "2903c0a0-e43c-4ed6-8e0c-1b682be1cc03": { "type": "WaitStep", "next": null, "waitStep": 1, "waitStepUnit": "day" } }, "rootStep": "4e7dcb3e-4f3a-4a6c-be3f-a37b09acbac7", "entryCriteria": { "name": "browse", "segmentId": "912056", "description": "" }, "milestone": null, "exitCriterias": null } ] }, "relationships": { "parentFolder": { "data": { "id": "183985", "type": "folder-segment" } }, "createdBy": { "data": { "id": "1020", "type": "user" } }, "updatedBy": { "data": { "id": "1020", "type": "user" } }, "activations": { "data": [] }, "journeyActivationSteps": { "data": [ { "id": "153668", "type": "journeyActivationStep" } ] }, "jumpFrom": { "data": [] }, "jumpTo": { "data": [] } } } } ``` ## A/B Testing Statistics You can choose to show the number of profiles in a particular A/B test variant within the same parent segment using the following rule. ```json { "type": "And", "conditions": [ { "type": "Value", "exclude": true, "leftValue": { "journeyId": 1, "journeyStageId": 2, "location": "ab_test__variant_", "tense": "intime" }, "operator": { "type": "IsNull", "not": true } } ] } ```