DOC-12484 XDCR Conflict Logging feature #3806

rao-shwe · 2025-05-09T14:07:11Z

Link to the preview doc: https://preview.docs-test.couchbase.com/DOC-12484/server/current/learn/clusters-and-availability/xdcr-conflict-logging-feature.html

Get login credentials from here.

Preview pages:

What's New entry.
XDCR Conflict Logging
Viewing Conflict Logs.
Replication Settings for XDCR Conflict Logging
Added conflictLogging in the Curl syntax and explained briefly about conflictLogging in the paragraphs that follow.
Updated for ECCV Edit a Bucket.
Added the "NOTE: XDCR Conflict Logging can be enabled only in the Edit a Bucket mode." in Create a Bucket.
conflictLogging entry in List of Advanced Settings.

DON'T review the following files: The following are 7.6.6 release docs which were missing in the release/8.0 branch. So I added them again.

xdcr-active-active-sgw.adoc
xdcr-conflict-resolution.adoc
xdcr-enable-crossclusterversioning.adoc
The section [Create an XDCR Replication with mobile=Active](Create an XDCR Replication with mobile=Active).
Creating and Editing Buckets.

modules/manage/pages/manage-xdcr/create-xdcr-replication.adoc

modules/rest-api/pages/rest-xdcr-adv-settings.adoc

modules/rest-api/pages/rest-xdcr-create-replication.adoc

modules/rest-api/pages/rest-xdcr-adv-settings.adoc

modules/learn/pages/clusters-and-availability/xdcr-enable-crossclusterversioning.adoc

sumukhbhat2701

Some review feedback which hold true for all pages:

For this feature specifically, we need to use the term "true conflicts" more than just mentioning "conflicts". That means we need to first define what a true conflict is and set the expectation.
There should be a warning that this feature is best effort (and that true conflicts is assumed to be very low). Everything that's in this slide - https://couchbase.slack.com/archives/C0963TSUU0N/p1752763776316649.
The setting is quite complex to understand just from textual description. An example will do a lot of help to someone new reading this.
There should be a mention that on every true conflict detected, XDCR will log 3 documents to the conflict collection - CRD (Conflict record document - contains metadata of detected true conflict), source document in conflict & target document in conflict. It should be mentioned that the CRD will contain the document IDs of source and target documents logged. Maybe an example of source and target document IDs in CRD.
Continuation of (3), I think there should be some examples on how to make use of the detected and logged conflicts. Eg: Use SDK, N1QL, range scan, eventing etc.
There should be a mention that the logged documents will not be replicated by XDCR if conflict collection is a source collection of any XDCR.

sumukhbhat2701 · 2025-07-21T07:47:00Z

I think I missed one of the pages from reviewing, so if somethings are already done from last comment, please ignore.

modules/learn/pages/clusters-and-availability/xdcr-conflict-logging-feature.adoc

rao-shwe

@sumukhbhat2701

I've implemented most of your review inputs and closed the comments.

modules/learn/pages/clusters-and-availability/xdcr-conflict-logging-feature.adoc

modules/learn/pages/clusters-and-availability/xdcr-enable-crossclusterversioning.adoc

modules/manage/pages/manage-xdcr/create-xdcr-replication.adoc

modules/rest-api/pages/rest-xdcr-adv-settings.adoc

modules/rest-api/pages/rest-xdcr-create-replication.adoc

rao-shwe · 2025-07-21T12:39:33Z

@sumukhbhat2701

Points 1 and 2 are fixed.
Point 3: Already has examples and descriptions. Not okay to repeat the same content in multiple locations. So I've added a link to examples wherever necessary.
Point 4, 5, and 6: Already exists.

Edit: All fixed in draft-2.

Some review feedback which hold true for all pages:

For this feature specifically, we need to use the term "true conflicts" more than just mentioning "conflicts". That means we need to first define what a true conflict is and set the expectation.

There should be a warning that this feature is best effort (and that true conflicts is assumed to be very low). Everything that's in this slide - https://couchbase.slack.com/archives/C0963TSUU0N/p1752763776316649.

The setting is quite complex to understand just from textual description. An example will do a lot of help to someone new reading this.

There should be a mention that on every true conflict detected, XDCR will log 3 documents to the conflict collection - CRD (Conflict record document - contains metadata of detected true conflict), source document in conflict & target document in conflict. It should be mentioned that the CRD will contain the document IDs of source and target documents logged. Maybe an example of source and target document IDs in CRD.

Continuation of (3), I think there should be some examples on how to make use of the detected and logged conflicts. Eg: Use SDK, N1QL, range scan, eventing etc. @hyunjuV I think you had a document prepared for this, was that for public docs?

There should be a mention that the logged documents will not be replicated by XDCR if conflict collection is a source collection of any XDCR.

modules/learn/pages/clusters-and-availability/xdcr-conflict-logging-feature.adoc

modules/learn/pages/clusters-and-availability/xdcr-viewing-conflict-logs.adoc

modules/learn/pages/clusters-and-availability/xdcr-conflict-logging-feature.adoc

hyunjuV · 2025-08-28T09:37:26Z

modules/manage/pages/manage-buckets/create-bucket.adoc

@@ -175,6 +175,8 @@ For more information about flushing, see xref:manage-buckets/flush-bucket.adoc[F
 [#add-data-bucket-dialog-expanded]
 image::manage-buckets/addBucketWithMagmaOption.png[,400,align=center, alt="An image that displays the Add Data Bucket dialog, with a Couchbase Bucket Type and CouchStore Storage Backend selected. The Advanced bucket settings are expanded and to show the default selections for a Couchbase and Couchstore bucket."]

+NOTE: XDCR Conflict Logging can be enabled only in the xref:manage:manage-buckets/edit-bucket.adoc[Edit a Bucket] mode.


I think that this note is incorrect -- I think that you meant "Enable Cross Cluster Versioning".

NOTE: Enable Cross Cluster Versioning can be enabled only in the xref:manage:manage-buckets/edit-bucket.adoc[Edit a Bucket] mode.

Enabling Cross Cluster Versioning is a prerequisite for features like XDCR Conflict Logging and XDCR Active-Active with Sync Gateway 4.0+. See xref:learn:clusters-and-availability/xdcr-enable-crossclusterversioning.adoc[XDCR enableCrossClusterVersioning] for more info.

No, I meant what I've written.
To enable "XDCR Conflict Logging", the property "Enabling Cross Cluster Versioning" must be enabled as a prerequisite. The property "Enabling Cross Cluster Versioning" can be enabled only in the "Edit Bucket" mode and not in the "Create Bucket" mode. So, as a transitive rule, "XDCR Conflict Logging" can be enabled only in the "Edit Bucket" mode.

Even on the UI, when you try to use the toggle button to enable "Specify collections for storing conflict logs and documents", there is an info icon which says that you must enable Cross Cluster Versioning on both source and target buckets. And again, Cross Cluster Versioning can be enabled only in the "Edit Bucket" mode.

hyunjuV · 2025-08-28T09:40:37Z

modules/manage/pages/manage-buckets/edit-bucket.adoc

+image::manage-buckets/edit-bucket-with-eccv.png[,399,align=center]
+
+When you enable the bucket setting [.ui]*Enable Cross Cluster Versioning*, for each document processed by XDCR, XDCR stores additional metadata for the document in the extended attributes.
+This metadata is also called xref:learn:clusters-and-availability/xdcr-enable-crossclusterversioning.adoc#hlv-data-maintained-in-xattr[Hybrid Logical Vector (HLV)].


This metadata is called xref:learn:clusters-and-availability/xdcr-enable-crossclusterversioning.adoc#hlv-data-maintained-in-xattr[Hybrid Logical Vector (HLV)], or version vectors.

hyunjuV · 2025-08-28T09:48:27Z

modules/manage/pages/manage-xdcr/create-xdcr-replication.adoc

+* *Specify collections for storing conflict logs and documents*: Use this setting to enable choosing a conflict collection for storing conflict logs and documents.
+This setting allows you to specify the bucket, scope, and collection where conflict logs and documents will be stored during XDCR conflict logging process. Conflict Logging Rules panel displays the mapping of conflict collection that you have chosen.
+
+* *Enable Conflict Logging*: Use this setting to enable or disable conflict logging for the replication. When enabled, XDCR starts logging conflicts in a specified bucket, scope, and collection.


Typo -- "the" instead of "a".

"in the specified bucket, scope, and collection" instead of "in a specified bucket, scope, and collection."

hyunjuV · 2025-08-28T09:58:32Z

modules/manage/pages/manage-xdcr/create-xdcr-replication.adoc

+
+For information about XDCR Conflict Logging, see xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc[XDCR Conflict Logging].
+
+For information about creating collections (including conflict collections), see xref:manage:manage-scopes-and-collections/manage-scopes-and-collections.adoc[Manage Scopes and Collections].


We should leave out the "(including conflict collections)" -- the conflict collections (or conflict log collections) are not special -- they are just regular collections. We don't want people to think that they are a special type of collection. I get their purpose is special when people specify them to be conflict collections when they configure XDCR conflict logging, but when people create them, they are just collections like any other collection. Also, XDCR does not have anything to do with creating collections, so I don't want people to think that XDCR creates collections. People specify existing collections to serve as conflict collections when they configure XDCR Conflict Logging. (I get that they may need to create those collections prior to using them for conflict logging, but creating the collections is not part of the feature -- e.g. they can use existing collections.)

For information about creating collections, see xref:manage:manage-scopes-and-collections/manage-scopes-and-collections.adoc[Manage Scopes and Collections].

hyunjuV · 2025-08-28T10:04:22Z

modules/manage/pages/manage-xdcr/create-xdcr-replication.adoc

@@ -137,6 +143,38 @@ image::manage-xdcr/xdcr-advanced-settings.png[,400,align=left]
 The values displayed in the fields are defaults, which can be modified interactively, and saved: this may help in achieving optimal replication-performance.
 For details on the significance of each field, see the xref:xdcr-reference:xdcr-reference-intro.adoc[XDCR Reference].

+[#xdcr-ui-settings-for-conflict-logging]
+=== Replication Settings for XDCR Conflict Logging
+


I don't think you mention in this section that enableCrossClusterVersioning is a prerequisite to being able to use XDCR Conflict Logging. Seems like it should be mentioned some where.

Note: To configure and enable XDCR Conflict Logging, you must enable the bucket property enableCrossClusterVersioning on all buckets of the XDCR topology. This bucket property cannot be disabled once it is enabled. For information about the bucket property enableCrossClusterVersioning, see xref:learn:clusters-and-availability/xdcr-enable-crossclusterversioning.adoc[XDCR enableCrossClusterVersioning].

hyunjuV · 2025-08-28T10:31:29Z

modules/rest-api/pages/rest-xdcr-adv-settings.adoc

+
+If the call is successful, an object is returned, which contains the replication settings for fine tuning the Conflict Logging feature:
+
+NOTE: For more information about fine tuning the replication settings in the Conflict Logging feature, see xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#tunable-replication-settings[Tunable Replication Settings].


Don't say "fine tuning" -- we don't want customers to tune anything unless directed by Support. I had suggested changing the heading "Tunable Replication Settings" to just "Conflict Log Settings" and "Configuring the conflictLogging Setting" to "Enabling and Configuring Conflict Logging" in the xdcr-conflict-logging-feature.adoc.

NOTE: For more information about the replication settings in the Conflict Logging feature, see xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#tunable-replication-settings[Conflict Log Settings] and xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#configure-conflictlogging-settings[Enabling and Configuring Conflict Logging].

hyunjuV · 2025-08-28T10:32:43Z

modules/rest-api/pages/rest-xdcr-adv-settings.adoc

+curl -X GET -s -u Administrator:'password' http://localhost:8091/settings/replications/f6f3a51e988f305b16433237d02bec7d%2Fbucket3%2Fbucket3 | jq `.`
+----
+
+If the call is successful, an object is returned, which contains the replication settings for fine tuning the Conflict Logging feature:


Don't say "fine tuning" -- we don't want customers to tune these unless directed by Support.

If the call is successful, an object is returned, which contains the replication settings for the Conflict Logging feature:

hyunjuV · 2025-08-28T10:42:04Z

modules/rest-api/pages/rest-xdcr-adv-settings.adoc

+Within the JSON Object, `conflictLogging` setting is enabled by default (`disabled=false`).
+
+This setting uses a JSON object named `loggingRules` to define its values.


Please remove lines 434 through 436 -- the two statements are not clear.

For example, by default, conflictLogging is disabled:
"conflictLogging": {},
So, line 434 is incorrect.
Also, loggingRules is just a part of the configuration for conflictLogging, and it's only an optional configuration, so line 436 is incorrect.

Should just refer to xdcr-conflict-logging-feature.adoc which has the complete context like you do on line 437.

hyunjuV · 2025-08-28T10:43:04Z

modules/rest-api/pages/rest-xdcr-adv-settings.adoc

+Within the JSON Object, `conflictLogging` setting is enabled by default (`disabled=false`).
+
+This setting uses a JSON object named `loggingRules` to define its values.
+For more  information, see xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#configure-conflictlogging-settings[Configuring the conflictLogging Setting].


For more information, see xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#configure-conflictlogging-settings[Enabling and Configuring Conflict Logging].

hyunjuV · 2025-08-28T10:44:37Z

modules/rest-api/pages/rest-xdcr-create-replication.adoc

@@ -28,6 +28,7 @@ curl -v -X POST -u [admin]:[password]
  -d fromBucket=[bucket-name]
  -d toCluster=[cluster-name]
  -d toBucket=[bucket-name]
+  -d conflictLogging [JSON-Document]


-d conflictLogging=[JSON-Document]

hyunjuV · 2025-08-28T10:53:36Z

modules/rest-api/pages/rest-xdcr-create-replication.adoc

+The `conflictLogging` setting enables or disables logging conflicts for the replication, and has a JSON Object `loggingRules` as its value.
+Within the JSON Object, `conflictLogging` setting is enabled by default (`disabled=false`). You can specify the target bucket, scope, and collection for logging conflicts, and source collections of the replication.
+This helps to track and resolve document conflicts manually during replication.
+`loggingRules` is a JSON object that defines the custom conflict logging rules for specific collections.
+An empty JSON object value must be specified to disable the `conflictLogging` (Conflict Logging feature).
+For configuration information and examples, see xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#configure-conflictlogging-settings[Configuring the conflictLogging Settings].
+


Lines 64, 65, 66, 67, 68 are all misleading, confusing, or only partially correct. Should just say:

The conflictLogging setting is used to configure and enable or disable logging conflicts for the replication. For configuration information and examples, see xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#configure-conflictlogging-settings[Enabling and Configuring Conflict Logging].

Note: To configure and enable Conflict Logging, you must enable the bucket property enableCrossClusterVersioning on all buckets of the XDCR topology. This bucket property cannot be disabled once it is enabled. For information about the bucket property enableCrossClusterVersioning, see xref:learn:clusters-and-availability/xdcr-enable-crossclusterversioning.adoc[XDCR enableCrossClusterVersioning].

hyunjuV

I've left review comments and suggested changes.

FYI -- I've noticed that if you are using the new github experience, you will not see all the comments when you are looking at the "Files" tab (you'll see the comments in the Conversation tab, but not in the Files). That might have been because I made some of the comments in the new experience and some in the old. Anyways, if you are using the Preview of the New, need to click on "Switch back" to see all the comments in the files when you are in the "Files changed" tab. It's pretty confusing because you'll see some of the comments in the files, but not all. So, be sure that you are not in the "new experience" to see all of the comments when you are in the "Files" tab.

rao-shwe added 8 commits May 9, 2025 19:35

first-commit

d3c3a1c

small change

2fcf652

hlv-info

7b22571

conflict-logging-first-updates

6b20668

test

fc6ee7f

test-nested-content

75c108f

test2-nested-content

d8dcb3b

full-draft-document

18e7837

rao-shwe requested review from hyunjuV, sumukhbhat2701 and staticgc July 21, 2025 06:53

sumukhbhat2701 reviewed Jul 21, 2025

View reviewed changes

modules/manage/pages/manage-xdcr/create-xdcr-replication.adoc Show resolved Hide resolved

sumukhbhat2701 reviewed Jul 21, 2025

View reviewed changes

modules/manage/pages/manage-xdcr/create-xdcr-replication.adoc Outdated Show resolved Hide resolved

sumukhbhat2701 reviewed Jul 21, 2025

View reviewed changes

modules/manage/pages/manage-xdcr/create-xdcr-replication.adoc Outdated Show resolved Hide resolved

sumukhbhat2701 reviewed Jul 21, 2025

View reviewed changes

modules/rest-api/pages/rest-xdcr-adv-settings.adoc Show resolved Hide resolved

sumukhbhat2701 reviewed Jul 21, 2025

View reviewed changes

modules/rest-api/pages/rest-xdcr-create-replication.adoc Outdated Show resolved Hide resolved

sumukhbhat2701 reviewed Jul 21, 2025

View reviewed changes

modules/rest-api/pages/rest-xdcr-create-replication.adoc Outdated Show resolved Hide resolved

sumukhbhat2701 reviewed Jul 21, 2025

View reviewed changes

modules/rest-api/pages/rest-xdcr-create-replication.adoc Outdated Show resolved Hide resolved

sumukhbhat2701 reviewed Jul 21, 2025

View reviewed changes

modules/rest-api/pages/rest-xdcr-create-replication.adoc Outdated Show resolved Hide resolved

sumukhbhat2701 reviewed Jul 21, 2025

View reviewed changes

modules/rest-api/pages/rest-xdcr-create-replication.adoc Show resolved Hide resolved

sumukhbhat2701 reviewed Jul 21, 2025

View reviewed changes

modules/rest-api/pages/rest-xdcr-adv-settings.adoc Outdated Show resolved Hide resolved

sumukhbhat2701 reviewed Jul 21, 2025

View reviewed changes

modules/learn/pages/clusters-and-availability/xdcr-enable-crossclusterversioning.adoc Show resolved Hide resolved

sumukhbhat2701 suggested changes Jul 21, 2025

View reviewed changes

review-implementation-engg-1

3cfb3e2

rao-shwe commented Jul 21, 2025

View reviewed changes

rao-shwe added 2 commits July 21, 2025 18:18

minor-edit-in-image-dimension

6cb2d6a

minor-edit-in-image-2

d0f00b1

hyunjuV reviewed Aug 27, 2025

View reviewed changes

modules/learn/pages/clusters-and-availability/xdcr-conflict-logging-feature.adoc Show resolved Hide resolved

hyunjuV reviewed Aug 27, 2025

View reviewed changes

modules/learn/pages/clusters-and-availability/xdcr-conflict-logging-feature.adoc Show resolved Hide resolved

hyunjuV reviewed Aug 27, 2025

View reviewed changes

modules/learn/pages/clusters-and-availability/xdcr-conflict-logging-feature.adoc Show resolved Hide resolved