[improve][pip]PIP-422 Support global topic-level policy: replicated clusters and new API to delete topic-level policies #24368
Conversation
pip/pip-422.md
Outdated
| ### Public API | ||
|
|
||
| **new Pulsar Admin API** | ||
| - `pulsar-admin topicPolicies --clusters <clusters> --global <topic>` |
There was a problem hiding this comment.
We already have bin/pulsar-admin topics set-replication-cluster, it seems we can only add an option instead of a new command.
There was a problem hiding this comment.
And for the set-replication-cluster without --global = true. We should only allow to remove the non-local cluster?
There was a problem hiding this comment.
We already have bin/pulsar-admin topics set-replication-cluster, it seems we can only add an option instead of a new command.
Only the command topicPolicies supports the param --global so far, add the new command into topicPolicies is better the topics.
Renamed the sub command name --clusters -> set-replication-cluster
And for the set-replication-cluster without --global = true. We should only allow to remove the non-local cluster?
Users can set a global policy from the remote side, then it will be replicated to the local cluster. After we added this limitation, users only can set a global policy on the local cluster
pip/pip-422.md
Outdated
| Passed solutions: | ||
| 1. Disable namespace-level Geo-Replication then delete the partitioned topic: this will trigger a deletion for other topics, which is not expected. | ||
| 2. Disable topic-level Geo-Replication under each cluster, and delete the partitioned topic by Admin API: it will delete the shared partitioned topic metadata, which was also deleted from another cluster. | ||
| 3. Disable topic-level Geo-Replication under each cluster, and delete the sub-topics manually by Admin API: this is the best solution without the current PIP, the defect is that the schema data and topic-level policies are still there. |
There was a problem hiding this comment.
Is it better to move to # Alternatives?
There was a problem hiding this comment.
Moved them into #Alternatives
pip/pip-422.md
Outdated
| - Disable global topic level `replicated clusters`, in other words, set `replicated clusters` with a single list. | ||
| - The clusters are not contained in the `replicated clusters` will delete the sub-topics automatically, such as the namespace-level implementation. |
There was a problem hiding this comment.
If there is any active producers or consumers connected the cluster which been removed from the replicated cluster list, the topic will be create again if topic auto-creation is enabled?
Or it will rely on the global topic policy which defined the replication_cluster. All the producers and consumers will be rejected if the connected cluster is not listed in replicated_clusters?
There was a problem hiding this comment.
Or it will rely on the global topic policy which defined the replication_cluster. All the producers and consumers will be rejected if the connected cluster is not listed in replicated_clusters?
This one. I have added to the proposal. https://github.com/apache/pulsar/pull/24368/files#diff-fca5852dd938f5905286f6ed91af543d8037c02e12726389f9f2ade9ae427c14R46
Co-authored-by: Penghui Li <[email protected]>
Co-authored-by: Penghui Li <[email protected]>
Motivation & Modifications
Documentation
docdoc-requireddoc-not-neededdoc-completeMatching PR in forked repository
PR in forked repository: x