diff --git a/modules/ROOT/nav.adoc b/modules/ROOT/nav.adoc index f8be13d194..300d274986 100644 --- a/modules/ROOT/nav.adoc +++ b/modules/ROOT/nav.adoc @@ -111,6 +111,8 @@ include::third-party:partial$nav.adoc[] *** xref:manage:manage-nodes/failover-graceful.adoc[Graceful] *** xref:manage:manage-nodes/failover-hard.adoc[Hard] ** xref:manage:manage-nodes/recover-nodes.adoc[Recover a Node and Rebalance] + ** xref:manage:manage-nodes/manage-data-service-and-rebalance.adoc[Adding or Removing the Data Service on Existing Nodes] + ** xref:manage:manage-nodes/modify-services-and-rebalance.adoc[Modify Services and Rebalance] ** xref:manage:manage-nodes/apply-node-to-node-encryption.adoc[Manage Node-to-Node Encryption] ** xref:manage:manage-nodes/manage-address-families.adoc[Manage Address Families] * xref:manage:manage-buckets/bucket-management-overview.adoc[Manage Buckets] @@ -325,7 +327,9 @@ include::cli:partial$cbcli/nav.adoc[] **** xref:rest-api:rest-establish-credentials.adoc[Establishing Credentials] **** xref:rest-api:rest-name-node.adoc[Naming a Node] **** xref:rest-api:rest-configure-memory.adoc[Configuring Memory] - **** xref:rest-api:rest-set-up-services.adoc[Assigning Services] + **** xref:rest-api:rest-get-memory-information.adoc[Getting Memory Information] + **** xref:rest-api:rest-set-up-services.adoc[Assigning Services to a New Single Node] + **** xref:rest-api:rest-set-up-services-existing-nodes.adoc[Assigning Services to an Existing Node] **** xref:rest-api:rest-name-cluster.adoc[Naming a Cluster] *** xref:rest-api:rest-adding-and-removing-nodes.adoc[Node Addition and Removal] @@ -400,6 +404,7 @@ include::cli:partial$cbcli/nav.adoc[] ** xref:rest-api:rest-memory-and-storage.adoc[Memory and Storage API] *** xref:rest-api:rest-initialize-node.adoc[Initializing a Node] *** xref:rest-api:rest-configure-memory.adoc[Configuring Memory] + *** xref:rest-api:rest-get-memory-information.adoc[Getting Memory Information] *** xref:rest-api:rest-reader-writer-thread-config.adoc[Setting Thread Allocations] *** xref:rest-api:rest-getting-storage-information.adoc[Getting Storage Information] *** xref:rest-api:rest-compact-post.adoc[Performing Compaction Manually] diff --git a/modules/introduction/partials/new-features-80.adoc b/modules/introduction/partials/new-features-80.adoc index 0dd2c20030..51c9170f81 100644 --- a/modules/introduction/partials/new-features-80.adoc +++ b/modules/introduction/partials/new-features-80.adoc @@ -59,6 +59,23 @@ curl --get -u \ -d clusterLabels=none|uuidOnly|uuidAndName ---- +[#section-new-feature-800-services] +=== Services + +https://jira.issues.couchbase.com/browse/MB-60349[MB-60349]:: +Now you can add or remove non-Data Services dynamically on existing nodes in a cluster, without adding or removing nodes. +Rebalancing is automatically triggered to distribute service workloads and complete the modification. +These are the services that can be modified: + +** `index` (Index Service) +** `n1ql` (Query Service) +** `fts` (Search Service) +** `cbas` (Analytics Service) +** `eventing` (Eventing Service) +** `backup` (Backup Service) + +You can modify these services using the Couchbase xref:manage:manage-nodes/modify-services-on-nodes-and-rebalance.adoc#modify-mds-services-from-ui[UI], xref:rest-api:rest-set-up-services-existing-nodes.adoc[REST API], or xref:manage:manage-nodes/modify-services-on-nodes-and-rebalance.adoc#modify-mds-services-using-cli[CLI]. + [#section-new-feature-data-service] === Data Service diff --git a/modules/learn/pages/clusters-and-availability/nodes.adoc b/modules/learn/pages/clusters-and-availability/nodes.adoc index efa1413853..29c998b01c 100644 --- a/modules/learn/pages/clusters-and-availability/nodes.adoc +++ b/modules/learn/pages/clusters-and-availability/nodes.adoc @@ -87,9 +87,8 @@ Alternatively, the default itself can be changed, provided that it does not requ The available node must be at stage 2 or 3: that is, it must have been started, and may have had its data, index, analytics, and eventing paths configured. However, it cannot have been provisioned in any way: if the routine for joining is executed on a provisioned node, an error is flagged, and the operation fails. + -Note that services can nevertheless be assigned to the new node during the join operation itself. -The -memory quota for each service defaults to the setting for the existing cluster. +NOTE: Services can be assigned to the new node during the join operation or on-demand. +The memory quota for each service defaults to the setting for the existing cluster. An error occurs if the new node does not have sufficient memory. + If Couchbase Web Console is used to perform the join, the data, index, analytics, and eventing paths can be modified as part of the join operations. @@ -150,14 +149,26 @@ Fast failover can be achieved by deploying an arbiter node, which is a node that Such a deployment ensures that the Cluster Manager automatically deploys critical system infrastructure on the arbiter node, rather than on any other, so as to avoid clashes. This may lead to several seconds of latency reduction, when a busy cluster undergoes failover. -For information about service deployment with: +=== Deploying Services on a Node -* The REST API, see xref:rest-api:rest-set-up-services.adoc[Assigning Services]. +You can add services to a new node when adding the node to a cluster. -* Couchbase Server Web Console during node-addition and node-joining, see the demonstrated uses of checkboxes, in xref:manage:manage-nodes/add-node-and-rebalance#arbiter-node-addition[Add a Node and Rebalance] and in xref:manage:manage-nodes/join-cluster-and-rebalance#arbiter-node-join[Join a Cluster and Rebalance]. +You can also add or remove non-Data Services on an existing node in a cluster, without adding or removing nodes. + +For information about the deployment of services on a new node using: + +* The REST API, see xref:rest-api:rest-set-up-services.adoc[Assigning Services to a New Single Node], xref:rest-api:rest-cluster-addnodes.adoc[Adding Nodes to Clusters], and xref:rest-api:rest-cluster-joinnode.adoc[Joining Nodes into Clusters]. + +* The Couchbase Server Web Console during node-addition and node-joining, see xref:manage:manage-nodes/add-node-and-rebalance#arbiter-node-addition[Add a Node and Rebalance] and in xref:manage:manage-nodes/join-cluster-and-rebalance#arbiter-node-join[Join a Cluster and Rebalance]. * The CLI during node-addition, see xref:cli:cbcli/couchbase-cli-server-add[server-add]. +You can add or remove non-Data Services on an existing node in a cluster using the following methods: + +* From the xref:manage:manage-nodes/modify-services-on-nodes-and-rebalance.adoc#modify-mds-services-from-ui[UI] or xref:manage:manage-nodes/modify-services-on-nodes-and-rebalance.adoc#modify-mds-services-using-cli[CLI]. +For more information, see xref:manage:manage-nodes/modify-services-and-rebalance.adoc[Modify Services and Rebalance]. +* Using the REST API. For more information, see xref:rest-api:rest-set-up-services-existing-nodes.adoc[Assigning Services to an Existing Node]. + [#Node Quantity] == Node Quantity diff --git a/modules/learn/pages/services-and-indexes/services/analytics-service.adoc b/modules/learn/pages/services-and-indexes/services/analytics-service.adoc index 17d14cabb4..b78106f57c 100644 --- a/modules/learn/pages/services-and-indexes/services/analytics-service.adoc +++ b/modules/learn/pages/services-and-indexes/services/analytics-service.adoc @@ -112,9 +112,15 @@ This continuous data ingestion allows operational and analytic queries to run co Due to the large scale and duration of operations it is likely to perform, the Analytics Service should be run _alone_, on its node or nodes in the cluster, _with no other Couchbase Service running on the Analytics nodes_. -For the practical steps required to initialize or join a cluster, and to deploy services, see -xref:manage:manage-nodes/create-cluster.adoc[Create a Cluster]. -For information on how to run analytic queries, see the xref:analytics:introduction.adoc[Introduction] to Couchbase Analytics. +For more information about: + +* How to initialize a cluster, see xref:manage:manage-nodes/create-cluster.adoc[Create a Cluster]. + +* How to join or add a new node to a cluster to support the Analytics Service, see xref:manage:manage-nodes/join-cluster-and-rebalance.adoc[Join a Cluster and Rebalance] and see xref:manage:manage-nodes/add-node-and-rebalance.adoc[Add a Node and Rebalance]. + +* How to add or remove services on an existing node of a cluster, see xref:manage:manage-nodes/modify-services-and-rebalance.adoc[Modify Services and Rebalance]. + +* How to run analytic queries, see the xref:analytics:introduction.adoc[Introduction] to Couchbase Analytics. [WARNING] .Non-Uniform Memory Access (NUMA) @@ -125,3 +131,7 @@ NUMA architectures divide memory into different zones, each with a specific CPU, Couchbase Analytics is not configured to align with the NUMA architecture. Deploying Couchbase Analytics on a server with the NUMA architecture may result in uneven memory distribution, increased latency, and degraded performance. Specifically, queries can become slower, and overall system efficiency can drop as memory access times vary significantly. ==== + +== See Also + +* For more information about adding or removing the Analytics Service on an existing node of a cluster, see xref:manage:manage-nodes/modify-services-and-rebalance.adoc[Modify Services and Rebalance]. \ No newline at end of file diff --git a/modules/learn/pages/services-and-indexes/services/backup-service.adoc b/modules/learn/pages/services-and-indexes/services/backup-service.adoc index 396e1269f9..e24e3c1438 100644 --- a/modules/learn/pages/services-and-indexes/services/backup-service.adoc +++ b/modules/learn/pages/services-and-indexes/services/backup-service.adoc @@ -44,11 +44,8 @@ the Cluster Manager elects one of them to be the leader. The leader is responsible for: * Dispatching backup tasks. - * Adding and removing nodes from the Backup Service. - * Cleaning orphaned tasks. - * Ensuring that all Backup Service nodes can reach the global storage locations. If the leader becomes unresponsive or fails over, the Backup Service stops until a rebalance takes place. @@ -250,3 +247,4 @@ You may want to experiment with different thread settings to find a balance betw * See xref:rest-api:backup-rest-api.adoc[Backup Service API] for information about using the Backup Service from the REST API. * To learn about the port numbers the Backup Service uses, see xref:install:install-ports.adoc[Couchbase Server Ports]. * For a list of Backup Service audit events, see xref:audit-event-reference:audit-event-reference.adoc[Audit Event Reference]. +* For more information about adding or removing the Backup Service on an existing node of a cluster, see xref:manage:manage-nodes/modify-services-and-rebalance.adoc[Modify Services and Rebalance]. \ No newline at end of file diff --git a/modules/learn/pages/services-and-indexes/services/data-service.adoc b/modules/learn/pages/services-and-indexes/services/data-service.adoc index f6e67f2079..5ed64bda63 100644 --- a/modules/learn/pages/services-and-indexes/services/data-service.adoc +++ b/modules/learn/pages/services-and-indexes/services/data-service.adoc @@ -48,8 +48,9 @@ For further information, see xref:buckets-memory-and-storage/memory.adoc[Memory] For information on how to activate, see xref:manage:manage-buckets/flush-bucket.adoc[Flush a Bucket]. ** *Expiry Pager*: Scans for items that have expired, and erases them from memory and disk; after which, a _tombstone_ remains for a default period of 3 days. -The expiry pager runs every 10 minutes by default: for information on changing the interval, see `cbepctl` xref:cli:cbepctl/set-flush_param.adoc[set flush_param]. -For more information on item-deletion and tombstones, see xref:data/expiration.adoc[Expiration]. +The expiry pager runs every 10 minutes by default. +For information about changing the interval, see xref:rest-api:rest-bucket-create.adoc#expirypagersleeptime[expiryPagerSleepTime]. +For more information about item-deletion and tombstones, see xref:data/expiration.adoc[Expiration]. ** *Batch Reader*: Enhances performance by combining changes made to multiple items into _batches_, which are placed on the disk queue, to be written to disk. * *Scheduler*: A pool of threads mainly used for handling I/O. The threads are divided into four kinds, which run independently of and without effect on one another: @@ -61,3 +62,5 @@ The threads are divided into four kinds, which run independently of and without + For more information on the Data Service' threading model, see xref:buckets-memory-and-storage/storage-settings.adoc[Storage Properties]. + +For more information about adding or removing the Data Service on an existing node, see xref:manage:manage-nodes/manage-data-service-and-rebalance.adoc[Adding or Removing the Data Service on Existing Nodes]. \ No newline at end of file diff --git a/modules/learn/pages/services-and-indexes/services/eventing-service.adoc b/modules/learn/pages/services-and-indexes/services/eventing-service.adoc index 3a73b4ef6c..2290a651a6 100644 --- a/modules/learn/pages/services-and-indexes/services/eventing-service.adoc +++ b/modules/learn/pages/services-and-indexes/services/eventing-service.adoc @@ -12,4 +12,6 @@ Events include changes made to specified items, and the arrival of scheduled poi The Eventing Service depends on the xref:services-and-indexes/services/data-service.adoc[Data Service], which must therefore be running on at least one cluster node. -For information on using the Eventing Service, see xref:eventing:eventing-overview.adoc[Eventing Service: Fundamentals]. +For more information about using the Eventing Service, see xref:eventing:eventing-overview.adoc[Eventing Service: Fundamentals]. + +For more information about adding or removing the Eventing Service on an existing node of a cluster, see xref:manage:manage-nodes/modify-services-and-rebalance.adoc[Modify Services and Rebalance]. \ No newline at end of file diff --git a/modules/learn/pages/services-and-indexes/services/index-service.adoc b/modules/learn/pages/services-and-indexes/services/index-service.adoc index d3f50fa3b4..5eef2a6a15 100644 --- a/modules/learn/pages/services-and-indexes/services/index-service.adoc +++ b/modules/learn/pages/services-and-indexes/services/index-service.adoc @@ -68,3 +68,7 @@ _Secondary_ Indexes, often referred to as _Global Secondary Indexes_ or _GSIs_, Secondary Indexes can be created on specified fields; placed on specific cluster-nodes; and replicated. For information on using the Indexing Service, see xref:indexes:indexing-overview.adoc[]. + +== See Also + +* For more information about adding or removing the Index Service on an existing node of a cluster, see xref:manage:manage-nodes/modify-services-and-rebalance.adoc[Modify Services and Rebalance]. \ No newline at end of file diff --git a/modules/learn/pages/services-and-indexes/services/query-service.adoc b/modules/learn/pages/services-and-indexes/services/query-service.adoc index 7883d36165..8c2be40750 100644 --- a/modules/learn/pages/services-and-indexes/services/query-service.adoc +++ b/modules/learn/pages/services-and-indexes/services/query-service.adoc @@ -43,3 +43,7 @@ Note the degree of parallelism with which operations are frequently performed, r The Query Service supports queries made in {sqlpp}. As well as providing a rich variety of query-options, {sqlpp} allows statements to be _prepared_, so that the parsing and compiling of plans is completed prior to execution; and permits _consistency-levels_ to be configured. For detailed information, see xref:n1ql:n1ql-language-reference/index.adoc[{sqlpp} Language Reference]. + +== See Also + +* For more information about adding or removing the Query Service on an existing node of a cluster, see xref:manage:manage-nodes/modify-services-and-rebalance.adoc[Modify Services and Rebalance]. \ No newline at end of file diff --git a/modules/learn/pages/services-and-indexes/services/search-service.adoc b/modules/learn/pages/services-and-indexes/services/search-service.adoc index c43696e539..fa3b76f7d6 100644 --- a/modules/learn/pages/services-and-indexes/services/search-service.adoc +++ b/modules/learn/pages/services-and-indexes/services/search-service.adoc @@ -64,3 +64,7 @@ In Couchbase Server 6.6 Enterprise Edition and later, the _Flex Index_ feature p For detailed information, see xref:n1ql:n1ql-language-reference/flex-indexes.adoc[Flex Indexes]. To use the Search Service in {sqlpp}, the Query, Index, and Search Services must all be running on the cluster. + +== See Also + +* For more information about adding or removing the Search Service on an existing node of a cluster, see xref:manage:manage-nodes/modify-services-and-rebalance.adoc[Modify Services and Rebalance]. \ No newline at end of file diff --git a/modules/learn/pages/services-and-indexes/services/services.adoc b/modules/learn/pages/services-and-indexes/services/services.adoc index 68d5b14e9a..eb0ec96287 100644 --- a/modules/learn/pages/services-and-indexes/services/services.adoc +++ b/modules/learn/pages/services-and-indexes/services/services.adoc @@ -1,99 +1,119 @@ = Services -:description: pass:q[Couchbase _Services_ support access to and maintenance of data.] +:description: pass:q[Couchbase Server Services provide data access and maintenance. You can deploy services flexibly across available hardware to support Multi-Dimensional Scaling, which allows the tuning of the cluster for optimal performance as workloads change.] +:page-toclevels: 3 :page-aliases: understanding-couchbase:services-and-indexes/services/services,architecture:services-archi-multi-dimensional-scaling,clustersetup:services-mds,learn:services-and-indexes/services-and-indexes,understanding-couchbase:services-and-indexes/services-and-indexes [abstract] -{description} Services -can be deployed with flexibility across available hardware-resources, providing -_Multi-Dimensional Scaling_, whereby a cluster can be tuned for optimal -handling of emergent workloads. - -Services are configured and deployed by the Full Administrator who initializes Couchbase Server on one or more nodes. -The standard configuration-sequence allows a subset of services to be selected per node, with an individual memory-allocation for each. -Each service supports a particular form of data-access. -Services not required need not be deployed. -Services intended to support a heavy workload can be deployed across multiple cluster-nodes to ensure optimal performance and resource-availability. - -In Couchbase Server Version 7.6 and later, you can add one or more arbiter nodes to your cluster. +{description} + +Services in Couchbase Server are configured and deployed during initialization on 1 or more nodes by a Full Administrator. +During configuration, the administrator can select a subset of services for each node and assign memory to each service individually. + +Each service provides a specific type of data access. +Couchbase recommends deploying only required services. +To handle heavy workloads, you can deploy a service across multiple nodes in the cluster to improve performance and resource availability. + +In Couchbase Server 7.6 and later versions, you can add 1 or more arbiter nodes to your cluster. include::learn:partial$arbiter-node-benefits.adoc[] == Services and Multi-Dimensional Scaling Couchbase Server provides the following services: -* *Data*: Supports the storing, setting, and retrieving of data-items, specified by a key. -* *Query*: Parses queries specified in the _N1QL_ query-language, executes the queries, and returns results. -The Query Service interacts with both the Data and Index services. -* *Index*: Creates indexes, for use by the Query and Analytics services. -* *Search*: Create indexes specially purposed for _Full-Text Search_. -This supports language-aware searching; allowing users to search for, say, the word `beauties`, and additionally obtain results for `beauty` and `beautiful`. -* *Analytics*: Supports join, set, aggregation, and grouping operations; which are expected to be large, long-running, and highly consumptive of memory and CPU resources. -* *Eventing*: Supports near real-time handling of changes to data: code can be executed both in response to document mutations and as scheduled by timers. -* *Backup*: Supports the scheduling of full and incremental data backups, either for specific individual buckets, or for all buckets on the cluster. -Also allows the scheduling of _merges_ of previously made backups. - -These services can be deployed, maintained, and provisioned independently of -one another, by means of _Multi-Dimensional Scaling_, to ensure the most -effective ongoing response to changing business conditions and emergent -workload-requirements. +* xref:learn:services-and-indexes:services/data-service.adoc[Data Service]: Stores, updates, and retrieves data items by key. +* xref:learn:services-and-indexes:services/query-service.adoc[Query Service]: Parses and executes {sqlpp} queries, and returns results. +Interacts with the Data and Index services. +* xref:learn:services-and-indexes:services/index-service.adoc[Index Service]: Creates indexes, for use by the Query and Analytics services. +* xref:learn:services-and-indexes:services/search-service.adoc[Search Service]: Creates indexes for a fully customizable search experience. +Supports near real-time search capabilities for a diverse range of data types, such as structured or unstructured text, dates, numbers, CIDR notation, geospatial data, and vectors. +* xref:learn:services-and-indexes:services/analytics-service.adoc[Analytics Service]: Performs join, set, aggregation, and grouping operations. +Designed for large, long-running workloads that require significant memory and CPU resources. +* xref:learn:services-and-indexes:services/eventing-service.adoc[Eventing Service]: Handles near real-time responses to data changes. +Executes code in response to document mutations or on scheduled timers. +* xref:learn:services-and-indexes:services/backup-service.adoc[Backup Service]: Schedules full and incremental backups for specific buckets or all buckets in a cluster. +Supports merging and pruning of existing backups. + +With Multi-Dimensional Scaling (MDS), you can deploy, maintain, and provision these services independently to optimize performance as business needs and workloads change. [#setting-up-services] == Setting Up Services -Services are set up on a per-node basis. -Each node can run at most one instance of a service. -Each node can run any number of services, up to the maximum, which is seven. -In Couchbase Enterprise Server Version 7.6+, a node can be configured to run _no_ service. -The _Data Service_ must run on at least one node of the cluster. -Some services are interdependent and therefore require at least one instance of -each of their dependencies to be running on the cluster — (for example, the -_Query Service_ depends on the _Index Service_ and on the _Data Service_). +Services are configured per node. +Each node can run 1 instance of a service and up to 7 services in total. +In Couchbase Enterprise Server 7.6 and later, you can configure a node to run no services. + +The Data Service must run on at least 1 node in the cluster. +Some services are interdependent and require at least 1 instance of each dependency to be running. +For example, the Query Service depends on the Index and Data services. + +When you initialize the first node in a cluster, that node's service configuration becomes the default for nodes added later. +You can change the default configuration for individual nodes by removing services, adding services, or both. + +In Couchbase Server 8.0 and later versions, you can add or remove the following non-Data Services on-demand on existing nodes in a cluster, without adding or removing nodes. + +* index (Index Service). +* n1ql (Query Service). +* fts (Search Service). +* cbas (Analytics Service). +* eventing (Eventing Service). +* backup (Backup Service). + +Then a rebalance operation is automatically triggered to distribute the service workload across the nodes and complete the modification. +For more information about adding or removing non-Data Services on an existing node, see xref:manage:manage-nodes/modify-services-and-rebalance.adoc[Modify Services and Rebalance]. + +Adding or removing of the Data Service (kv) on an existing node is supported only through adding or removing nodes. +For more information about adding or removing the Data Service on an existing node, see xref:manage:manage-nodes/manage-data-service-and-rebalance.adoc[Adding or Removing the Data Service on Existing Nodes]. -When the first node in a cluster is initialized, the services assigned to it become the default assignment for each other node subsequently to be added to the cluster. -However, this default can be departed from, node by node, with one or more services omitted from the default, and one or more added. +You can change the services running on nodes to support Multi-Dimensional Scaling (MDS). +This flexibility lets you adjust the cluster as per your requirements. -When first allocated to a node, a service requires the assignment of a specific memory quota, which becomes standard for that service in each of its instances across the cluster. -(The exceptions to this are the Query and Backup Services, which never require a memory quota.) +For example, you may start with a small cluster where some nodes run multiple services, such as Index and Search, because you have more need for services than the available nodes. +As your requirements grow, you can add nodes and move services to those new nodes to free resources on the original nodes. +If a service becomes resource-constrained, you can add it to more nodes in the cluster to increase its capacity and meet demand. -Service allocation should be designed based on workload-analysis: -if a particular service is expected to handle a heavy workload, -it should be allocated with a larger memory quota, and potentially as the only service on the node. -Alternatively, if a cluster is to be used for development only, -it may be convenient to allocate services in the quickest and most convenient way, -with some quotas being equal. +When you allocate a service to a node, you must assign it a memory quota. +This quota becomes the standard for that service across all its instances in the cluster. +The Query and Backup services are exceptions and do not require a memory quota. -For example, the following illustration shows how four services — Data, Index, Query, and Search — might be allocated evenly across the five nodes of a _development_ cluster: +You must design your service allocation based on workload analysis. +If a service is expected to handle a heavy workload, assign it a larger memory quota and consider running it as the only service on the node. +For development clusters, you can allocate services quickly and conveniently, with some quotas set equally. + +The following example shows how 4 services such as Data, Index, Query, and Search that can be evenly allocated across 5 nodes in a development cluster: [#cb_cluster_with_services_development] include::partial$cluster-example-diagrams.adoc[tag="five-cluster-setup"] -This configuration might provide perfectly acceptable performance for each of the four services, in the context of development and testing. -However, if a large amount of data is required to be intensively indexed and addressed by means of Query and Search, the following production configuration would be more efficient: +This configuration may provide adequate performance for development and testing. +However, if large volumes of data require intensive indexing and querying with Search, the following production configuration is more efficient: [#cb_cluster_with_services_production] include::partial$cluster-example-diagrams.adoc[tag="six-cluster-setup"] -In this revised configuration, the Data Service is the only service to run on three of the nodes; -the Index Service is running on two further nodes; and the Query and Search Services share the sixth and final node. +In the revised configuration, the Data Service runs exclusively on 3 nodes, the Index Service runs on 2 nodes, and the Query and Search services share the sixth node. + +For more information about: + +* Service memory quotas, see xref:buckets-memory-and-storage/memory.adoc[Memory]. + +* Initializing a cluster, see xref:manage:manage-nodes/create-cluster.adoc[Create a Cluster]. + +* Adding or removing the Data Service on a node, see xref:manage:manage-nodes/manage-data-service-and-rebalance.adoc[Adding or Removing the Data Service on Existing Nodes]. -For a more detailed explanation of service memory quotas, see xref:buckets-memory-and-storage/memory.adoc[Memory]. -For information on the practical steps required to initialize a cluster, including the allocation of services to nodes, see -xref:manage:manage-nodes/create-cluster.adoc[Create a Cluster]. +* Adding or removing non-Data services on a node, see xref:manage:manage-nodes/modify-services-and-rebalance.adoc[Modify Services and Rebalance]. + +* Using the REST API to assign services to a new single node cluster, see xref:rest-api:rest-set-up-services.adoc[Assigning Services to a New Single Node]. + +* Using the REST API to assign services to an existing node of a cluster, see xref:rest-api:rest-set-up-services-existing-nodes.adoc[Assigning Services to an Existing Node]. [#multi-dimensional-scaling] == Multi-Dimensional Scaling -The ability to deploy Couchbase Services with flexibility across hardware-resources -supports _Multi-Dimensional Scaling_, whereby a cluster can be fine-tuned for -optimal handling of emergent workload-requirements. If, for example, a greater -Search workload-requirement is encountered, one or more existing non-Search -nodes can be removed, reconfigured to run the Search Service, and re-added to the -cluster. - -Alternatively, additional hardware-resources (CPU, memory, disk-capacity) can be -added to targeted nodes in the cluster to support the performance of -key services. -This ability to provision services independently of one another, and -thereby scale their performance individually up and down as required, provides the greatest -flexibility in terms of handling changing business requirements and redeploying -existing resources to ensure continuously heightened efficiency. +Couchbase Services can be deployed flexibly across hardware resources to support Multi-Dimensional Scaling. +This allows you to fine-tune a cluster for optimal performance as workloads change. + +For example, if Search workloads increase, you can remove other services from 1 or more non-Search nodes and reconfigure those nodes to run the Search Service in the cluster. + +You can also add hardware resources such as CPU, memory, or disk capacity to specific nodes to improve the performance of key services. +Because services can be provisioned independently, you can scale their performance up or down as needed. +This flexibility helps meet changing business requirements and redeploy resources efficiently. diff --git a/modules/manage/pages/manage-nodes/add-node-and-rebalance.adoc b/modules/manage/pages/manage-nodes/add-node-and-rebalance.adoc index d6d7be7fdb..0a5b342ee9 100644 --- a/modules/manage/pages/manage-nodes/add-node-and-rebalance.adoc +++ b/modules/manage/pages/manage-nodes/add-node-and-rebalance.adoc @@ -24,7 +24,8 @@ The examples on this page assume that the default, _system-generated_ certificat In a production or similar context, to ensure security, only administrator-configured certificates should be used on a cluster: these should rely on a well known _Certificate Authority_, whose certificate is loaded as the cluster's _root_ certificate. (For more information, see xref:learn:security/certificates.adoc#server-certificates[Default Certificates and Certificate Substitution].) -In such a context, no node can be added to the cluster until conformant administrator-configured certificates have been loaded onto it -- such activities will thus need to be performed in addition to those shown by the examples on this page. +In this context, a node can be added to the cluster only after conformant administrator-configured certificates are loaded to the node. +You must configure certificates as well as completing the steps shown by the examples on this page. For more information, see xref:manage:manage-security/configure-server-certificates.adoc#adding-new-nodes[Adding and Joining New Nodes]. A complete overview of Couchbase-Server certificate-management is provided in xref:learn:security/certificates.adoc[Certificates]. diff --git a/modules/manage/pages/manage-nodes/manage-data-service-and-rebalance.adoc b/modules/manage/pages/manage-nodes/manage-data-service-and-rebalance.adoc new file mode 100644 index 0000000000..80afd86e3b --- /dev/null +++ b/modules/manage/pages/manage-nodes/manage-data-service-and-rebalance.adoc @@ -0,0 +1,40 @@ += Adding or Removing the Data Service on Existing Nodes +:description: pass:q[You can add or remove the Data Service on an existing node of a cluster by adding or removing the node from the cluster, and then completing the addition or removal of the node by running a rebalance operation.] +:page-toclevels: 3 + +[abstract] +{description} + +Unlike other services, you cannot dynamically add or remove the Data Service on an existing node of a cluster. + +You can add any service, including the Data Service (kv), when you add or join a new node to a cluster. +Also, when you remove a node, you are removing all the services on that node. +To complete the process of adding or removing a node, the rebalance operation must be run. +For more information about adding or removing a node and rebalacing using the UI, REST API, or CLI, see: + +* xref:manage:manage-nodes/add-node-and-rebalance.adoc[Add a Node and Rebalance] +* xref:manage:manage-nodes/remove-node-and-rebalance.adoc[Remove a Node and Rebalance] + +== Add the Data Service on an Existing Node + +To add the Data Service on an existing node of a cluster: + +. Remove the node that you want to add the Data Service to using xref:manage:manage-nodes/remove-node-and-rebalance.adoc[Remove a Node and Rebalance]. +. Then xref:manage:manage-nodes/add-node-and-rebalance.html[add the node] back to the cluster after enabling the Data Service. + +Alternatively, you can add a new node with the necessary services configuration, including the Data Service, using xref:manage:manage-nodes/add-node-and-rebalance.adoc[Add a New Node and Rebalance]. +Then remove the extraneous node that you no longer need using xref:manage:manage-nodes/remove-node-and-rebalance.adoc[Remove a Node and Rebalance]. + +== Remove the Data Service from an Existing Node + +To remove the Data Service from an existing node of a cluster: + +. Remove the node with the Data Service that you want to remove using xref:manage:manage-nodes/remove-node-and-rebalance.adoc[Remove a Node and Rebalance]. +. Then xref:manage:manage-nodes/add-node-and-rebalance.html[add the node] back to the cluster after disabling the Data Service. + +NOTE: During the node removal operation, if there are other services on the node that you're removing for the Data Service reconfiguration, and you do not want those services to be impacted when the node is removed from the cluster, you may need to add those services to another node temporarily. +You can also add a new node temporarily with those services enabled. + +== See Also + +For more information about modifying non-Data Services on existing nodes, see xref:manage:manage-nodes/modify-services-and-rebalance.adoc[Modify Services and Rebalance]. \ No newline at end of file diff --git a/modules/manage/pages/manage-nodes/modify-services-and-rebalance.adoc b/modules/manage/pages/manage-nodes/modify-services-and-rebalance.adoc new file mode 100644 index 0000000000..11b0aa8c11 --- /dev/null +++ b/modules/manage/pages/manage-nodes/modify-services-and-rebalance.adoc @@ -0,0 +1,155 @@ += Modify Services and Rebalance +:description: pass:q[Add or remove non-Data Services on existing nodes in a cluster and rebalance the cluster without adding or removing nodes.] +:page-toclevels: 3 + +[abstract] +{description} + +You can dynamically add or remove the following Multi-Dimensional Scaling (MDS) services on existing nodes in a cluster from the Couchbase xref:manage:manage-nodes/modify-services-on-nodes-and-rebalance.adoc#modify-mds-services-from-ui[UI], xref:rest-api:rest-set-up-services-existing-nodes.adoc[REST API], or xref:manage:manage-nodes/modify-services-on-nodes-and-rebalance.adoc#modify-mds-services-using-cli[CLI]: + +* `index` (Index Service) +* `n1ql` (Query Service) +* `fts` (Search Service) +* `cbas` (Analytics Service) +* `eventing` (Eventing Service) +* `backup` (Backup Service) + +Then a rebalance operation is automatically triggered to distribute the service workload across the nodes and complete the modification. + +You do not have to add or remove a node to add or remove a non-Data Service on a node in a cluster. + +NOTE: You cannot add or remove the Data Service (kv) using this method. +Adding or removing of the Data Service on an existing node is supported only through adding or removing nodes. +For more information about adding or removing the Data Service on an existing node, see xref:manage:manage-nodes/manage-data-service-and-rebalance.adoc[Adding or Removing the Data Service on Existing Nodes]. + +== Prerequisites + +Before modifying non-data services on nodes, ensure you have the following: + +* Full Administrator access. +For more information, see xref:learn:security/roles.adoc#full-admin[Full Admin]. +* A clear understanding of the current service distribution across the nodes. +* Service Memory Quota: +When you add a service that was not earlier present in the cluster, the new service may use the service-specific default memory quota. +This applies to the Index, Search, Eventing, and Analytics services. ++ +Verify that the memory quota for the new service meets your requirements using one of the following: + +** The REST API from xref:rest-api:rest-get-memory-information.adoc[Getting Memory Information]. +** The UI from xref:manage:manage-settings/general-settings.adoc#configure-general-settings-with-the-ui[Configure General Settings with the UI]. ++ +NOTE: The service memory quota setting is per server node and is reflected in the `GET /pools/default` REST API response. + +* Node Disk Storage Paths: +The node disk storage paths are set for all services that require disk storage paths (Data, Indexes, Eventing, Analytics) when a node is initialized. +These storage paths cannot be changed after the node becomes a part of a cluster. +This means that if you are adding a service (Index, Search, Analytics, Eventing) to an existing node, that service is using the disk storage path that was specified when the node was initialized. +Ensure that you know the node disk storage path set for the service that you are adding. +For information on how to view the node disk storage paths, see xref:manage:manage-nodes/list-cluster-nodes.adoc#list-nodes-with-the-rest-api[List Nodes with the REST API]. +Use the `GET /pools/default` REST API to find the otpNode value for the node and then use the `GET /nodes/` REST API to view the disk storage paths. + +[#modify-mds-services-from-ui] +== Modify Services from the UI + +To manage non-Data services from the UI, follow these steps: + +. Open the Couchbase Server Web Console. + +. Click [.ui]*Servers*. + +. Click btn:[Modify Services]. ++ +The [.ui]*Modify Cluster Services* dialog appears, which lists all the nodes in all the clusters and their current services. ++ +-- +Services that can be modified are: + +** *index* (Index Service) +** *n1ql* (Query Service) +** *fts* (Search Service) +** *cbas* (Analytics Service) +** *eventing* (Eventing Service) +** *backup* (Backup Service). +-- ++ +kv (Data Service) cannot be modified. + +. Select the services you want to add for each node and deselect the services you want to remove from each node. ++ +NOTE: Adding or removing any service triggers a rebalance operation. + +. Click btn:[Rebalance and Change Services]. ++ +If you're only removing services, adding new instances of services that already exist on the cluster, or adding the backup service, then rebalancing is triggered immediately. + +. If you are adding a service that is not already on the cluster, and if that service has a memory quota setting option, then the *New Service Settings* dialog appears. +This dialog lists the services you selected to add, the existing active services, and their current memory quotas. + +.. Enter the memory quota for each service you're adding. +You can also edit the memory quota for existing services. ++ +For the Index Service, choose either *Standard Global Secondary* or *Memory-Optimized*. For more information about index storage, see xref:learn:services-and-indexes:indexes/storage-modes.adoc[Index Storage Settings]. ++ +For more information about managing memory quotas for services from the UI, see xref:manage:manage-settings/general-settings.adoc#memory-quotas[Memory Quotas]. + +.. Click btn:[Save Settings]. + +The rebalancing operation is automatically triggered, which distributes the service workload across the nodes and completes the modification. +Status progress is displayed on the UI. + +When the rebalance operation is successful, an acknowledgment message appears on the UI and you can download the rebalance report by clicking btn:[Download Report]. + +[#modify-mds-services-using-cli] +== Modify Services Using CLI + +To modify non-Data services on the existing nodes of a cluster using the CLI, use the following `couchbase-cli rebalance` command: + +---- +couchbase-cli rebalance -c \ +--username \ +--password \ +--update-services [--index-[add|remove] ] [--n1ql-[add|remove] ] [--fts-[add|remove] ] [--cbas-[add|remove] ] [--eventing-[add|remove] ] [--backup-[add|remove] ] +---- + +These are the options for modifying non-data services: + +* `--update-services`: Indicates that the rebalance will make changes to the services on the cluster. + +* `---add `: Adds the specified service to the specified node. +`` can be one of the following: + +** `index` for the Index Service. +** `n1ql` for the Query Service. +** `fts` for the Search Service. +** `cbas` for the Analytics Service. +** `eventing` for the Eventing Service. +** `backup` for the Backup Service. + +* `---remove `: Removes the specified service from the specified node. + +=== Examples + +* The `–fts-add` argument adds the full-text search service to the list of nodes. + +* The `–index-remove` argument removes the index service from the list of nodes. + +* To add the index service to `node2`, use the following command: ++ +---- +couchbase-cli rebalance -c 127.0.0.1 -u Administrator -p password --update-services --index-add node2 +---- + +* To remove the index service from `node3`, use the following command: ++ +---- +couchbase-cli rebalance -c 127.0.0.1 -u Administrator -p password --update-services --index-remove node3 +---- + +* Trying to remove a service that's not present on the node results in an error. +From the previous example, if `node3` was not running the index service, the following error occurs: ++ +---- +ERROR: Node ns_1@node3 does not provide the index service +---- + +For more information about the `couchbase-cli rebalance` command, see xref:cli:cbcli/couchbase-cli-rebalance.adoc[rebalance]. \ No newline at end of file diff --git a/modules/manage/pages/manage-nodes/node-management-overview.adoc b/modules/manage/pages/manage-nodes/node-management-overview.adoc index 7147984023..1ba8bdf4df 100644 --- a/modules/manage/pages/manage-nodes/node-management-overview.adoc +++ b/modules/manage/pages/manage-nodes/node-management-overview.adoc @@ -8,19 +8,20 @@ [#managing-nodes-and-clusters] == Managing Nodes and Clusters -A Couchbase Server _node_ is a physical or virtual machine that hosts a single instance of Couchbase Server. +A Couchbase Server node is a physical or virtual machine that hosts a single instance of Couchbase Server. A conceptual overview is provided in xref:learn:clusters-and-availability/nodes.adoc[Nodes]. The current section provides step-by-step procedures for node-management. This includes: -* _Initializing_ and _provisioning_ a node, thereby making it a one-node cluster. -* _Adding_ and _removing_ nodes from clusters. -* _Listing_ nodes currently in the cluster. -* Performing _failover_ on nodes, when they need to be removed from the cluster. -* _Rebalancing_ the cluster, after a node has been removed for scheduled or otherwise planned maintenance. -* _Recovering_ a node, following failover. -* _Applying_ node-to-node encryption, to ensure security of communications across the cluster. -* _Changing_ the cluster's address family. +* Initializing and provisioning a node, thereby making it a one-node cluster. +* Adding and removing nodes from clusters. +* Listing nodes currently in the cluster. +* Performing failover on nodes, when they need to be removed from the cluster. +* Rebalancing the cluster, after a node has been removed for scheduled or otherwise planned maintenance. +* Recovering a node, following failover. +* Modifying services on nodes, to add or remove Multi-Dimensional Scaling (MDS) services, and rebalancing. +* Applying node-to-node encryption, to ensure security of communications across the cluster. +* Changing the cluster's address family. [#prerequisites] == Prerequisites @@ -33,11 +34,11 @@ For information on starting and stopping Couchbase Server on different platforms == Certificate Management -The examples in this section assume that nodes are protected with the out-of-the-box, _self-signed_ SSL/TLS certificate that is provided with Couchbase Server by default. +The examples in this section assume that nodes are protected with the out-of-the-box, self-signed SSL/TLS certificate that is provided with Couchbase Server by default. This allows nodes to be added to the cluster with no need for explicit certificate-related management; since each node has the same certificate. However, this certificate is only intended for use in pre-production environments. -If, across the cluster, an _authority-signed_ certificate is substituted for the default (as would be required for production deployments), no further node can subsequently be added until a certificate signed by the same authority has been installed on it. +If, across the cluster, an authority-signed certificate is substituted for the default (as would be required for production deployments), no further node can subsequently be added until a certificate signed by the same authority has been installed on it. An attempt incorporate into a cluster a new node that is not appropriately certificate-protected results in an error. For more information, see xref:learn:clusters-and-availability/nodes.adoc#node-certificates[Node Certificates]. @@ -45,7 +46,7 @@ For more information, see xref:learn:clusters-and-availability/nodes.adoc#node-c == Node Management and Community Edition Couchbase has modified the license restrictions to its Community Edition package for Couchbase Server Version 7.0 and higher. -In consequence, the size of an individual cluster running Community Edition is restricted to _five_ nodes. +In consequence, the size of an individual cluster running Community Edition is restricted to 5 nodes. See https://blog.couchbase.com/couchbase-modifies-license-free-community-edition-package/[Couchbase Modifies License of Free Community Edition Package^], for further information on the new restrictions. If an administrator attempts to add a sixth node to a five-node cluster running Community Edition Version 7.0 or higher, the following notification is provided, at the lower-left of Couchbase Web Console: diff --git a/modules/rest-api/pages/rest-cluster-addnodes.adoc b/modules/rest-api/pages/rest-cluster-addnodes.adoc index ea4601dd5d..e16d432419 100644 --- a/modules/rest-api/pages/rest-cluster-addnodes.adoc +++ b/modules/rest-api/pages/rest-cluster-addnodes.adoc @@ -95,7 +95,14 @@ See xref:rest-api:rest-cluster-rebalance.adoc[Rebalancing the Cluster]. [#see-also] == See Also -For a conceptual overview of nodes, including options for adding nodes to clusters, see xref:learn:clusters-and-availability/nodes.adoc[Nodes]. -For information on rebalancing, see xref:rest-api:rest-cluster-rebalance.adoc[Rebalancing the Cluster]. -For information on adding nodes and rebalancing by means of Couchbase Web Console and the CLI, see xref:manage:manage-nodes/add-node-and-rebalance.adoc[Add a Node and Rebalance]. -For information on correctly specifying the node-name on its certificate, see xref:learn:security/certificates.adoc#node-certificate-validation[Node-Certificate Validation]. +* For a conceptual overview of nodes, including options for adding nodes to clusters, see xref:learn:clusters-and-availability/nodes.adoc[Nodes]. + +* For more information about rebalancing using the REST API, see xref:rest-api:rest-cluster-rebalance.adoc[Rebalancing the Cluster]. + +* For more information about adding nodes and rebalancing with the Couchbase Server Web Console and the CLI, see xref:manage:manage-nodes/add-node-and-rebalance.adoc[Add a Node and Rebalance]. + +* For more information about correctly specifying the node-name on its certificate, see xref:learn:security/certificates.adoc#node-certificate-validation[Node-Certificate Validation]. + +* For more information about assigning services to a new single node using the REST API, see xref:rest-api:rest-set-up-services.adoc[Assigning Services to a New Single Node]. + +* For more information about assigning non-Data Services to an existing node using the REST API, see xref:rest-api:rest-set-up-services-existing-nodes.adoc[Assigning Services to an Existing Node]. \ No newline at end of file diff --git a/modules/rest-api/pages/rest-cluster-rebalance.adoc b/modules/rest-api/pages/rest-cluster-rebalance.adoc index a352dd23e2..79ab249706 100644 --- a/modules/rest-api/pages/rest-cluster-rebalance.adoc +++ b/modules/rest-api/pages/rest-cluster-rebalance.adoc @@ -22,6 +22,14 @@ The process occurs while the cluster continues to service requests for data. A conceptual overview of nodes, and how they are combined into clusters, is provided in xref:learn:clusters-and-availability/nodes.adoc[Nodes]. +This REST API reference explains the `POST /controller/rebalance` command usage after: + +* The node additions and removals. +* A graceful failover and recovery. +* A hard failover. + +For more information about using the REST API `POST /controller/rebalance` command to reconfigure (add or remove) the services on the existing nodes of the cluster, see xref:rest-api:rest-set-up-services-existing-nodes.adoc[Assigning Services to an Existing Node]. + [#curl-syntax] == Curl Syntax @@ -236,10 +244,9 @@ curl -X POST -u Administrator:password 'http://10.5.2.54:8091/internalSettings' [#see-also] == See Also -For conceptual information on rebalance, see xref:learn:clusters-and-availability/rebalance.adoc[Rebalance]. -For information on how to retrieve status on an ongoing rebalance, see xref:rest-api:rest-get-rebalance-progress.adoc[Getting Rebalance Progress] and xref:rest-api:rest-get-cluster-tasks.adoc[Getting Cluster Tasks]. - -For conceptual information on hard failover, see xref:learn:clusters-and-availability/hard-failover.adoc[Hard Failover]. -For information on performing a hard failover with the REST API, see xref:rest-api:rest-node-failover.adoc[Failing over Nodes]. -For information on retrieving details of a cluster, including its current nodes, see xref:rest-api:rest-cluster-details.adoc[Viewing Cluster Details]. -For information on obtaining and reading _rebalance reports_, see the xref:rebalance-reference:rebalance-reference.adoc[Rebalance Reference]. +* For conceptual information on rebalance, see xref:learn:clusters-and-availability/rebalance.adoc[Rebalance]. +* For more information on how to retrieve the status of an ongoing rebalance, see xref:rest-api:rest-get-rebalance-progress.adoc[Getting Rebalance Progress] and xref:rest-api:rest-get-cluster-tasks.adoc[Getting Cluster Tasks]. +* For conceptual information on hard failover, see xref:learn:clusters-and-availability/hard-failover.adoc[Hard Failover]. +* For more information on performing a hard failover with the REST API, see xref:rest-api:rest-node-failover.adoc[Failing over Nodes]. +* For more information on retrieving the details of a cluster, including its current nodes, see xref:rest-api:rest-cluster-details.adoc[Viewing Cluster Details]. +* For more information on obtaining and reading rebalance reports, see the xref:rebalance-reference:rebalance-reference.adoc[Rebalance Reference]. diff --git a/modules/rest-api/pages/rest-configure-memory.adoc b/modules/rest-api/pages/rest-configure-memory.adoc index 941bac4fa2..4285f10e8a 100644 --- a/modules/rest-api/pages/rest-configure-memory.adoc +++ b/modules/rest-api/pages/rest-configure-memory.adoc @@ -1,5 +1,5 @@ = Configuring Memory -:description: pass:q[By means of the REST API, custom memory-allocation can be performed per service.] +:description: pass:q[Use REST API to configure custom memory allocation per service.] :page-topic-type: reference :page-aliases: rest-api:rest-node-memory-quota.adoc @@ -16,12 +16,18 @@ POST /pools/default [#description] == Description -Allows a custom memory quota to be established for the Data, Index, Search, Eventing, and Analytics Services. -(The Query and Backup Services do not require a memory allocation.) -If no custom quota is specified for one or more services, those services retain the default allocations: these are, for the Data Service 256 Mb; for the Search Service 256 Mb; for the Index Service 512 Mb; for the Eventing Service 256 Mb; and for the Analytics Service 1024 Mb. -Note that in each case except the Analytics Service, the minimum allowed allocation is 256 Mb: for the Analytics Service, the minimum allowed allocation is 1024 Mb. +Allows a custom memory quota to be established for the Multi-Dimensional Scaling (MDS) services Data, Index, Search, Eventing, and Analytics. +The Query and Backup Services do not require a memory allocation. +If no custom quota was specified during configuration for 1 or more services, those services retain the default allocations. +These are the default allocations: -For information on the maximum memory allocation permitted for a node, see xref:learn:buckets-memory-and-storage/memory.adoc#service-memory-quotas[Service Memory Quotas]. +* 256 Mb for kv (Data Service) +* 256 Mb for fts (Search Service) +* 512 Mb for index (Index Service) +* 256 Mb for eventing (Eventing Service) +* 1024 Mb for cbas (Analytics Service) + +For more information about the maximum memory allocation permitted for a node, see xref:learn:buckets-memory-and-storage/memory.adoc#service-memory-quotas[Service Memory Quotas]. == Curl Syntax @@ -35,15 +41,32 @@ curl -v -X POST http://10.144.220.101:8091/pools/default \ -u : ---- -Note that during the process of provisioning a single-node cluster, `username` and `password` are required after the administrator has established credentials, as described in xref:rest-api:rest-establish-credentials.adoc[Establishing Credentials]. +During the process of provisioning a single-node cluster, `username` and `password` are required after the administrator has established credentials, as explained in xref:rest-api:rest-establish-credentials.adoc[Establishing Credentials]. +[#responses] == Responses -Success returns `200 OK`. -Failure to specify the URI correctly returns `404 Object Not Found`. -Failure to specify a flag correctly returns `400 Bad Request`, and an error message such as: `{"errors":{"cbasMemoryQuo3ta":"Unsupported key"}}`. +|=== +|Value | Description + +| `200 OK` and JSON array containing the expired backups and their details. +| Successful call. + +| `400` +| Invalid parameter. + +| `401 Unauthorized` +| Authorization failure due to incorrect username or password. -If, as part of the provisioning process, a username and password have already been assigned to the cluster, failure to authenticate returns `401 Unauthorized`. +| `403 Forbidden`, plus a JSON message explaining the minimum permissions. +| The provided username has insufficient privileges to call this method. + +| `404 Object Not Found` +| Error in the URI path. + +| `500 Internal Server Error` +| Error in Couchbase Server. +|=== == Example @@ -59,11 +82,12 @@ curl -v -X POST http://10.144.220.101:8091/pools/default \ -d 'cbasMemoryQuota=1024' ---- - == See Also -For information on the maximum memory allocation permitted for a node, see xref:learn:buckets-memory-and-storage/memory.adoc#service-memory-quotas[Service Memory Quotas]. +* For more information about the maximum memory allocation permitted for a node, see xref:learn:buckets-memory-and-storage/memory.adoc#service-memory-quotas[Service Memory Quotas]. + +* For more information about the other aspects of provisioning a single-node cluster, see xref:rest-api:rest-initialize-node.adoc[Initializing a Node], xref:rest-api:rest-name-node.adoc[Naming a Node], xref:rest-name-cluster.adoc[Naming a Cluster], xref:rest-api:rest-set-up-services.adoc[Assigning Services to a New Single Node], and xref:rest-api:rest-establish-credentials.adoc[Establishing Credentials]. -For the other aspects of the provisioning of a single-node cluster, see xref:rest-api:rest-initialize-node.adoc[Initializing a Node], xref:rest-api:rest-name-node.adoc[Naming a Node], xref:rest-name-cluster.adoc[Naming a Cluster], xref:rest-api:rest-set-up-services.adoc[Assigning Services], and xref:rest-api:rest-establish-credentials.adoc[Establishing Credentials]. +* For more information on initialization and provisioning -- using the UI, the CLI, and the REST API -- see xref:manage:manage-nodes/node-management-overview.adoc[Manage Nodes and Clusters]. -For further information on initialization and provisioning -- using the UI, the CLI, and the REST API -- see xref:manage:manage-nodes/node-management-overview.adoc[Manage Nodes and Clusters]. +* For more information about retrieving memory information for services using the REST API, see xref:rest-api:rest-get-memory-information.adoc[Getting Memory Information]. \ No newline at end of file diff --git a/modules/rest-api/pages/rest-establish-credentials.adoc b/modules/rest-api/pages/rest-establish-credentials.adoc index 99c246d347..c12adea564 100644 --- a/modules/rest-api/pages/rest-establish-credentials.adoc +++ b/modules/rest-api/pages/rest-establish-credentials.adoc @@ -21,7 +21,7 @@ The administrator will have Full Admin permissions, but will not appear as a lis The administrator will be able to add other administrators, with specific roles, including administrators with Full Admin permissions: all will appear in the Security window of Couchbase Web Console. During the process of provisioning a new single-node cluster, prior to the establishing of a username and password, no authentication is required to execute the APIs that support provisioning. -This permits xref:rest-api:rest-name-node.adoc[Naming a Node], xref:rest-api:rest-name-cluster.adoc[Naming a Cluster], xref:rest-api:rest-configure-memory.adoc[Configuring Memory], and xref:rest-api:rest-set-up-services.adoc[Assigning Services] to be performed without any username or password being specified. +This permits xref:rest-api:rest-name-node.adoc[Naming a Node], xref:rest-api:rest-name-cluster.adoc[Naming a Cluster], xref:rest-api:rest-configure-memory.adoc[Configuring Memory], and xref:rest-api:rest-set-up-services.adoc[Assigning Services to a New Single Node] to be performed without any username or password being specified. However, once a username and password have been established, all further calls, including those used in provisioning, require authentication. Following the establishment of username and password, the single-node cluster can no longer be initialized; nor can services be assigned to the node. @@ -63,6 +63,6 @@ curl -X POST http://:8091/settings/web \ Couchbase Server roles are described in xref:learn:security/roles.adoc[Roles]. -For each of the specific steps required in the provisioning process, see xref:rest-api:rest-configure-memory.adoc[Configuring Memory], xref:rest-api:rest-name-node.adoc[Naming a Node], xref:rest-name-cluster.adoc[Naming a Cluster], xref:rest-api:rest-set-up-services.adoc[Assigning Services], and xref:rest-api:rest-establish-credentials.adoc[Establishing Credentials]. +For each of the specific steps required in the provisioning process, see xref:rest-api:rest-configure-memory.adoc[Configuring Memory], xref:rest-api:rest-name-node.adoc[Naming a Node], xref:rest-name-cluster.adoc[Naming a Cluster], xref:rest-api:rest-set-up-services.adoc[Assigning Services to a New Single Node], and xref:rest-api:rest-establish-credentials.adoc[Establishing Credentials]. For further information on initialization and provisioning -- using the UI, the CLI, and the REST API -- see xref:manage:manage-nodes/node-management-overview.adoc[Manage Nodes and Clusters]. diff --git a/modules/rest-api/pages/rest-get-memory-information.adoc b/modules/rest-api/pages/rest-get-memory-information.adoc new file mode 100644 index 0000000000..e325687d85 --- /dev/null +++ b/modules/rest-api/pages/rest-get-memory-information.adoc @@ -0,0 +1,107 @@ += Getting Memory Information +:description: pass:q[Use the REST API to retrieve the current memory quota information for services in a cluster.] +:page-toclevels: 2 + +[abstract] +{description} + +[#http-method-and-uri] +== HTTP Method and URI + +---- +GET /pools/default +---- + +[#description] +== Description + +Allows you to view memory quotas for the Multi-Dimensional Scaling (MDS) services Data, Index, Search, Eventing, and Analytics. +The Query and Backup Services do not require a memory allocation. +If no custom quota was specified during configuration for 1 or more services, those services retain the default allocations. +These are the default allocations: + +* 256 Mb for kv (Data Service) +* 512 Mb for index (Index Service) +* 256 Mb for fts (Search Service) +* 256 Mb for eventing (Eventing Service) +* 1024 Mb for cbas (Analytics Service) + +For information about the maximum memory allocation permitted for a node, see xref:learn:buckets-memory-and-storage/memory.adoc#service-memory-quotas[Service Memory Quotas]. + +== curl Syntax + +---- +curl -v -X GET :/pools/default \ + -u : +---- + +== Required Permissions + +`Full Admin` role. + +[#responses] +== Responses + +|=== +|Value | Description + +| `200 OK` and JSON array containing the expired backups and their details. +| Successful call. + +| `400` +| Invalid parameter. + +| `401 Unauthorized` +| Authorization failure due to incorrect username or password. + +| `403 Forbidden`, plus a JSON message explaining the minimum permissions. +| The provided username has insufficient privileges to call this method. + +| `404 Object Not Found` +| Error in the URI path. + +| `500 Internal Server Error` +| Error in Couchbase Server. +|=== + +[#example] +== Example + +The following example retrieves memory quota (per server node) information for each service. + +The output of the REST API call is piped to https://stedolan.github.io/jq/[jq^] and then the output of the jq is piped to grep to retrieve only the memory quota info. + +---- +curl -v -X GET http://127.0.0.1:8091/pools/default -u Administrator:password | jq '.' | grep -i memoryquota +---- + +If the call is successful, the response is similar to the following. + +The memory quota for the Query service is a soft limit target that the garbage collector works to respect, including running more frequently as the limit is approached or crossed. +When set to 0, the soft limit target is disabled, and the garbage collector runs as normal. + +---- +"memoryQuota": 7424, +"queryMemoryQuota": 0, +"indexMemoryQuota": 3243, +"ftsMemoryQuota": 512, +"cbasMemoryQuota": 2218, +"eventingMemoryQuota": 711, +---- + +In the example output, the services memory quotas are listed as follows in these parameters: + +* `memoryQuota` for the Data Service. +* `queryMemoryQuota` for the Query Service. +* `indexMemoryQuota` for the Index Service. +* `ftsMemoryQuota` for the Search Service. +* `cbasMemoryQuota` for the Analytics Service. +* `eventingMemoryQuota` for the Eventing Service. + +== See Also + +* For more information on the maximum memory allocation permitted for a node, see xref:learn:buckets-memory-and-storage/memory.adoc#service-memory-quotas[Service Memory Quotas]. + +* For more information about adding or removing services on an existing node in a cluster using the UI and the REST API, see xref:manage:manage-nodes/modify-services-and-rebalance.adoc[Modify Services and Rebalance]. + +* For more information about configuring memory for services, see xref:rest-api:rest-configure-memory.adoc[Configuring Memory]. \ No newline at end of file diff --git a/modules/rest-api/pages/rest-initialize-cluster.adoc b/modules/rest-api/pages/rest-initialize-cluster.adoc index 6bc4512c6b..423367e499 100644 --- a/modules/rest-api/pages/rest-initialize-cluster.adoc +++ b/modules/rest-api/pages/rest-initialize-cluster.adoc @@ -22,7 +22,7 @@ xref:rest-api:rest-initialize-node.adoc[Initializing a Node], xref:rest-api:rest-establish-credentials.adoc[Establishing Credentials], xref:rest-api:rest-name-node.adoc[Naming a Node], xref:rest-api:rest-configure-memory.adoc[Configuring Memory], -xref:rest-api:rest-set-up-services.adoc[Assigning Services], +xref:rest-api:rest-set-up-services.adoc[Assigning Services to a New Single Node], and xref:rest-api:rest-name-cluster.adoc[Naming a Cluster]. diff --git a/modules/rest-api/pages/rest-initialize-node.adoc b/modules/rest-api/pages/rest-initialize-node.adoc index cc4a68d581..d8514af4fa 100644 --- a/modules/rest-api/pages/rest-initialize-node.adoc +++ b/modules/rest-api/pages/rest-initialize-node.adoc @@ -74,7 +74,7 @@ curl -X POST \ The sequence of tasks divided into _initialization_ and _provisioning_ is explained in xref:rest-api:rest-cluster-init-and-provisioning.adoc[Cluster Initialization and Provisioning]. -For each of the specific steps required in the provisioning process, see xref:rest-api:rest-configure-memory.adoc[Configuring Memory], xref:rest-api:rest-name-node.adoc[Naming a Node], xref:rest-name-cluster.adoc[Naming a Cluster], xref:rest-api:rest-set-up-services.adoc[Assigning Services], and xref:rest-api:rest-establish-credentials.adoc[Establishing Credentials]. +For each of the specific steps required in the provisioning process, see xref:rest-api:rest-configure-memory.adoc[Configuring Memory], xref:rest-api:rest-name-node.adoc[Naming a Node], xref:rest-name-cluster.adoc[Naming a Cluster], xref:rest-api:rest-set-up-services.adoc[Assigning Services to a New Single Node], and xref:rest-api:rest-establish-credentials.adoc[Establishing Credentials]. Information on nodes is provided in xref:learn:clusters-and-availability/nodes.adoc[Nodes]. diff --git a/modules/rest-api/pages/rest-name-cluster.adoc b/modules/rest-api/pages/rest-name-cluster.adoc index e471026962..f67e6203fc 100644 --- a/modules/rest-api/pages/rest-name-cluster.adoc +++ b/modules/rest-api/pages/rest-name-cluster.adoc @@ -55,7 +55,7 @@ curl -X POST http://10.144.220.101:8091/pools/default \ The sequence of tasks divided into _initialization_ and _provisioning_ is explained in xref:rest-api:rest-cluster-init-and-provisioning.adoc[Cluster Initialization and Provisioning]. -For each of the other specific steps required in the provisioning process, see xref:rest-api:rest-configure-memory.adoc[Configuring Memory], xref:rest-api:rest-name-node.adoc[Naming a Node], xref:rest-api:rest-set-up-services.adoc[Assigning Services], and xref:rest-api:rest-establish-credentials.adoc[Establishing Credentials]. +For each of the other specific steps required in the provisioning process, see xref:rest-api:rest-configure-memory.adoc[Configuring Memory], xref:rest-api:rest-name-node.adoc[Naming a Node], xref:rest-api:rest-set-up-services.adoc[Assigning Services to a New Single Node], and xref:rest-api:rest-establish-credentials.adoc[Establishing Credentials]. For general information on naming, see xref:learn:clusters-and-availability/nodes.adoc#naming-clusters-and-nodes[Naming Clusters and Nodes]. diff --git a/modules/rest-api/pages/rest-name-node.adoc b/modules/rest-api/pages/rest-name-node.adoc index 2855f8c89c..33be05bec9 100644 --- a/modules/rest-api/pages/rest-name-node.adoc +++ b/modules/rest-api/pages/rest-name-node.adoc @@ -59,7 +59,7 @@ http://10.144.220.101:8091/node/controller/rename \ The sequence of tasks divided into _initialization_ and _provisioning_ is explained in xref:rest-api:rest-cluster-init-and-provisioning.adoc[Cluster Initialization and Provisioning]. -For each of the other specific steps required in the provisioning process, see xref:rest-api:rest-configure-memory.adoc[Configuring Memory], xref:rest-name-cluster.adoc[Naming a Cluster], xref:rest-api:rest-set-up-services.adoc[Assigning Services], and xref:rest-api:rest-establish-credentials.adoc[Establishing Credentials]. +For each of the other specific steps required in the provisioning process, see xref:rest-api:rest-configure-memory.adoc[Configuring Memory], xref:rest-name-cluster.adoc[Naming a Cluster], xref:rest-api:rest-set-up-services.adoc[Assigning Services to a New Single Node], and xref:rest-api:rest-establish-credentials.adoc[Establishing Credentials]. For general information on naming, see xref:learn:clusters-and-availability/nodes.adoc#naming-clusters-and-nodes[Naming Clusters and Nodes]. diff --git a/modules/rest-api/pages/rest-set-up-services-existing-nodes.adoc b/modules/rest-api/pages/rest-set-up-services-existing-nodes.adoc new file mode 100644 index 0000000000..d82d19de57 --- /dev/null +++ b/modules/rest-api/pages/rest-set-up-services-existing-nodes.adoc @@ -0,0 +1,150 @@ += Assigning Services on an Existing Node + +:description: pass:q[Use the REST API to assign or remove services on an existing node in a cluster, and rebalance the cluster.] +:page-topic-type: reference + +[abstract] +{description} + +[#http-method-and-uri] + +=== HTTP Method and URI + +---- +POST /controller/rebalance +---- + +== Description + +Use the following REST API to add or remove non-Data services on the existing nodes in a cluster. +Then trigger a rebalance operation. + +[NOTE] +==== +* Follow the xref:manage:manage-nodes/modify-services-and-rebalance.adoc#prerequisites[Prerequisites] in xref:manage:manage-nodes/modify-services-and-rebalance.adoc[Modify Services and Rebalance] before using the REST API. + +* For information about using the `POST /controller/rebalance` REST API to rebalance after node additions and removals, after a graceful failover and recovery, and after a hard failover, see xref:rest-api:rest-cluster-rebalance.adoc[Rebalancing the Cluster]. +==== + +=== curl Syntax + +You must specify all known nodes in the cluster and the necessary cluster services topology. + +---- +curl -v -X POST -u [admin]:[password] \ +http://[localhost]:8091/controller/rebalance \ +[-d knownNodes | -d topology] +---- + +These are services related parameters: + +* `knownNodes`: The value is a list containing all nodes that are in the cluster at the start of the rebalance process. + +* `topology`: The value is a list of services and the nodes in which services must reside, after the rebalance operation completes. ++ +The value is `topology[]=`. ++ +Where, + +** `` can be one of the following services: + +*** `fts` for the Search Service. +*** `index` for the Index Service. +*** `n1ql` for the Query Service. +*** `backup` for the Backup Service. +*** `cbas` for the Analytics Service. + +** `` indicate the nodes on which the `` must reside in after the rebalance operation is complete. +The node names are separated by commas. ++ +If a `` is running on any node in the cluster that's not in this list, Couchbase Server removes that `` during the rebalance operation. +For more information, see xref:rest-api:rest-cluster-rebalance.adoc[Rebalancing the Cluster]. + +** For example, `-d topology[]=otp_node1,otp_node2`. + +NOTE: The rebalance occurs even if no `` is changed. + +Each service you assign uses the memory quota currently configured for that service on the cluster. +You can modify the services memory quotas, if needed, prior to running the services reconfiguration command for the cluster nodes. + +For information about allocating memory per service, see xref:rest-api:rest-configure-memory.adoc[Configuring Memory]. + +To fetch the details of current memory allocation for each service, see xref:rest-api:rest-get-memory-information.adoc[Getting Memory Information]. + +== Required Permissions + +`Full Admin` role. + +[#responses] +== Responses + +|=== +|Value | Description + +| `200 OK` and JSON array containing the expired backups and their details. +| Successful call. + +| `400` +| Invalid parameter. + +| `400 Object Not found` +| The repository in the endpoint URI does not exist. + +| `401 Unauthorized` +| Authorization failure due to incorrect username or password. + +| `403 Forbidden`, plus a JSON message explaining the minimum permissions. +| The provided username has insufficient privileges to call this method. + +| `404 Object Not Found` +| Error in the URI path. + +| `500 Internal Server Error` +| Error in Couchbase Server. +|=== + +=== Example + +For example, a cluster with 5 nodes could have the following service configuration: + +* 172.23.108.69 – Data, Index, Query +* 172.23.108.70 – Data, Search, Query +* 172.23.108.71 – Query +* 172.23.108.72 – Backup +* 172.23.108.73 – Analytics + +You could reconfigure the services on the existing nodes to match the following: + +* 172.23.108.69 – Data, Query +* 172.23.108.70 – Data, Index +* 172.23.108.71 – Index, Query +* 172.23.108.72 – Search, Query +* 172.23.108.73 – Analytics + +Because the Backup service is not in the topology list, it will be removed from the cluster. +After the topology change and rebalance, two nodes (172.23.108.71 and 172.23.108.72) that previously had no services using the index disk storage path will now have services using it. + +To reconfigure the 5 nodes to the new topology, run the following command: + +---- +curl -u Administrator:password -X POST 172.23.108.69:8091/controller/rebalance \ +-d "knownNodes=ns_1@172.23.108.69,ns_1@172.23.108.70,ns_1@172.23.108.71,ns_1@172.23.108.72,ns_1@172.23.108.73" \ +-d "topology[index]=ns_1@172.23.108.70,ns_1@172.23.108.71" \ +-d "topology[fts]=ns_1@172.23.108.72" \ +-d "topology[n1ql]=ns_1@172.23.108.69,ns_1@172.23.108.71,ns_1@172.23.108.72" \ +-d "topology[cbas]=ns_1@172.23.108.73" +---- + +== See Also + +* To retrieve the list of services running on a node, see xref:rest-api:rest-list-node-services.adoc[Listing Node Services]. + +* For more information about allocating memory per service, see xref:rest-api:rest-configure-memory.adoc[Configuring Memory]. + +* To fetch the details of current memory allocation for each service, see xref:rest-api:rest-get-memory-information.adoc[Getting Memory Information]. + +* For more information about assigning services to existing nodes from the UI and CLI, see xref:manage:manage-nodes/modify-services-and-rebalance.adoc[Modify Services and Rebalance]. + +* For more information about assigning services to a new node, see xref:rest-api:rest-set-up-services.adoc[Assigning Services to a New Single Node]. + +* For more information about rebalancing, see xref:rest-api:rest-cluster-rebalance.adoc[Rebalancing the Cluster]. \ No newline at end of file diff --git a/modules/rest-api/pages/rest-set-up-services.adoc b/modules/rest-api/pages/rest-set-up-services.adoc index be4e9dd260..47ba730464 100644 --- a/modules/rest-api/pages/rest-set-up-services.adoc +++ b/modules/rest-api/pages/rest-set-up-services.adoc @@ -1,6 +1,6 @@ -= Assigning Services += Assigning Services to a New Single Node -:description: pass:q[By means of the REST API, services can be assigned to a new, single-node cluster.] +:description: pass:q[Use the REST API to assign services to a new, single-node cluster.] :page-topic-type: reference [abstract] @@ -68,9 +68,11 @@ curl -X POST http://10.144.220.101:8091/node/controller/setupServices \ == See Also -The sequence of tasks divided into _initialization_ and _provisioning_ is explained in xref:rest-api:rest-cluster-init-and-provisioning.adoc[Cluster Initialization and Provisioning]. +* For more information about cluster initialization and provisioning with the REST API, see xref:rest-api:rest-cluster-init-and-provisioning.adoc[Cluster Initialization and Provisioning]. -For each of the other specific steps required in the provisioning process, see xref:rest-api:rest-configure-memory.adoc[Configuring Memory], xref:rest-api:rest-name-node.adoc[Naming a Node], xref:rest-name-cluster.adoc[Naming a Cluster], and xref:rest-api:rest-establish-credentials.adoc[Establishing Credentials]. +* For each of the other specific steps required in the provisioning process, see xref:rest-api:rest-configure-memory.adoc[Configuring Memory], xref:rest-api:rest-name-node.adoc[Naming a Node], xref:rest-name-cluster.adoc[Naming a Cluster], and xref:rest-api:rest-establish-credentials.adoc[Establishing Credentials]. Specifically, see xref:rest-api:rest-configure-memory.adoc[Configuring Memory] for information on allocating memory per service, and on the default minimum values per service. -For further information on initialization and provisioning -- using the UI, the CLI, and the REST API -- see xref:manage:manage-nodes/node-management-overview.adoc[Manage Nodes and Clusters]. +* For more information on initialization and provisioning -- using the UI, the CLI, and the REST API -- see xref:manage:manage-nodes/node-management-overview.adoc[Manage Nodes and Clusters]. + +* For more information about assigning non-Data Services to an existing node using the REST API, see xref:rest-api:rest-set-up-services-existing-nodes.adoc[Assigning Services to an Existing Node].