[RFC] Branching

[tylandavis]

Background

Payload has draft / published states as well as a full version history for any given document, but sometimes you might need multiple parallels tracks for content, which can span across multiple documents at the same time.
For example, let's say you are building a CMS for a large flower shop.
You might want to prepare multiple campaigns which will update your website and native app, in parallel:

• A Christmas campaign
• A Valentine's Day campaign
• An Easter campaign

You want to begin working on this year's Valentine's Day campaign in October, well in advance. This year is going to be a big one. But your website is set up for Halloween currently, and you have a bunch of draft content staged for the Christmas campaign that you don't want to lose!

• Have your main website run off the main branch, looking like Halloween
• Create a branch for Christmas, and have all of your editors uploading content / tweaking homepage messaging / etc. for Christmas
• In parallel, have a Valentine's Day branch created which is beginning to take updates as well

Once Halloween is over, all you'd need to do is merge your Christmas branch with the main branch - then everything's live!
And after Christmas, the same thing is possible - continue to have your editors tweak the Valentine's branch, getting it looking perfectly. Once it's done, just merge the Valentine's branch and everything goes live!

How it would work

Right now, for all version-enabled collections in Payload, we maintain two "tables". The first is the main document table, and one is a _versions table.
The main document table would be unchanged from this work and reflect the main branch. But the _versions table would get some updates.

A new branch property

Building on Payload's existing _versions table, we could augment each saved version with a new branch property, which would track which branch a given version is associated to. Then, as you're browsing the admin UI or fetching content via APIs, you could specify which branch you'd like to retrieve content from. If you do not specify a branch, you'll get back main content (how it works now). But if you specify a branch, then we'll only look for versions of that matching branch name.

A new merge view

Editors will need a way to view all changes from branch to branch in one place as they merge. They should be able to collapse certain documents, leave feedback on certain changes, etc. - all in a newly designed and built view within Payload.

An expanded versions diff view

In Payload's existing Versions Diff view, editors will want to be able to easily compare changes between versions on different branches. We will want to design UX shortcuts that allow editors to accomplish this.

Possible config implementation

  collections: [
    {
      slug: 'pages',
      versions: {
        drafts: {
          // on by default for all draft-enabled collections,
          // if branching is enabled Payload-wide
          // but you can opt-out of certain collections by setting this to false
          branching: true,
        }
      } 
    }
  ],
  versions: {
    branching: true,
    branching: {
      // as necessary, we can allow branching to accept other args
    }
  }
})

Open questions

• What if I want to stage documents on a branch as published? How can I prevent my users from reading published docs if my access control is set up just to restrict _status: 'draft' documents from public visibility?
• Can I cherry-pick changes?
• Can I leave comments on branches?
• What does a branch diff view look like?
• Who can merge? Do you need to have access to every document in order to merge?
• Do we retain old branches? What does this look like?
• Who is allowed to create branches?
• In our existing version diff view, how can I compare a version from Branch A to a version from Branch B?
• Scenario:
  • I create a Christmas branch, making lots of updates
  • I merge Christmas over top of prod
  • How do I go back to the point in time before I merged Christmas?
    • Does this imply some sort of content-wide checkpoint system?
    • Do we want to have "merge" messages associated to merges?
• Can we have documents that exist only in certain branches? => We’d need to be able to filter by branch in, say, the list view => Do we have a branch selector permanently visible throughout the admin panel?

Tracking changes over time

Do we need a "background" collection to track all create, update, and delete changes made to branches? This collection could store metadata of changes such as a timestamp, an ID of the version / parent doc, and similar. This might improve the queries needed to be made while producing a "diff" view or similar. But it could be duplicative, because we might be able to infer this level of detail as it is now, solely from querying the _versions tables.
This background collection might be necessary in order to store additional information such as:

• when a branch was created
• who created a branch
• when a branch was merged
• "snapshots" of pre-merge states or similar

We'd love to hear your thoughts, comments, and questions about this feature!

[cbratschi]

In such cases we often create a new staging environment which is a clone of the current production version. Quite often not only the content gets modified, the data models and frontend rendering changes as well e.g. for Christmas you need some new fields in collections. Doing this on the production system would be too dangerous.

However, the merging use case is a big problem: how do we get production changes since the cloning together with the new content in staging? If you build such a feature we would like to be able to merge two separate Payload instances. This would be a very powerful feature beyond features of the import/export plugin.

[hassanthegooner]

Will this support multi-tenancy?

[hades200082]

How would previews be handled in this?