How can I make MountainDuck work with macOS bundles?

[crenz]

tl;dr: How can I reliably store and work with GB-sized Logic Pro projects using MountainDuck with WebDAV? Long explanation follows.

macOS offers a special project format called bundles. These are essentially directories with an extension name, containing subdirectories and files. You can try it with a quick mkdir Test.app in the Terminal. In the finder, that directory will show as a (0 KB size) Application rather than a directory, and you can right-click it to show the contents.

A number of macOS applications use bundles to store their projects. In my case, I use Logic Pro a lot, which stores configuration data, note data as well as audio files all within a bundle.

I'm using MountainDuck with a NextCloud instance, accessing it via WebDAV using the new integrated mode (which I love! Thanks for implementing it!). When I place a Test.logicx bundle in a cloud directory, MountainDuck will treat it as a file called Test.logicx.pkgf and try to sync it with the server.

The problem is that these bundles (for me) typically are several GB large, containing hundreds of files. First of all, the upload often doesn't work – for example, I tried it with a smaller 3 GB sized project, and during the upload, MountainDuck created 15 (!) conflicting versions as well as uploaded only 156 MB in what it considered the final file. Even worse, if I manage to upload it, then just open it with Logic Pro and close it, MountainDuck will aim to re-upload the whole 3 GB again. The underlying reason is that within the bundle, a small property file gets updated (containing window positions etc.) – that's about a 100 KB worth of changes.

Looking at my Nextcloud instance, I found that instead of directories, MountainDuck seems to create .pkgf files. Confusingly enough, in my local MountainDuck-managed folder, these are directories that can be entered on the CLI, but not be read (ls: fts_read: Operation timed out).

I regularly collaborate with customers on Logic projects, and currently I use Dropbox for this purpose. Dropbox actually applies an - on the surface - seemingly dumb but highly effective strategy to deal with bundles: It treats them just as regular directories with subdirectories and files inside, rather than as a single file. That means when I put a large Logic project into my Dropbox, I'll be shown there are hundreds of files to be synced. Online, they show as regular directories. But the thing is: it works, and it works highly reliably (unless people work on the same project at the same time - you have to avoid that).

So, finally, my question is: Is there any way to make MountainDuck treat macOS bundles just as regular directories and sync them like any other file, without any magic happening behind the scenes?

[dkocher]

The FileProvider API implementation in Mountain Duck that is Integrated connect mode sets NSExtensionFileProviderWantsFlattenedPackages which causes package files as used in .pages, .numbers, .keynote documents and others to be treated as single files for possibly easier synchronization with cloud storage. This should allow for faster synchronization for smaller packages as only a single file must be uploaded but may cause the adverse you describe when only parts of the package file change. We will check if we want to change the configuration to retain package files as is.

[crenz]

Thanks, @dkocher ! I'm assuming this does not only affect Integrated connect mode. Using Intelligent Synchronization, I was unable to sync large packages as well.

It would be great to have it at least as an option (although perhaps it might confuse novice users), I think this would also benefit users with documents that contain lots of images (such as the Pages or Keynote documents that you gave as examples). The only issue is it would require complete resynchronization.