Could not edit Microsoft Office files from mountpoint

[ikmak]

Everything works fine except editing Microsoft Office files(e.g. Word, Excel, PowerPoint) from mountpoint.
It can be edited if the file open directly from source folder, but when opening from mountpoint the Microsoft Office shows "readonly".

• NFS Server: macOS, ./example/osnfs
• NFS Client: macOS, mount

[ikmak]

@willscott
I've tried to mount with options locallocks, and it looks better.
But there are still some problems.
Word docx(doc) files could be saved with some wired temporary directories like abc.docx.sb-xxxxxx-xxxxx
Excel xlsx(xls) files could be saved without any sb temporary directory
PowerPoint pptx(ppt) files could be created but couldn't be saved.

• NFS Server: macOS, sudo ./example/osnfs /path/to/files/ 45678
• NFS Client: macOS, sudo mount_nfs -vvv -o port=45678,mountport=45678,proto=tcp,nosuid,sync,locallocks "localhost:/nfs" /path/to/mount

[willscott]

I wonder if #72 was the underlying issue behind this

can you try again with the current version?

[ikmak]

@willscott
Same...

[ncw]

Does anyone have a clear idea what is happening here? This is affecting rclone users in rclone/rclone#7973 in exactly the same way.

Putting nolock or locallocks doesn't fix the problem though - the symptoms are as described above #54 (comment)

Is this a problem because the nfs server doesn't implement locking? If so could we implement a skeleton locking system?

I notice that memfs has a stubbed out Lock and Unlock which don't appear to be called anywhere

go-nfs/helpers/memfs/memfs.go

Lines 335 to 343 in 91bc389

	// Lock is a no-op in memfs.
	func (f *file) Lock() error {
	return nil
	}
	// Unlock is a no-op in memfs.
	func (f *file) Unlock() error {
	return nil
	}

[willscott]

@ncw - the reason memfs does not implement lock / unlock is because the accompanying 'Network Lock Manager (NLM)
protocol' is not implemented here. The protocol is documented in RFC 1813 - but it will be a bigger effort than the method you point to.

[ncw]

Thanks @willscott

What do you think the best way forward with this issue is? Rclone can't support locking so how can we tell the clients not to try?

[willscott]

• If we can get logging of the NFS calls made while opening a word document, I'm not totally convinced that the underlying issue is lack of the locking protocol (especially since it has been observed in cases where the mount is done with local / no locks). If there's some other NFS call that is giving an error or otherwise appears to be acting in a way that might be suspicious I don't know if that's been fully ruled out at this point.

• Otherwise, the next step may be to try to scaffold enough of NLMv4 to accept lock/test calls and see if that changes behavior. https://pubs.opengroup.org/onlinepubs/9629799/chap14.htm

[ncw]

• If we can get logging of the NFS calls made while opening a word document, I'm not totally convinced that the underlying issue is lack of the locking protocol (especially since it has been observed in cases where the mount is done with local / no locks). If there's some other NFS call that is giving an error or otherwise appears to be acting in a way that might be suspicious I don't know if that's been fully ruled out at this point.

What is the best way of doing this? The logging in the library doesn't seem to do a full rpc in/out trace or at least I haven't figured out how to make it do that if it can.

I'm not 100% convinced that this problem is to do with locking so tracing sound like a great idea

• Otherwise, the next step may be to try to scaffold enough of NLMv4 to accept lock/test calls and see if that changes behavior. https://pubs.opengroup.org/onlinepubs/9629799/chap14.htm

It doesn't look too complicated...

[willscott]

There's a log trace on each request: https://github.com/willscott/go-nfs/blob/master/conn.go#L62

[willscott]

There's a major issue with NLM in this context - which is that from what i can tell mount'ing of NFS (the clients in Mac and Linux at least) don't allow specification of an explicit NLM port. As such, the server needs to be running in the standard privileged context where portmap is running and used to discover the port the lock manager is running on. I don't see a way to allow interaction with the locking sideband protocol with the same explicit mount-time behavior as with mountd and nfs itself

[ncw]

There's a log trace on each request: https://github.com/willscott/go-nfs/blob/master/conn.go#L62

Here is a trace of those log statements

nfs-logs.txt

I extracted these from the rclone log which gives a bit more context

Terminal.Saved.Output (2).txt

I did not notice this line before

2024/07/30 17:58:02 DEBUG : [NFS DEBUG] failing create to indicate lack of support for 'exclusive' mode.

I wonder if that is the problem? Can we just ignore the exclusive mode?

That comes from here

go-nfs/nfs_oncreate.go

Lines 36 to 44 in 91bc389

	} else if how == createModeExclusive {
	// read createverf3
	var verf [8]byte
	if err := xdr.Read(w.req.Body, &verf); err != nil {
	return &NFSStatusError{NFSStatusInval, err}
	}
	Log.Errorf("failing create to indicate lack of support for 'exclusive' mode.")
	// TODO: support 'exclusive' mode.
	return &NFSStatusError{NFSStatusNotSupp, os.ErrPermission}

[ncw]

I'm attempting an experiment using this patch. I've made an rclone binary with this patch and sent it to the user with the mac (not me!).

diff --git a/nfs_oncreate.go b/nfs_oncreate.go
index 63d7901..262eb8f 100644
--- a/nfs_oncreate.go
+++ b/nfs_oncreate.go
@@ -39,9 +39,8 @@ func onCreate(ctx context.Context, w *response, userHandle Handler) error {
                if err := xdr.Read(w.req.Body, &verf); err != nil {
                        return &NFSStatusError{NFSStatusInval, err}
                }
-               Log.Errorf("failing create to indicate lack of support for 'exclusive' mode.")
-               // TODO: support 'exclusive' mode.
-               return &NFSStatusError{NFSStatusNotSupp, os.ErrPermission}
+               // We ignore the createverf3 which is the key to the lock
+               Log.Errorf("ignoring 'exclusive' mode on create file.")
        } else {
                // invalid
                return &NFSStatusError{NFSStatusNotSupp, os.ErrInvalid}
@@ -88,9 +87,11 @@ func onCreate(ctx context.Context, w *response, userHandle Handler) error {
 
        fp := userHandle.ToHandle(fs, newFile)
        changer := userHandle.Change(fs)
-       if err := attrs.Apply(changer, fs, newFilePath); err != nil {
-               Log.Errorf("Error applying attributes: %v\n", err)
-               return &NFSStatusError{NFSStatusIO, err}
+       if attrs != nil {
+               if err := attrs.Apply(changer, fs, newFilePath); err != nil {
+                       Log.Errorf("Error applying attributes: %v\n", err)
+                       return &NFSStatusError{NFSStatusIO, err}
+               }
        }
 
        writer := bytes.NewBuffer([]byte{})

Which I think should work (cross fingers!)

There was one thing I didn't understand in that code though - the code appears to assume that the file already exists (ie has a handle and can have Stat called on it without an error) whereas the RFC doesn't say anything about the file existing or not. I've probably mis-understood the code or the RFC as I'm not very familiar with either but I just thought I'd mention it.

[willscott]

• I'm working on implementing the expected verifier for exclusive mode semantics. I'll try to have that up in the next day or two
• The attempt at 'Stat' that you mention is used to not attempt creation if the file exists already as should be done in 'Guarded' mode. The call to Create is still done after the stat call unless the guarded mode failure is returned.

[willscott]

@ncw perhaps also try with my branch at https://github.com/willscott/go-nfs/tree/feat/exclusive-create and see if it makes a difference