Need to implement WAIT , tips ?

[bklooste]

Have read the other thread but i need this anyway (not immediately but soon), don't need other blocking operations just WAIT

My options are

• new ConnectionMultiplexer possibly in a separate domain , i see people have issues but try to disable thread steal and set higher sync time outs. Look at stalling code in stack exchange
• Try get a connection from the lib and just send the commands.
• fork stack exchange , make a change there.

Just after some tips / suggestions of the best approach.

[mgravell]

We know that WAIT is a problem. We have investigated some alternative internal implementations that will enable WAIT and other blocking operations - essentially, switching to a connection pool internally. This also resolves a number of other outstanding problems. The prototype was very successful - but we have not yet merged this work, because: it takes time, and this isn't our day jobs. I have a few other plates that I'm spinning at the moment, but this is very high on my list.

Blocking operations are anathema to the current implementation; there are no supported workarounds; you can hack them in via Execute etc, but: this is not advisory, and no support will be offered to make it work, because it cannot work reliably.

My advice: wait

[bklooste]

Thank you much appreciated. I presume this will be a major release with a long cycle ?

[mgravell]

No, I was going to do this as 2.2, and the proof-of-concept is complete - it just needs integrating and testing, but: this isn't my only project and isn't my day job, so: it is tied on my availability. It is something I really want to get in, because it simplifies the IO path hugely, which should resolve a ton a network pain points.

[MikaelElkiaer]

Does this mean that there are currently no means of ensuring replication?

I am planning to set up master-replicant with sentinels. I suspect I have some data that I need to ensure is the same no matter which node you reach. It seems this is only possible with WAIT.

Is there anything going against an approach using a lua script to perform synchronous operations - i. e. using WAIT.

[mgravell]

That Lua script will block your entire server, and could even get forcibly terminated. I don't recommend that.

…

On Thu, 19 Aug 2021, 19:19 Mikael Elkiær, ***@***.***> wrote: Does this mean that there are currently no means of ensuring replication? I am planning to set up master-replicant with sentinels. I suspect I have some data that I need to ensure is the same no matter which node you reach. It seems this is only possible with WAIT. Is there anything going against an approach using a lua script to perform synchronous operations - i. e. using WAIT. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#1474 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAEHMGXDM6NWWPNRQEKFTLT5VDKJANCNFSM4NLQYN3A> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email> .

[MikaelElkiaer]

That Lua script will block your entire server, and could even get forcibly
terminated. I don't recommend that.

Ok, if that is more or less ruled out, then what else is there left to do?
Even when sticking with DemandMaster, a failover will not be consistent either.
Is there then only going full cluster and manually write all masters, while keeping DemandMaster?

Is this a problem of client and/or server blocking? I'd assume that server blocking always would be possible despite any limitations in the client - what am I missing here?

Edit: I see now that WAIT only knows which writes to confirm based on the current connection, and I guess that in a multiplexer scenario it would be hard to discern which writes to confirm, unless it should be all of them. However, in the context of a transaction (i. e. MULTI), wouldn't the WAIT command only confirm the writes within that transaction?

[MikaelElkiaer]

@mgravell Sorry for bumping, but I was wondering if you could come with some clarification regarding the last bit of my previous post. Wouldn't it be possible to do WAIT within the context of a MULTI - even in a multiplexer scenario?

[mgravell]

Honestly, until I get the time to implement pooled connections: I am not, in any scenario, going to advocate anything that introduces a blockage at the server, no matter how transient. I can see many ways it go go badly. Are there scenarios where it might work acceptably under modest load? Possibly, but I'm not going to try and think through what the limits are.

[MikaelElkiaer]

Honestly, until I get the time to implement pooled connections: I am not, in any scenario, going to advocate anything that introduces a blockage at the server, no matter how transient. I can see many ways it go go badly. Are there scenarios where it might work acceptably under modest load? Possibly, but I'm not going to try and think through what the limits are.

Fair enough. Thanks for the reply, will keep an eye out for connection pooling.