Asynchronous functions / callback support

[caoimhebyrne]

It would be amazing if we could call asynchronous Swift functions from Rust.

The only problem is: _@cdecl doesn't allow us to mark our functions as asynchronous:

src-swift/lib.swift

@_cdecl("my_async_fn") // ERROR: _@cdecl global function cannot be asynchronous
func myAsyncFunc() async {
    print("Hello, world!")
}

Since that's not possible, it would be nice if we could have better support for closures:

src-swift/lib.swift

@_cdecl("my_closure_func")
func myClosureFunc(closure: @escaping @convention(c) () -> Void) {
    closure()
}

src/main.rs

use swift_rs::{swift, SRClosure};

swift!(pub(crate) fn my_closure_func(SRClosure<Void>));

fn main() {
    let closure: SRClosure<Void> = || {
        println!("Hello!!");
    }.into();

    my_closure_func(closure);
}

The Rust code above is just a proof-of-concept, and I'm not sure if everything I described there is possible, since we can't just pass a pointer to the callback around (from my understanding at least). We may have to implement something like this C-callbacks example from the Rustonomicon (with some sort of Swift code generator?)

[caoimhebyrne]

For some extra context, I implemented a workaround in one of my own crates for calling methods with closures, but it's messy and not ideal:

@_cdecl("my_func")
func myFunc() -> String {
    let semaphore = DispatchSemaphore(value: 0)
    var returnValue = ""

    // getStringValue is not real, but has a signature something like this:
    // func getStringValue(closure: @escaping (String) -> Void)
    getStringValue() { (value) in   
        returnValue = value
        semaphore.signal()
    }

    semaphore.wait()
    return value
}

Being able to use closures in Rust directly, or even better, do some magic to allow asynchronous functions, would be better than this approach.

[Brendonovich]

since we can't just pass a pointer to the callback around

I think you'd be surprised 😉

SRClosure is a really smart idea and may actually work, since @lucasfernog already did something similar for Tauri

type PluginMessageCallbackFn = unsafe extern "C" fn(c_int, c_int, *const c_char);
pub struct PluginMessageCallback(pub PluginMessageCallbackFn);

impl<'a> SwiftArg<'a> for PluginMessageCallback {
  type ArgType = PluginMessageCallbackFn;

  unsafe fn as_arg(&'a self) -> Self::ArgType {
    self.0
  }
}

I could even try an SRFuture that gets instantiated with an async block in Swift and converts to a Future in Rust, but I'm not 100% confident about that. In theory it'd just be a matter of handling success + failure closures.

Duplicate of #29, but I'll keep this open since it's more detailed.

[caoimhebyrne]

I could even try an SRFuture that gets instantiated with an async block in Swift and converts to a Future in Rust, but I'm not 100% confident about that. In theory it'd just be a matter of handling success + failure closures.

That would be amazing, I would love to contribute but this level of Rust FFI is out of my league :P

[drewcrawford]

While I'm in the area, braindump on this.

The "right way" to call an async function from Rust is:

//consider an async function with arg and return types
func foo(arg: Int) async -> String {
    arg.description
}

//we need a function signature that works in the C ABI
@_cdecl("fooExported")
public func fooExported(arg: Int, completion: @Sendable @convention(c)(String) -> ()) {
    /*
     In general, async functions need to be called from a Swift executor.
     This arbitrary Rust thread is probably not on one, hence the use of Task.  As a corollary
     we can't return a value to caller in the normal C way.
     
     The impulse to solve via semaphore is well-intentioned, however Swift's design is such that ye arbitrary
     Swift code is probably running on an executor thread, and if you block an executor thread your program may deadlock.  Therefore a pattern guaranteed to not deadlock is usually desired.
     */
    Task {
        let r = await foo(arg: 2)
        completion(r)
    }
}

This then raises the question about how to implement the completion in Rust. In general it needs to be an extern "C" global function, but then how do we relate the completion with a particular request? The answer is we usually need to smuggle an opaque context:

//consider an async function with arg and return types
func foo(arg: Int) async -> String {
    arg.description
}

//we need a function signature that works in the C ABI
@_cdecl("fooExported")
public func fooExported(arg: Int, context: UInt64, completion: @Sendable @convention(c)(SRString,UInt64) -> ()) {

    Task {
        let r = await foo(arg: 2)
        completion(SRString(r), context)
    }
}

On Swift side, context is an opaque value. On Rust side, we can choose some value to smuggle in via Box::into_raw and access from the completion handler with Box::from_raw. Box is ok for this code where we have 1:1 between call and completion and thus we can transfer ownership from call-side to complete-side. In cases where there is more than one completion per caller we need something else.

Anyway, what value do we choose to smuggle? I use the continue crate for this. The key idea is

 async fn foo(arg: i64) -> String {
    let (sender,receiver) = continuation();
    let boxed_sender = Box::new(sender);
    unsafe {
        fooExported(1337, Box::into_raw(boxed_sender) as u64, completion)
    }
    receiver.await
}

extern "C" fn completion(string: SRString, context: u64) {
    let unbox = unsafe {Box::from_raw(context as *mut Sender<String>)};
    unbox.send(string.to_string());
}

Ideally, I'd like to improve this boilerplate in some way by contributing some gadget to this crate. But it is not obvious to me how to design a gadget that works well:

1. Box is often sensible but occasionally terrible, in a way tough to abstract over
2. Seems difficult to avoid trampoline functions like fooExported here, and automatically code-generating C-ABI Swift functions overly expands the scope of the crate
3. Gadgets that are primarily interested in how to express the callback type feel relatively unhelpful as ultimately it's written in two languages that aren't reconciled

It may be the best way to do this is the kind of manually-written pattern here.