Reconcile thread model for clients on the top of Rust Client

#1

According to the core client proposal, we plan to maintain three native implementations of TiKV clients, Go Client, Java Client and Rust Client. Then other clients could be built on the top of the Rust Client.

However, while walking through the implementation of C++ Client, I found that we actually block on the async method call when implement the C++ version API.

From my understanding this likely blocks all threads if network suffers while it should have suspended the thread on IO waiting.

Is it a problem in production? What is the best practice reconcile thread model between Rust and other languages when we implement clients in other languages? Does Python Client also suffer this issue?

1 Like
Request a new category for tikv-client
#2

The python client provides asynchrous interfaces.

I agree it is not a good idea for the C++ client to block on RPC. Maybe it is viable to use std::future in C++.

#3

Is it real asynchronous? While I was trying to implement Java Client on the top of the Rust one, I thought about wrapping an asynchronous interface based on blocking call within the Rust side, e.g.,

return CompletableFuture.supplyAsync(() -> nativeCall(), executorService);

fn nativeCall() -> ... {
  block_on(...);
  return ...;
}

which actually consumes a thread but fits in async interface. It suffers the same problem as mentioned above.

The solution I found is either the pair of languages share the same thread model, or we build a thin communication layer like mailbox implemented in Spark, in order to fire an async call within the Rust side, and register a callback handle which is called on completed. We cannot inline the callback but use a mailbox middleware because JNIEnv and jobject are not thread-safe.

What is the Python case?

1 Like
#4

In client-py, we would instantiate a Python’s awaitable object(PyCoroutine) which implements PyAsyncProtocol in rust, so it’s fully asynchronous. But the synchronous API of client-py is implemented by blocking the asynchronous API.

    pub fn get(&self, key: Vec<u8>, cf: &str) -> PyCoroutine {
        // ... 
        PyCoroutine::new(async move {
            // get vaule via client
            Ok(val)
        })
    }
#5

std::future is actually a wrapper of std::thread.

So far, there isn’t a unified way to write asynchronous code in C++ world. I have discussed this with @andylokandy previously and decide to just let it block, so user can easily finish their POC.

In future(as soon as rust-client release), we can provide some asynchronous API (e.g. completion queue) to solve the performance issue.

2 Likes
#6

How about considering a specific runtime? For example, libunifex. https://github.com/facebookexperimental/libunifex

#7

With an offline discussion with @SchrodingerZhu I try to workaround JNI limitation by using global reference. Here is an implementation.

However, due to the number limit of global reference (<65536) which bounds the throughput of native calls if we generate one global reference per native call, it is not a good solution.

We agree with that a thin mailbox alike middleware is the final answer. It will be like the PyCoutine solution in client-py.

@iosmanthus I take a look at the PyCoroutine implementation and here is a major question:

In order to get the result of awaitable, it is the coroutine to notify the caller, or the caller round-robin the coroutine? It seems like the latter which is not quite elegant also.

1 Like
#8

The latest version of the repo above shows a resolution with a simple bookkeeping implementation.

If boilerplate code clean I think it is a sample implementation for real asynchronous JNI, with little overhead though.

#9

client-py uses the latter. This is exactly how async in python is supposed to work.

#10

Sounds reasonable, in other words, java client should implement its own wake-up facility.

#11

@leiysky Well, I agree with you. But std::future is not a wrapper of std::thread. (maybe std::async current is).
std::future is just promise that: “I’ll provide the value in the future, and maybe I will throw some exceptions”.
In my company, C++ libraries use folly::Future<T> as the client result ( and it will throw some exceptions)

#12

Actually I’m curious how Rust asynchronization is implemented. I notice that Future has a method named poll but round-robin and poll sounds like a CPU-busy way.

#13

@tison
https://rust-lang.github.io/async-book/

Here’s the async book which introduces design of async in Rust.

poll is just an action to drive the continuition, invocation of which depends on implementation of executor(generally not round-robin).

#14

It depend on runtime.

#15

@tison Hi, I think a better documentation on the implementation of async is at https://os.phil-opp.com/async-await/