"Blocking" FFI actors run as separate scheduler threads

[mneumann]

• The Pony runtime uses cooperative scheduling
• Sometimes, FFI needs to block. Examples are: Many file system related operation like mkdir, readdir etc.
• In order to not block other actors scheduled on the same scheduler thread, one would have to spawn a separate thread in C-land and communicate via a pipe. That involves quite a bit of serialization in both Savi and C.
• What if ponyruntime would support starting FFI actors within their own scheduler thread.
• The code running within such a FFI actor could simply use blocking syscalls. No need to create a special pipe, no need for serialization and deserialization (except FFI requires a convertion). It could use normal Pony message passing.
• IMHO, this would make some FFI code a lot simpler.

[mneumann]

Dunno if Pony uses some optimizations related to the number of scheduler threads, which is fixed to the number of CPUs. Allowing to dynamically create new scheduler threads might be difficult in Pony...

One could go with a fixed number of FFI schedulers in the runtime. But this clearly is not ideal.