The whole point of FFI is that it's a lowest common denominator - it's a foreign function interface, meant for interop between two potentially wildly different runtimes, up to and including assembly language. That means it can't rely on quirks like language-specific type information or bounds checking. The only thing that all runtimes on a machine are guaranteed to have in common is that they use the same set of memory and CPU registers, so that is all that FFI can use to define subroutine interfaces. Hence terms like 'ABI', 'calling convention' and 'memory layout'.
The good news is that whatever issue you have can probably be solved without resorting to FFI. There are plenty of other interop options - for example, if you control the library, you can use your operating system's IPC mechanisms with an agreed-upon serialization format. Or, if your library and application are within the same ecosystem, you can use language-specific library management features such as Python's import or Rust's crates. FFI is a bit like C itself - if you're reaching for it as a solution to your problem, then your finger is already on the trigger of the footgun.
I think there's a conflict between FFIs in practice and theoretical FFIs. The few papers I've read for sound FFIs between languages A and B rely on the fact that the common interop target is a high-level language whose features subsumes the features of A and B. ABI, calling convention, etc. are merely implementation details.
So from this perspective, the interop target should actually not be the lowest common denominator, but the opposite: a target that can describe all possible user languages in a common format.
In fact, I would actually argue that the implementation details are NOT that interesting and only a distraction: that we need an abstract way of declaring compatibility. For instance, one can imagine a high-level FFI target where function types are parameterized by their calling convention i.e.:
`foo: (VOID<C>->VOID<C>)<C>`
`bar: (UNIT<Rust> -> UNIT<RUST>)<RUST>`
and attempting to call `bar` from `foo` involves a compiler intrinsic like `RUST_CALL(bar)()` to convert the type (and the calling convention) appropriately.
IPC is, I think, the concept that has been neglected and led to the desire for some kind of really intelligent FFI.
If you have code in Ruby and in Julia that you want to mix together it's probably because both pieces of code leverage the distinct capabilities of those two languages, which have nice capabilities because they accepted very tradeoffs.
Would you really want Julia with Ruby's Number class? It would entirely kill performance. You want it to be an integer maybe, or a double, or sometimes something else. It's going to be application specific is the point, because another user want some different subset of Number's behavior. I don't think there's a shortcut here besides making a very limited or opinionated FFI system.
Not necessarily. Sometimes I want to use FFI because the "hard work" (e.g. implementing a complicated protocol) was done in language A, and I want to use language B in my project, but I don't want to redo all that hard work.
And most people used to believe that ergonomic, zero cost memory safety was impossible too. And yet it moves. There is no reason why we have to settle for a poor FFI solution just because it was the choice of our predecessors.
Memory safety and FFI operate on two different layers of abstraction entirely. What does it mean to enforce 'memory safety' across the boundaries of two different applications/libraries that may have entirely different ways of keeping track of memory allocations and deallocations? Again, FFI has to be agnostic to such implementation details in order to work, and it does so by forcing you to define interfaces in terms of the lowest possible level of abstraction common to all applications. That means explicitly specifying expected memory layout and calling convention on both the application and library sides, i.e. an ABI.
The good news is that whatever issue you have can probably be solved without resorting to FFI. There are plenty of other interop options - for example, if you control the library, you can use your operating system's IPC mechanisms with an agreed-upon serialization format. Or, if your library and application are within the same ecosystem, you can use language-specific library management features such as Python's import or Rust's crates. FFI is a bit like C itself - if you're reaching for it as a solution to your problem, then your finger is already on the trigger of the footgun.