[mypyc] Move API table definitions to .c files

[p-sawicki]

lib-rt API tables are defined as static variables in lib-rt headers. This means that each translation unit gets its own independent instance of these variables.

That becomes a problem in multi-file compilation mode when using BytesWriter.write, as this function is translated to CPyBytesWriter_Write by mypyc which is defined in bytes_writer_extra_ops.c. With multi-file compilation, bytes_writer_extra_ops.c is compiled as its own translation unit and gets linked with the C extension compiled from python files.

The C extension TU copies the librt.strings capsule contents into the global table but because it's static this is not visible in the table in the bytes_writer_extra_ops.c TU, which stays zero-initialized. This results in a seg fault with the following call chain:

CPyBytesWriter_Write -> CPyBytesWriter_EnsureSize -> LibRTStrings_ByteWriter_grow_buffer_internal -> LibRTStrings_API[5]-> oops all zeros

To fix this, declare the tables as extern and define them in .c files so there's only one global version of each variable. There's one new .c file for each lib-rt module that is compiled or included when mypyc detects a dependency on that module.

[JukkaL]

Now all the API tables will always be included in each build, which increases the memory usage of each module a little. It's not a big deal right now, but this is not scalable if we add many librt submodules in the future. Could we instead have librt_vecs_static.c etc. and add them as dependencies?

[p-sawicki]

yeah that's a good point. i've moved the definitions to separate files and extended the dependency handling to compile/include them when needed.