Skip to content

Add librt functionality for lazy deserialization#21158

Open
ilevkivskyi wants to merge 3 commits intopython:masterfrom
ilevkivskyi:lazy-1
Open

Add librt functionality for lazy deserialization#21158
ilevkivskyi wants to merge 3 commits intopython:masterfrom
ilevkivskyi:lazy-1

Conversation

@ilevkivskyi
Copy link
Copy Markdown
Member

I am going to add actual lazy deserialization in a separate PR, probably next week. It will require one more preparatory PR setting up global state for lookup_fully_qualified() that I will probably add on the weekend.

cc @JukkaL

@ilevkivskyi
Copy link
Copy Markdown
Member Author

Hm, IIUC I should bump LIBRT_INTERNAL_API_VERSION but then this creates a chicken and egg problem in tests. So that I can't really merge this until I publish new librt version on PyPI.

I guess we should use the local version of librt during self-compilation in tests.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 3, 2026

According to mypy_primer, this change doesn't affect type check results on a corpus of open source code. ✅

@ilevkivskyi
Copy link
Copy Markdown
Member Author

This also makes me think that if anyone is using compiled mypy form master, it will break as soon as I merge this. So maybe a better approach is to actually release this in librt first.

@JukkaL
Copy link
Copy Markdown
Collaborator

JukkaL commented Apr 9, 2026

So maybe a better approach is to actually release this in librt first.

Makes sense.

return _skip_int(data);
if (tag == INSTANCE)
return _skip_instance(data);
if (tag > MYPY_FILE && tag < RESERVED)
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be >=?

Copy link
Copy Markdown
Member Author

@ilevkivskyi ilevkivskyi Apr 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is actually intentional, a little sanity check. We know that we should only read symbol nodes using this function, and MypyFile should be always handled using cross_ref in cache, and never appear explicitly in symbol tables.

Py_ssize_t size = _read_size(data);
if (size < 0)
return CPY_NONE_ERROR;
int i;
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this have type Py_ssize_t? (Here and other similar functions.)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I guess I just copied this from somewhere and all test passed. I will check what is the best cross-platform type here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants