Add msgpack-white package #386
Conversation
This package contains a C extension, so micropip can't install it directly from PyPI, hence packaging it here for use in Pyodide. A short test of basic functionality is included.
Package Build ResultsTotal packages built: 28 Package Build Times (click to expand)
Longest build: libopenssl (2m 37s) |
There was a problem hiding this comment.
Thanks, @sometimes-i-send-pull-requests! I have attempted to perform some sanity checks about the package, as it is not one that I have heard of prior to your PR. I would like to seek answers concerning the following points, before we end up including this package in the Pyodide distribution:
- Currently, there is no user base for
msgpack-white, given that the number of downloads is quite minimal to suggest that there is one: https://pypistats.org/packages/msgpack-white. While we don't require this per se for every package out there,msgpack-whitehas only one release on PyPI that was uploaded on October 10th, 2025, which is pretty recent. - I am unable to locate
msgpack-white's source code in a GitHub repository or any other code hosting platform, and there are only three public search results for "msgpack-white" on two search engines I tried (Kagi and Google). There is no documentation, either, besides the PyPI README. - I was able to see that Safety CLI Cybersecurity, Inc., a software supply chain security company, reports this package's only version, v1.0.0, with a known vulnerability: https://data.safetycli.com/packages/pypi/msgpack-white/versions. While I am not able to find out what vulnerability this is because it's not stated explicitly (the website simultaneously states both: that it has a vulnerability in v1.0.0, and that it doesn't), it is not a good sign for a package of its intended purpose. Popular packages, for example, NumPy, have proper information about CVEs and affected versions: https://data.safetycli.com/packages/pypi/numpy/versions
- I tried to check if this was published using a CI job and has attestations I could use to prove its provenance, and it turns out it wasn't, as PyPI reports that trusted publishing was not used. See https://pypi.org/project/msgpack-white/#msgpack_white-1.0.0-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.whl for example. The package was uploaded manually from a local computer using Twine.
- I extracted the package files, and they only contain a shared object file with all compiled code exposed as an extension module with no other Python code besides imports. Coupled with the lack of a public repository to view the code, this brings forth no visibility for me to examine the package's structure and contents or validate the sources used to compile the extension module.
- Lastly, with no other resort available, I tried to perform a background check on you, the author of this PR. It is unclear if you are the author of the package on PyPI as well. I found https://kirchhoff.digital/ from the PyPI publishing email used, which led me to a placeholder website for a person named "Alexander Kirchhoff," with no other links or additional details available about their personal and professional history, including but not limited to previous and current employment data, experiences, or educational information.
With all these reasons combined, I am unable to accept this PR in its current state. I have reason to believe that it poses a risk to our users' security posture, and that is not a useful addition to the Pyodide distribution. Please let me know if you have any comments, questions, or can alleviate my concerns on this, and I would be happy to discuss the inclusion of this package with you further.
|
Thank you @agriyakhetarpal for the thorough review.
That's correct. I am the author of this package, and I just published it recently. There are no uses in the wild at this time; every download would have been from me testing it or some crawler downloading it. As context, the main reason for this package's existence is the lack of support for streaming decode in
I finally got around to polishing it up into a package, and msgpack-white is the result. I don't have any pressing need for using it in Pyodide, but I am a big fan of Pyodide, and I wanted to make sure that my package would work on as many platforms as possible. That's part of why I built so many binary wheels, even for less common architectures like s390x: I want everyone to be able to use my package, without building from source causing it to be a pain in the butt. To me, Pyodide is another place where someone might want to use my package (especially since msgspec, which I built this for, is also available in Pyodide), and I wanted it to work out of the box. Unfortunately I ran straight into pypi/warehouse#10416 when trying to upload my Pyodide wheel to TestPyPI, and learned pyodide-recipes is the only way to get a non-pure-Python package to work out of the box with Pyodide.
I have not currently published the repository. I tend not to use public code hosting sites for personal reasons unless required for work. I considered putting a copy of the repository and a cgit instance for browsing it on my web site, but I haven’t gotten around to it yet. The sdist contains all the code in any case.
Yes, the only documentation is the README. The package has incredibly small surface area: It has one module containing a single class containing a single method. It is very single purpose, but it tries to do this one job very well. I may add more documentation later, but given its narrow scope, I believe this should be sufficient for now.
I believe you are misreading the web site. The "🟥 has known vulnerabilities" text you see is a legend saying versions with a red background have a known vulnerability. Versions with a white background are not known to have a vulnerability. msgpack-white version 1.0.0 has a white background, and hence does not contain a known vulnerability. You can confirm this by looking at the results for NumPy you also linked: There are a number of versions with a white background indicating no known vulnerability for that version, along with other versions with a red background indicating a vulnerability in that version. The legend is a little confusing when there's nothing for it to apply to. 😅 SafetyCLI's site aside, I care deeply about security. I fuzzed the heck out of my package with libFuzzer before publishing it, to be as sure as I could there were no vulnerabilities in it. I even started trying to use some formal verification tools to make sure nothing could have slipped by the fuzzer, either (though I haven't been able to fully formally verify it yet). If there's a vulnerability, I would very much like to know about it, both to fix it and to figure out where my processes have failed me. But, to the best of my knowledge, there is no vulnerability in it.
That's right. I built all the wheels myself. This is related to my personal reasons for not using public code hosting sites. I understand this leaves something to be desired for auditability. I would like to investigate ways to do deterministic builds to increase auditability in the future, which I trust more than attestation (though both together is better still), but I have not done so currently.
In the absence of a browsable code repository (which I may look into in the future, but is not a high priority for me at this time), I would recommend inspecting the sdist. The sdist is authoritative in any case, and is what's linked in this PR so is what the Pyodide build would use. The package is very small, consisting of two C files totaling 331 lines, so it should be easy to review.
I keep a low profile online for personal reasons. I am happy to provide this information for you in private; please e-mail me at the maintainer address listed on PyPI and I would be happy to provide information about my background.
I hope these responses are satisfactory. I understand this is a niche package, so even if there were no other concerns, I understand if it were to be rejected solely on the lack of usage. If this package is deemed inappropriate for inclusion into the Pyodide distribution, I would be curious if you would have any insight into what I could do to best help resolve pypi/warehouse#10416, as if that were resolved, I would be able to support Pyodide without needing explicit inclusion into the Pyodide distribution. While I am not committing to doing so, I could for example take a crack at writing up the PEP for Emscripten/Pyodide wheels (the lack of which sounded like the blocker preventing Warehouse from allowing uploads of such wheels), but I don't want to step on the Pyodide developers' toes, as I acknowledge I am a newcomer here. |
2 participants
This package contains a C extension, so micropip can't install it directly from PyPI, hence packaging it here for use in Pyodide. A short test of basic functionality is included.