Linux, Open Source & Unrelated Topics - pythonhttps://pbarker.dev/2023-05-24T00:00:00+00:00Thoughts on PyPI, PGP and Sigstore2023-05-24T00:00:00+00:002023-05-24T00:00:00+00:00Paul Barkertag:pbarker.dev,2023-05-24:/posts/2023-05-24/thoughts-on-pypi-pgp-and-sigstore/<p class="first last">A recent grumble about PGP signatures on PyPI has quickly led to PyPI
dropping support for PGP. While I agree that there are major issues with
PGP, I don't agree that its use in PyPI is "worse than useless" and I'm
disappointed to see support dropped before a replacement has been deployed.
Sigstore seems to be a promising replacement, but I think further work is
needed before this can become a key pillar for securing the open source
ecosystem.</p>
<!-- Copyright (c) 2023 Paul Barker <paul@pbarker.dev>
SPDX-License-Identifier: CC-BY-NC-4.0 -->
<div class="section" id="pgp-grumbles-footguns">
<h2>PGP: Grumbles & Footguns</h2>
<p>A recent <a class="reference external" href="https://blog.yossarian.net/2023/05/21/PGP-signatures-on-PyPI-worse-than-useless">grumble about PGP signatures on PyPI</a>
has quickly led to <a class="reference external" href="https://blog.pypi.org/posts/2023-05-23-removing-pgp/">PyPI dropping support for PGP</a>.
I'm a little torn on what to make of this.</p>
<p>The PGP ecosystem is difficult to make the best use of and suffers from a
conceptual design which is stuck in the 90's. This isn't helped by the most
commonly used tool, gnupg, being rather obtuse. Improvements have definitely
been made in recent years though - for example key discoverability and questions
of how much trust to give to unknown keys have been made easier by the
introduction of <a class="reference external" href="https://keys.openpgp.org/about">keys.openpgp.org</a> which
verifies ownership of email addresses before indexing PGP keys. And the
<a class="reference external" href="https://sequoia-pgp.org/">Sequoia-PGP</a> library and command line tools bring
memory safety (Rust FTW), a cleaner API and a simpler command line interface to
users. But are these infrastructure and tooling improvements enough?</p>
<p>It's just fundamentally non-trivial to make use of PGP to securely sign software
releases, backup archive and other such artifacts
(we'll be ignoring the use of PGP to sign & encrypt email here as this is
another can of worms which could take up a full blog post).
For the Linux kernel, there is an extensive <a class="reference external" href="https://docs.kernel.org/process/maintainer-pgp-guide.html">Kernel Maintainer PGP guide</a>
based on the Linux Foundation's <a class="reference external" href="https://github.com/lfit/itpol/blob/master/protecting-code-integrity.md">Protecting code integrity with PGP</a>
IT policy document. There's a lot to take in here, and footguns abound.</p>
</div>
<div class="section" id="minisign">
<h2>Minisign</h2>
<p>I've seen many people point at <a class="reference external" href="https://jedisct1.github.io/minisign/">Minisign</a>
as the solution. It's certainly easier to use! And somewhat harder to misuse
since it only supports a single known-good signing algorithm instead of trying
to support everything under the sun. But it lacks key features which I think are
critical to the intended use-case:</p>
<ul class="simple">
<li>key expiry & revocation features are needed to limit the damage which may
occur if key material is leaked to unauthorised users.</li>
<li>support for crypto smart cards & Hardware Security Modules (HSMs) is needed to
reduce the likelihood of leaking key material in the first place.</li>
<li>integration with git is needed to be able to sign commits during development
and to sign tags at release time.</li>
</ul>
</div>
<div class="section" id="sigstore">
<h2>Sigstore</h2>
<p>A better solution would be <a class="reference external" href="https://www.sigstore.dev/">sigstore</a>, which
simplifies the process of both signing and verifying packages without
compromising on the security of key material. It does this by using a
centralized certificate authority which issues ephemeral signing keys each time
you want to make a signature. The signer's problem then becomes one of
authentication instead of one of key management, and this problem is delegated
to OpenID Connect (OIDC) identity providers. Assuming you already have an
account with an OIDC provider supported by sigstore (Google, GitHub, etc), you
simply authenticate with your chosen ID provider to allow you to create a
sigstore signature tied to your identity. And sigstore also provides <a class="reference external" href="https://docs.sigstore.dev/gitsign/overview">support
for signing git commits</a>, which
Minisign lacks.</p>
<p>The key assumption here is that you're willing to delegate trust to both the
sigstore root-of-trust and the limited number of existing OIDC providers. The
first of those two seems somewhat reasonable on first look, and it helps that
their <a class="reference external" href="https://github.com/sigstore/root-signing">root signing tools</a> are
open. The second is a larger assumption in my view - do we want to further
centralise the security of the open source ecosystem <a class="footnote-reference" href="#footnote-1" id="footnote-reference-1">[1]</a> around the platform
oligopoly of Microsoft, Google and co? I don't think we do. Instead, if OIDC is
going to be used in this way, we need to see a variety of other OIDC hosts who
can act as privacy-preserving, open and non-commercial identity providers for
the community. We will also need to see well-supported options for both
individuals and projects to self-host an OIDC identity provider.</p>
<p>It's also worth reviewing <a class="reference external" href="https://docs.sigstore.dev/security/#what-sigstore-doesnt-guarantee">What Sigstore Doesn't Guarantee</a>. The
main issue here to me is that there is no way to mitigate compromise of an OIDC
identity or provider.</p>
<p>Sigstore also integrates with <a class="reference external" href="https://github.com/sigstore/rekor">Rekor</a> to
provide "an immutable tamper resistant ledger" (quoting from the readme) of
signatures. This is an excellent feature, but the benefit isn't exclusive to
sigstore as other signature types (such as Minisign signatures) can be uploaded
to the Rekor transparency log.</p>
<p>On balance, I'm feeling positive about sigstore. More work is definitely needed,
both in integration with hosts like PyPI & GitHub and with support for a more
decentralised identity model. But the current state is a very good start. I'm
going to try it out for my next release of <a class="reference external" href="https://pypi.org/project/mirrorshades/">mirrorshades</a>.</p>
</div>
<div class="section" id="coming-back-to-python">
<h2>Coming back to Python</h2>
<p>To loop back round to PyPI, my complaint is that the (somewhat poor and
atrophied) support for PGP is being dropped before a replacement has been
integrated. If sigstore does prove to be the way forward then that's great, but
I would have preferred PyPI to keep the existing PGP support as-is until
sigstore integration can be deployed. I don't agree that the status quo is
"worse than useless", though I do agree that it has major issues.</p>
<p>While we're here, we should also talk briefly about the other recent improvement
to the PyPI trust model: "Trusted Publishing", as discussed in the <a class="reference external" href="https://blog.pypi.org/posts/2023-04-20-introducing-trusted-publishers/">PyPI Blog</a> and
the <a class="reference external" href="https://blog.trailofbits.com/2023/05/23/trusted-publishing-a-new-benchmark-for-packaging-security/">Trail of Bits Blog</a>.
Trusted Publishing allows package uploaders to authenticate with an OIDC
identity instead of a long-lived API key. My thoughts here are similar to those
above for sigstore - removing the need to manage a long-lived secret is very
welcome, but we need to have a wide array of non-commercial OIDC providers if
OIDC is going to become a foundational piece of infrastructure for open source
development.</p>
<p class="rubric">Footnotes</p>
<table class="docutils footnote" frame="void" id="footnote-1" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-1">[1]</a></td><td>I've deliberately avoided using the phrase "supply chain" in relation to
open source software here.</td></tr>
</tbody>
</table>
</div>
New release: mirrorshades v0.2.02022-02-15T00:00:00+00:002022-02-15T00:00:00+00:00Paul Barkertag:pbarker.dev,2022-02-15:/posts/2022-02-15/new-release-mirrorshades-v020/<p class="first last">I'm happy to announce that version 0.2.0 of
<a class="reference external" href="https://pypi.org/project/mirrorshades/">mirrorshades</a>,
a tool for mirroring data from remote sources, has been released.</p>
<!-- Copyright (c) 2022 Paul Barker <paul@pbarker.dev>
SPDX-License-Identifier: CC-BY-NC-4.0 -->
<p>I'm happy to announce that version 0.2.0 of <a class="reference external" href="https://pypi.org/project/mirrorshades/">mirrorshades</a>, a tool for mirroring data from
remote sources, has been released.</p>
<p>This release can be downloaded from
<a class="reference external" href="https://pypi.org/project/mirrorshades/0.2.0/">PyPI</a> or
<a class="reference external" href="https://github.com/unnecessary-abstraction/mirrorshades/releases/tag/v0.2.0">GitHub</a>.</p>
<p>The following changes have been made since the previous release (v0.1.3):</p>
<ul class="simple">
<li>Moved the project to GitHub, the new project URL is
<a class="reference external" href="https://github.com/unnecessary-abstraction/mirrorshades">https://github.com/unnecessary-abstraction/mirrorshades</a>.</li>
<li>Added GitHub mirroring support.</li>
<li>Pruned deleted branches when updating a mirrored git repository.</li>
<li>Added support for multiple attempts when running custom commands.</li>
<li>Added config validation using <a class="reference external" href="https://pypi.org/project/desert/">desert</a> and
<a class="reference external" href="https://pypi.org/project/marshmallow/">marshmallow</a>.</li>
<li>Updated and expanded README file.</li>
<li>Improved developer & contributor workflows with the addition of automatic
linting & formatting. These checks are ran in GitHub Actions when contributing
to the project via a pull request. They can also be ran locally using
<a class="reference external" href="https://pre-commit.com/">pre-commit</a>.</li>
<li>Satisfied the <a class="reference external" href="https://reuse.software/spec/">REUSE Specification</a> to ensure
licensing is clear.</li>
</ul>
<p>See the <a class="reference external" href="https://github.com/unnecessary-abstraction/mirrorshades/compare/v0.1.3...v0.2.0">full comparison between v0.1.3 and v0.2.0</a>
for more details.</p>