Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add uv publish: Basic upload with username/password or keyring #7475

Draft
wants to merge 5 commits into
base: main
Choose a base branch
from
Draft

Add uv publish: Basic upload with username/password or keyring #7475

wants to merge 5 commits into from
+1,813 −55

Conversation

konstin
Copy link
Member

@konstin konstin commented Sep 17, 2024 •
edited
Loading

The uv publish command allows uploading wheels and source distributions to PyPI or another registry. We support username/password auth, token auth and (follow-up) trusted publishing from GitHub Actions. Be default we upload files from dist/*, the output directory from uv build.

This is intended to support three different workflow, ordered by relevance:

  1. Publishing a pure Python package from CI
  • Call uv build
  • Clear the venv, install the wheel, run a smoke test
  • Clear the venv, install the source distribution, run a smoke test
  • Call uv publish
  1. Publishing a native module package from CI
  • In one job, call uv build, clear the venv, install the source distribution, run a smoke test, upload the source distribution as artifact
  • For each target, run a job, call uv build --wheel, clear the venv, install the wheel run a smoke test, upload the wheel as artifact
  • In a final job, download all artifacts to dist/* and run uv publish
  1. Publishing from a developer machine
  • Call uv build
  • Do a manual test routine
  • Call uv publish

Since we intend for smoke tests between build and publish, there is no combined build-and-publish command.

What works:

  • Basic upload to pypi with token or username/password (username must be __token__, but it uses the same HTTP headers)
  • Keyring integration. If you use scoped tokens, you need use url-string hacks such as Not obvious how to use multiple project API tokens with keyring pypa/twine#565 (comment) to differentiate the tokens, a problem we share with twine.
  • Error messages
  • Testing on CI. Every time something in the upload crate or its test script changes, we upload a new version of dummy packages to testpypi

Next PRs:

  • Trusted publishing from GitHub actions with OIDC, and testing for it.

What's missing:

  • Other registries?
  • Progress bars
  • A demo repo with pure Python package that demonstrates scenario 1 in a copy&paste-able GitHub actions configuration. We can only demo this live once the feature shipped.

Best reviewed commit-by-commit

@konstin konstin added the enhancement New feature or request label Sep 17, 2024
konstin
konstin commented Sep 17, 2024
View reviewed changes
Cargo.lock Outdated Show resolved Hide resolved
@zanieb
Copy link
Member

zanieb commented Sep 17, 2024

I have a test script for test PyPI for both username/password and keyring (next PR), but we need to run it in CI in a secure way.

Does it need to be secure? Or can we just have a test user that only uploads dummy packages that we can expose the password to?

@konstin konstin force-pushed the konsti/publish2 branch 3 times, most recently from ea8fc61 to 3775e2c Compare September 18, 2024 21:26
konstin added a commit that referenced this pull request Sep 19, 2024
@konstin
Add only_authenticated option to the client
49aa8f6 
When sending an upload request, we use HTTP formdata requests, which can't be cloned (seanmonstar/reqwest#2416, plus a limitation that formdata bodies are always internally streaming), but also know that we need to always have credentials.

The authentication middleware by default tries to clone the request and send an authenticated request first. By introducing an `only_authenticated` setting, we can skip this behaviour for publishing.

Split out from #7475
konstin added a commit that referenced this pull request Sep 19, 2024
@konstin
Add only_authenticated option to the client
e9571d5 
When sending an upload request, we use HTTP formdata requests, which can't be cloned (seanmonstar/reqwest#2416, plus a limitation that formdata bodies are always internally streaming), but also know that we need to always have credentials.

The authentication middleware by default tries to clone the request and send an authenticated request first. By introducing an `only_authenticated` setting, we can skip this behaviour for publishing.

Split out from #7475
konstin added a commit that referenced this pull request Sep 19, 2024
@konstin
Add only_authenticated option to the client
8a54b24 
When sending an upload request, we use HTTP formdata requests, which can't be cloned (seanmonstar/reqwest#2416, plus a limitation that formdata bodies are always internally streaming), but also know that we need to always have credentials.

The authentication middleware by default tries to clone the request and send an authenticated request first. By introducing an `only_authenticated` setting, we can skip this behaviour for publishing.

Split out from #7475
konstin added a commit that referenced this pull request Sep 19, 2024
@konstin
Add only_authenticated option to the client
0debcb3 
When sending an upload request, we use HTTP formdata requests, which can't be cloned (seanmonstar/reqwest#2416, plus a limitation that formdata bodies are always internally streaming), but also know that we need to always have credentials.

The authentication middleware by default tries to clone the request and send an authenticated request first. By introducing an `only_authenticated` setting, we can skip this behaviour for publishing.

Split out from #7475
@konstin konstin mentioned this pull request Sep 19, 2024
Open
konstin added a commit that referenced this pull request Sep 19, 2024
@konstin
Add only_authenticated option to the client
fbc0482 
When sending an upload request, we use HTTP formdata requests, which can't be cloned (seanmonstar/reqwest#2416, plus a limitation that formdata bodies are always internally streaming), but also know that we need to always have credentials.

The authentication middleware by default tries to clone the request and send an authenticated request first. By introducing an `only_authenticated` setting, we can skip this behaviour for publishing.

Split out from #7475
@konstin konstin force-pushed the konsti/publish2 branch 3 times, most recently from 7502383 to 6f83d3f Compare September 19, 2024 12:45
@konstin
Add only_authenticated option to the client
593fca0 
When sending an upload request, we use HTTP formdata requests, which can't be cloned (seanmonstar/reqwest#2416, plus a limitation that formdata bodies are always internally streaming), but also know that we need to always have credentials.

The authentication middleware by default tries to clone the request and send an authenticated request first. By introducing an `only_authenticated` setting, we can skip this behaviour for publishing.

Split out from #7475
@konstin
Add uv publish cli and configuration
26cc6f5
@konstin
Implement uv publish
441f58c
@konstin
CI integration testing for uv publish
f72da4d
@konstin konstin deployed to uv-test-publish September 19, 2024 14:29 — with GitHub Actions Active
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@konstin @zanieb