Page MenuHomeFreeBSD

Update MASTER_SITE_CHEESESHOP
ClosedPublic

Authored by dbaio on Jun 30 2017, 12:07 AM.
Tags
None
Referenced Files
Unknown Object (File)
Sat, Apr 27, 3:58 AM
Unknown Object (File)
Feb 2 2024, 6:36 AM
Unknown Object (File)
Jan 22 2024, 10:14 AM
Unknown Object (File)
Dec 20 2023, 6:52 AM
Unknown Object (File)
Nov 20 2023, 7:33 PM
Unknown Object (File)
Nov 2 2023, 12:05 AM
Unknown Object (File)
Oct 31 2023, 12:08 AM
Unknown Object (File)
Oct 27 2023, 5:36 AM
Subscribers

Details

Summary
Update MASTER_SITE_CHEESESHOP

Move `files.pythonhosted.org` mirror to the top

The mirror `pypi.python.org` soon will be replaced with the
new Warehouse [1][2], and now it's only serving the old files
and its returning `404 - Not Found` to the new files hosted

[1] https://pypi.org
[2] https://github.com/pypa/warehouse

Approved by:	garga (mentor), ???
Differential Revision:	https://reviews.freebsd.org/D11420
Test Plan

I've been noticing in updates of ports that uses CHEESESHOP as MASTER_SITES,
that the first request through pypi.python.org returns 404 Not Found.

With a make fetch test, of 1728 ports, 665 are receiving 404 Not Found (38%)
trough pypi.python.org. Seems that the new packages are being hosted in the
new repo/system called Warehouse.

Warehouse is a next generation Python Package Repository designed to replace the
legacy code base that currently powers PyPI (pypi.python.org).

The Warehouse is in pre-production, because of this I would suggest to just let
files.pythonhosted.org be the first mirror for now.

More info here:
https://github.com/pypa/warehouse
https://bitbucket.org/pypa/pypi/issues/438#comment-27606213

My test logs are here

Other informations:

# New file
https://pypi.python.org/packages/source/u/urllib3/urllib3-1.21.1.tar.gz
HTTP/1.1 403 Forbidden |

https://files.pythonhosted.org/packages/source/u/urllib3/urllib3-1.21.1.tar.gz
HTTP/1.1 302 Found -->
Location: https://pypi.python.org/packages/96/d9/40e4e515d3e17ed0adbbde1078e8518f8c4e3628496b56eb8f026a02b9e4/urllib3-1.21.1.tar.gz
HTTP/1.1 200 OK |

https://pypi.org/packages/source/u/urllib3/urllib3-1.21.1.tar.gz
HTTP/1.1 301 Moved Permanently -->
https://files.pythonhosted.org/packages/source/u/urllib3/urllib3-1.21.1.tar.gz
HTTP/1.1 302 Found -->
Location: https://pypi.python.org/packages/96/d9/40e4e515d3e17ed0adbbde1078e8518f8c4e3628496b56eb8f026a02b9e4/urllib3-1.21.1.tar.gz
HTTP/1.1 200 OK |

files.pythonhosted.org is converting all requests:

So, previously PyPI used URLs like : /packages/{python version}/{name[0]}/{name}/{filename}
Now it uses: /packages/{hash[:2]}/{hash[2:4]}/{hash[4:]}/{filename} Where hash is blake2b(file_content, digest_size=32).hexdigest().lower()

https://bitbucket.org/pypa/pypi/issues/438#comment-27230605
https://github.com/pypa/conveyor

Diff Detail

Repository
rP FreeBSD ports repository
Lint
Lint Not Applicable
Unit
Tests Not Applicable