The file Mk/bsd.sites.mk contains a list of mirror sites for software repos. Many of the sites supports HTTPS, but we are still using HTTP.
Details
Details
- Reviewers
lme bapt - Group Reviewers
portmgr O5: Ports Framework - Commits
- rP461262: use HTTPS whenever possible
Diff Detail
Diff Detail
- Repository
- rP FreeBSD ports repository
- Lint
Lint Not Applicable - Unit
Tests Not Applicable
Event Timeline
Comment Actions
Just a note, that does not improve anything security wise, given all the distfiles avec validated against checksums before getting used. that said, I'm not against that change :)
Comment Actions
The issue is about privacy, not security. To quote wikipedia: "The domain of privacy partially overlaps security (confidentiality), which can include the concepts of appropriate use, as well as protection of information."
Comment Actions
While I agree with bapt I think we should offer HTTPS where possible as this seems to be a concern for some people and many sites will redirect to HTTPS anyway.
That said, it wouldn't hurt to give Mk/bsd.sites.mk an overhaul in general.
I've done similar work for other open source distros and can give it a go using the following guidelines if there's interest.
I'd like to do it per master site and not do a massive diff.
- 8-10 (mirror)-sites in total, fall back to main site as last resort
- HTTPS (where possible) --> HTTP --> FTP (in that order)
- Prioritize fast mirrors in general with good connectivity (peering/backbone) for NA/EU users however provide at least 2 mirrors in Asia for connectivity and availability if possible
Comment Actions
I am thrilled with points 2. Point 2 would be HUGE for me.
I'm not sure that there's much benefit to limiting the number of mirrors. There's no reason to remove mirrors unless they don't work, or they're prohibitively slow. I'm fine with prioritizing NA and EU sites, but there's no reason to remove Asian mirrors.
Comment Actions
There's no point in having like 15+ mirrors, in general I'd say if 4+ fails there's another general issue than the mirrors themselves and it'll just take ages before you hit FreeBSD's own distfiles servers.
What I meant is that Asian mirrors shouldn't be the first choice as they're going to be slow for NA/EU users in general which is the majority of users (?) but still useful when it comes to connectivity and available hence why they should be included.
Given what I've mentioned earlier there's no need for a boatload of listed ones, it'll just be a hassle in the end.
I have no intention of down prioritize Asian users but unfortunately there's still quite a bit of latency for most users accessing sites which isn't ideal.
Comment Actions
Not all mirrors update at the same frequency. It's not uncommon (when following head) to go 10 mirrors down before one fetches.
I would rather have 4 of 20 mirrors fetch than 0 of 5.
I think a good place to start is https > http > ftp. I'd prefer to see https added at higher priority in each master site, than to see http replaced altogether (as this review does). I'm fine with reducing the number of http in each mirror (replacing them with https), but removing all http mirrors from a group is a disservice to a segment of our user base.
Comment Actions
One problem with a change like this is that it prevents most proxies from being able to cache the files for 'privacy'. It's still obvious a connection is made to these mirrors, just not which file is being fetched. Does that really matter?
Comment Actions
While it might be an potential issue I'm not sure it is in the end as most of these sites hosts files several megabytes in size which most proxies doesn't cache anyway?
Comment Actions
I think that caching becomes less important now. Network traffic is damn cheap (<1 Euro per TB). If you need to reduce the network traffic or latency for your FreeBSD machines, you may setup your own distcache server.
Comment Actions
Yes there is. To allow our users to choose which they want first, using the MASTER_SORT feature.
Comment Actions
I didn't know that existed at all so thanks for bringing that up however in all fairness many sites these days aren't easily geographically identifiable by domain and/or sub domains so it may only be useful to some extent.
Worth nothing is that back in August bapt did a remove a lot of dead hosts which leads me to believe that having lot of hosts that rarely or never gets used only adds to more potential dead hosts that "never" gets noticed. I highly doubt that all were closed down closely in time to each other in the first place. I do think however that mirrors should be unique which would being more diversity comped to just switching protocol for the same host and should bring enough of choice for most users?
Comment Actions
About the proxy caching, yes it is true that it might break but as it has been stated those files are anyway usually to big to be actually cached.
As for the privacy brought by https, it is quite easy to guess which distfiles are being downloaded base on the size.
And about fast network access, there are still plenty of countries/places without fast and/or cheap internet access.
Also in the case none of the above is relevant, so let's go forward and have this in :)
Comment Actions
I've seen this too late (the change was already committed), but for the record I'm also against HTTPS for master sites (HTTPS is generally a huge pain for me, and I tend to avoid it when it is not required). Oh well, I guess I can maintain a local patch to bsd.sites.mk (with this and other similar revisions reverted).
Perhaps it would make sense to make preferred order (http[s]/ftp/etc.) configurable?