Mk/bsd.sites.mk: Update MASTER_SITE_GCC entries

[diizzy]

Summary

Refresh mirror list with working mirrors and not stale mirrors (quite few doesn't carry snapshots for example)
Drop ftp mirrors
Use main site as fallback

[diizzy]

I'd to have a second pair of eyes in case I overlooked something :-)

[salvadore]

Thanks @diizzy, I will check the mirror list you propose as soon as possible.

[salvadore]

A few questions about your patch:

• on https://gcc.gnu.org/mirrors.html I find the following list of mirrors that is missing from your patch: https://ftp.gwdg.de/pub/misc/gcc/, http://robotlab.itk.ppke.hu/gcc/, http://ftp.tsukuba.wide.ad.jp/software/gcc/, https://mirror.marwan.ma/gcc/, https://ftp.nluug.nl/languages/gcc/, http://mirror.linux-ia64.org/gnu/gcc/, https://mirrorservice.org/sites/sourceware.org/pub/gcc/, https://bigsearcher.com/mirrors/gcc/, http://www.netgull.com/gcc/, http://mirrors.concertpass.com/gcc/. The page also lists http://gcc.cybermirror.org/ and http://gcc.fyxm.net/, but I could not manage to have them working;

• on https://gcc.gnu.org/mirrors.html I could not find the following mirrors included in your patch (or already present before): https://mirrors.kernel.org/sourceware/gcc/, https://ftp-stud.hs-esslingen.de/pub/Mirrors/sourceware.org/gcc/, https://ftp.funet.fi/pub/mirrors/sourceware.org/pub/gcc/, https://ftp.mirrorservice.org/sites/sourceware.org/pub/gcc/ ;

• according to https://www.gnu.org/prep/ftp.html, a nearby mirror could be automatically select by using https://ftpmirror.gnu.org and http://ftpmirror.gnu.org and main servers are available at https://ftp.gnu.org/gnu/, http://ftp.gnu.org/gnu/, ftp://ftp.gnu.org/gnu/, rsync://ftp.gnu.org/gnu. I have tested http://gcc.gnu.org/pub/gcc/ and it works, but I have not found any link to it on the gcc website;

• I am unsure why do you want to remove ftp mirrors. My guess is that it is for security reasons, but in that case I would also remove http mirrors and keep https only. Also, remember that we use checksums, so we always check file integrity, from ftp and http mirrors too. I think there are other security concerns for ftp, so maybe you want to disable it for them, but what about http? Is file integrity the only security concern in this case? Also, I have not found any warning against using ftp and http mirrors in the porter's handbook, nor recommendation to use https.

Finally, I would change the patch by putting, in order, https://ftpmirror.gnu.org , http://ftpmirror.gnu.org, https://ftp.gnu.org/gnu/, http://ftp.gnu.org/gnu/, ftp://ftp.gnu.org/gnu/ (excluding maybe ftp and http) and removing all the mirrors now listed. In alternative, I would use the mirrors list from https://gcc.gnu.org/mirrors.html (excluding maybe ftp and http and surely the few that do not work). The first approach has the advantage that users would be able to download the sources from a nearby mirror, while with the second approach they use the first mirror that works in the list. Also, it seems that https://ftpmirror.gnu.org and http://ftpmirror.gnu.org have more mirrors than those listed on https://gcc.gnu.org/mirrors.html.

Moreover, please note that we have a MASTER_SITE_GNU definition and that https://ftpmirror.gnu.org , http://ftpmirror.gnu.org, https://ftp.gnu.org/gnu/, http://ftp.gnu.org/gnu/, ftp://ftp.gnu.org/gnu/ are links for GNU software, which includes gcc. So it is probably better to update MASTER_SITE_GNU with this list of links, then define MASTER_SITE_GCC by using the value of MASTER_SITE_GNU, or we could also remove the MASTER_SITE_GCC variable and use MASTER_SITE_GNU directly in the gcc ports (those do not include only those that I maintain, there are a few more).

In case you want to touch the MASTER_SITE_GNU variable, I recommend asking for review from some gnu software maintainer.

[diizzy]

Hi,

That url ( https://gcc.gnu.org/mirrors.html ) is outdated and contains broken mirrors such as netgull.com and the other ones you've listed unfortunately which is annoying.
Confusingly GNU GCC mirrors used for the ftpmirror.gnu.org sub domain don't carry the "full set" such as snapshots which is why GNU mirrors (alias) isn't used in the first place afaik, see https://ftpmirror.gnu.org/gcc/ for example

A few sites such as mirrors.kernel.org (GeoDNS) have very good overall connectivity which why they're used, there's no need to force users to use slow mirrors just and we don't need a ton listed, if 9 sites are down you're likely to have other issues than mirrors being down.

ftp mirrors are removed as they're being deprecated overall including providers are removing them so there's no need to keep those around. If both https and http is broken you're unlikely to have working ftp connectivity anyway.

Hope that clears up things,

Best regards,
Daniel

[salvadore]

That url ( https://gcc.gnu.org/mirrors.html ) is outdated and contains broken mirrors such as netgull.com and the other ones you've listed unfortunately which is annoying.

I double checked and you are right about netgull.com: although it works, it is outdated, so we cannot use it. However, all the other mirrors I listed work and are up to date: do they work for me only? If they do not work for you too, of course we cannot add them. And anyway, as you write, we do not need a ton of mirrors.

Confusingly GNU GCC mirrors used for the ftpmirror.gnu.org sub domain don't carry the "full set" such as snapshots which is why GNU mirrors (alias) isn't used in the first place afaik, see https://ftpmirror.gnu.org/gcc/ for example

Indeed, you are right.

ftp mirrors are removed as they're being deprecated overall including providers are removing them so there's no need to keep those around. If both https and http is broken you're unlikely to have working ftp connectivity anyway.

I am fine with removing them then.

The patch looks good to me. I would only change one link if it works for you too (see inline comment).

Mk/bsd.sites.mk

254

Can we change this one with https://mirrorservice.org/sites/sourceware.org/pub/gcc/, which is listed on the official mirrors list?

[diizzy]

Mk/bsd.sites.mk

254

Sure, I'll fix that. Thanks!