Topic on Project:Support desk/Flow

Issue :

We followed the official mediawiki documentations and made a bash script that upgrade mediawiki following these steps :

1. Back up existing files and the database
2. Unpack the new files
3. Upgrade extensions
4. Run the update script to check the database
5. Test the update
6. Remove leftovers from old installations

The Script was working like charm few months ago, but now while we were trying to upgrade mediawiki the script failed at step 3. Upgrade extensions with 429 too many requests error specifically after running the command :

git submodule update --init --recursive

Part of the error :

Cloning into '/var/www/wiki-dev/wiki/fr/extensions/Wikibase/view/lib/wikibase-data-model'...
error: RPC failed; HTTP 429 curl 22 The requested URL returned error: 429
fatal: the remote end hung up unexpectedly
fatal: clone of 'https://phabricator.wikimedia.org/source/wikibase-data-model.git' into submodule path '/var/www/wiki-dev/wiki/fr/extensions/Wikibase/view/lib/wikibase-data-model' failed
Failed to clone 'view/lib/wikibase-data-model'. Retry scheduled
Cloning into '/var/www/wiki-dev/wiki/fr/extensions/WikibaseLexeme/resources/special/new-lexeme'...
fatal: unable to access 'https://phabricator.wikimedia.org/diffusion/NLSP/new-lexeme-special-page.git/': The requested URL returned error: 429
fatal: clone of 'https://phabricator.wikimedia.org/diffusion/NLSP/new-lexeme-special-page.git' into submodule path '/var/www/wiki-dev/wiki/fr/extensions/WikibaseLexeme/resources/special/new-lexeme' failed
Failed to clone 'resources/special/new-lexeme' a second time, aborting
fatal: Failed to recurse into submodule path 'extensions/Wikibase'
fatal: Failed to recurse into submodule path 'extensions/WikibaseLexeme'

My Question :

Is this normal?, It's been more than 12h and we are still getting 429 error, and are there some ways to prevent this from happening again?

I appreciate any help.

You are probably using anonymous access right now and I suspect that has much more stringent rate limits. Try using (ssh) authentication.

you could try cloning from github instead.

Thank you guys for answering.

@Bawolff im considering cloning each extension from GitHub if i don't find any other option.

@TheDJ Yes probably for this reason, i appreciate if you can point me out to setting up ssh connection with phabricator server to download the extensions without rate limiting.

i assume this documentation : Gerrit/Tutorial is the one

i've setup ssh connection between my server and gerrit server as well as configuring git as mentioned in the docs

here is a connection test :

ssh -p 29418 myusername@gerrit.wikimedia.org

  ****    Welcome to Gerrit Code Review    ****

  Hi myusername, you have successfully connected over SSH.

  Unfortunately, interactive shells are disabled.

  To clone a hosted Git repository, use:

  git clone ssh://myusername@gerrit.wikimedia.org:29418/REPOSITORY_NAME.git

Connection to gerrit.wikimedia.org closed

Git config :

user.email=myusername@mydmoain.io

user.name=myusername

url.ssh://myusername@gerrit.wikimedia.org:29418/.insteadof=https://gerrit.wikimedia.org/r/

and still when i run git submodule update --init --recursive i get the error 429, it's been over 20 days now.

The error is coming only from Phabricator server not Gerrit, so probably i need to setup ssh connection with phabricator as well.