Reported to plugins@:
—
Revision 105138 appears to be corrupted in the SVN repo:
http://plugins.trac.wordpress.org/changeset/105138
…
I ran into this because svnsync fails to
read it as well, so it’s impossible to mirror or backup the repo
because of this one revision (though there could also be more past
this revision that are also corrupt, but I won’t know until it gets
past this one).
—
Barry 2:35 pm on December 24, 2012 Permalink | Log in to Reply
We are looking into it
Bryan Petty 1:47 am on January 10, 2013 Permalink | Log in to Reply
Out of curiosity, are you able to perform a raw dump of this revision at least?
$ svnadmin dump -r105138 –incremental /var/svn/repos/plugins > r105138.dump
(using the appropriate path to the repo of course)
I’m not able to using svnrdump (SVN 1.7.5), but if it can be done from local filesystem, that dump should be helpful at least.
Bryan Petty 1:20 am on January 16, 2013 Permalink | Log in to Reply
What I’ve found is that r105305 is the actual commit that r105138 was intended to be considering r105138 was committed by “plugin-master” (the script for creating new plugins in the repos once they’ve been approved), starts with the same plugin name before hitting incorrectly encoded characters, and the log even mentions the same WP.org user account. At no point in between was there another commit to the completely broken tree that r105138 introduces.
Testing this out with a valid SVN repo on 1.6 (the same version running on the plugins SVN repo), I’m able to simply turn such a commit into an empty revision (with only revision properties) just like what happens when you run
svndumpfilterwithout any issues. However, this is while making use of svndump, which I know is possible with the plugins repo, but would require some *extensive* work with setting up a fixed mirror (from the filtered dump, and kept up-to-date either with additional dumps in sync with cron or possibly with svnsync) that would be turned into the production server at the flip of a switch somewhere (in order to minimize downtime).I’m not familiar with any other way to fix this though since this specifically involves a broken filesystem node, and not just a broken revision log message (which is all I’ve ever run into before, and those are simple – just a svn:log property edit).
Andrew Nacin 6:56 am on December 25, 2012 Permalink | Log in to Reply
From what I recall (from @westi), there were some corrupted revisions from way back when, so I would guess earlier than 2009, so I would not be surprised if there are more earlier than this.
Barry 2:23 pm on December 25, 2012 Permalink | Log in to Reply
I don’t think they are corrupted – it just look like some character set incompatibility. Should be fixable.
Peter Westwood 11:21 am on December 30, 2012 Permalink | Log in to Reply
Agreed, from memory they look corrupted but actually are fine if you use the right locale to fetch them.
Bryan Petty 4:50 am on January 3, 2013 Permalink | Log in to Reply
There’s not any revisions earlier than this one like this, I’ve already been through every revision before r105138. I’m just not sure about later revisions yet.
Peter Westwood 9:15 am on January 4, 2013 Permalink | Log in to Reply
To follow up I checked my IRC logs from 2009 and there was a corrupted revision but that was fixed it was somewhere in the 87000 numbers.
garnser 4:38 pm on February 1, 2013 Permalink | Log in to Reply
So I’ve looked at this a bit now, as mentioned the locale seams to be messed up upon creation of the plugin in the repo, I’m guessing this was some kind of temporary glitch somewhere given that the actual folders for the project was properly created and there’s multiple revisions later as well.
In theory this is fixable, however it would require doing a dump and load (which takes a good 5h when not using svnfilter) so right now I’m looking into whether it’s possible to make some modifications in the FSFS-db to mitigate the issue since dumping and loading really isn’t an option with this big of a repo.
Barry 10:40 pm on March 23, 2013 Permalink | Log in to Reply
Quick update. We have made some progress here and should have a working copy of the plugin repo without any broken revisions online soon-ish.
Scott Reilly 5:37 pm on March 28, 2013 Permalink | Log in to Reply
Related: core trac ticket #23107
Barry 10:19 pm on April 23, 2013 Permalink | Log in to Reply
Howdy! Thanks to @garnser – the plugins repo should be corruption-free. It should also be a bit faster – still working on some more improvements there.
garnser 11:56 pm on April 23, 2013 Permalink | Log in to Reply
Some background on this, as mentioned earlier the issue was related to locales. SVN <1.5 did check for non UTF-8 characters but didn't reject them even though SVN doesn't support it. In later versions SVN does reject non UTF-8 characters and issues like this shouldn't be able to appear again.
As the commit was corrupted it was replaced with a dummy-entry.