So I was pleasantly surprised by the wealth of wisdom and good discussion that followed on from my first post, and I hope that continues. For this post I'd like to pick up on one comment from that discussion that opens up a new discussion:
In my eyes endurance comes down to two different issues. Firstly, there is the number of rounds, and secondly there is the relative length of these rounds.my 2 cents to begin with, dont have too many rounds. it becomes a test of endurance rather than a test of your solving skills.
To begin to address the first issue, it is helpful to examine the typical schedule of a WSC. This has varied from year to year, but generally the entire competition, including individual rounds, team rounds and play-offs has fit into two days of competition. A day is typically split up into morning, early afternoon and late afternoon sessions. I've done a little (hasty - please point out any mistakes) research and summarised the information in a table.
In terms of the first issue, we can see that a WSC is decided over roughly 5-6 hours of competition, with not too much variance.
WSC Individual Team Play-off 2013 Beijing 5h 40m (7 rounds) 1h 25m (3 rounds) 1h 45m 2012 Kraljevica 4h 30m (7 rounds) 0h 55m (2 rounds) 3h 00m 2011 Eger 6h 40m (10 rounds) 2h 20m (2 rounds) 1h 00m 2010 Philadelphia 6h 05m (10 rounds) 3h 10m (5 rounds) 1h 00m 2009 Zilina 4h 10m (5 rounds) 3h 45m (4 rounds) 1h 15m 2008 Goa 5h 00m (7 rounds) 1h 30m (2 rounds) ? 2007 Prague 5h 30m (6 rounds) 2h 00m (2 rounds) 2h 00m 2006 Lucca 4h 15m (8 rounds) - - 2h 00m
This brings me on to the second issue, length of rounds. In my research there were basically 3 types of round. The first are shorter, 10-30 minute rounds, which are typically sprints or one-off novelty. The second is the more bread-and-butter style rounds which are typically 35-60 minutes. The third seems to be very much the exception, the longer 90+ minute rounds.
As far as I can tell, there have only ever been 3 such rounds: the 120 minute round from Zilina, and the two 88 minute rounds in Beijing. In contrast, I found it remarkable that no round was longer than 45 minutes in Philadelphia - which I have long regarded as the gold standard for a WSC.
My first post has already discussed the potentially skewing effects longer rounds can have, given they inevitably feature many harder variants; instead I'd like to look at things from an endurance point of view.
The first thing to say is that every now and again most solver will have a bad round. The days are long and intense at a WSC, you are perhaps battling the effects of jet lag and so it seems inevitable that your concentration will lapse. If this happens half way through a particular round, then the longer the round is, the more you will be punished.
A slightly different angle, which every solver is familiar with, is when you get halfway through solving and you find that you have made a mistake. I know that when this happens to me it can often cause me to lose focus and confidence and affect the rest of my round - particularly when the puzzle is worth lots of points and I don't want my time to have been completely wasted. Again, the longer the round is, the more you can end up punished.
The argument then goes that if you have more and shorter rounds, there is more chance to reset your mind and recover during the breaks, and approach the round after a bad one in a much better frame of mind. One bad round doesn't have to make or break your championship.
As a counter-point, I'd also like to remark that most online competitions are typical 120 minutes long, and might be argued to be bigger tests of endurance than any single WSC round. But regardless of the length of the competition, we tend to see the same old names at the top the majority of the time.
I could go on for longer, particularly with regards to WPC influence where rounds are often longer, but I think now is the appropriate moment to let my audience make up their own minds, and offer their own perspectives and insights into the issue of round length. I look forward to your comments!
The hardest aspect to compare are certainly the playoff times because they vary in format so much. Did you use official scheduled times for your playoff tally or something else? I'd say some of these should be different, but it does matter if you want to stress puzzle time or playoff clock time.
ReplyDeleteLucca was meant to last 2 hours but took about 4.5-5 given the media, room resetting, and an error in the final puzzle that organizers caught before it ran but needed to fix. We actually missed lunch in Lucca. Solving on an empty stomach is a different kind of endurance.
Prague, with 4 rounds of eliminations was basically 1.5-2 hours as you have it.
Goa had a semifinal and two finals. The first took ~45 minutes as I recall, and the latter probably averaged 45 minutes as well too with a max puzzle time of 15 over 4 puzzles but solvers could finish the round collectively early and often did.
In Goa, a solver good at all sudoku puzzles would be involved in 2.25 hours of playoffs. Other solvers perhaps just 1.5 hours or 45 minutes depending on what they qualified for.
Zilina had a semifinal and final that took altogether many more hours on the clock, but there was a defined gap between the events where most players were able to protest their results. It was also never exactly clear when the Final ended because of disputed scoring.
Philly certainly was meant to last at most an hour and basically did. We were about a half hour late due to final scoring considerations.
Eger ran for less than an hour, but was three hours late on the schedule. I had to be stewing in playoff mode for far longer than I wanted and I'm sure I frightened many children and some adults during that time.
Kraljevica did last 3 hours but an individual solver might be playing for just 1 hour during that time given the long time gaps between quarterfinals.
Beijing is accurate as you have it by total time, with an hour long semifinal or so and then a reset of the room to a finals that took about twenty minutes.
The data from the play-offs was largely taken from the schedules, which is why there is a blank for Goa - I wasn't there and I couldn't find a schedule.
DeletePlay-offs are definitely a topic that deserve their own post. Probably quite a long one where I attempt to analyse them one by one!
My perspective as a mediocre solver is: I don't like short rounds. The rational argument is, this is because of the additional timing issues and random factor this adds for me.
ReplyDeleteSince I need more time for a single puzzle, it's harder to choose the right puzzles and not waste too much time (compared to the round length).
Of course, this is also a question of available difficulty of the puzzles, but just take a 20 minutes classic sprint with comparable medium difficult puzzles. I have a good day and don't do any mistakes. Chances are, I'm finishing three puzzles and barely miss the fourth one. If I have a very good day, or I get a lucky guess when I realize that I will not finish otherwise, I will get four. The difference for me is huge. The (relative) difference between 9 and 10 puzzles is not that big.
Also, I need quite some time every round to get into "the tunnel". But this is personal preference, and might be the opposite for other solvers, who start to lose focus after an hour or so. From my experience, I get much better if the rounds get longer. For this reason, I really like the 2 hour online contests, but this might be too long for a WC round. This is also the reason, why I think, there should be different round lengths at a WC.
As a mediocre solver, I agree. A 15 minute round will probably net me a zero, while a 30+ minute round will not. As for very long rounds? I will just share this story. Round 9 WPC this year: after the round started, I had an urge to go to the bathroom. While I was able to held out for the 1 1/2 hours, my results were not too good. While I am not sure whether my bathroom issue was at fault, or it was just me being bad, I will blame the bathroom.
DeleteThe difference between finishing 3 or 4 puzzles vs. finishing 9 or 10 right at the end of a round is definitely a solvers frustration that should be considered - clearly you have more problems the more rounds you have, assuming a roughly constant total championship solving time. An entire championship of 20 minute sprints is probably not the way forward!
DeletePerhaps part of the issue here is reducing the element of puzzle choice from the rounds, and having much bigger proportions of solvers being able to finish a round.
I think I'm a good solver and I still face similar problems with shorter rounds. I generally disagree with this point you make -
Delete"If this happens half way through a particular round, then the longer the round is, the more you will be punished."
For me its more like, if it happens in a smaller round I don't have recovery time. If it happens in a bigger round, I have enough recovery time to make it up. Thats generally why longer rounds go relatively better for me (you can see my Beijing 88 minute round stats compared to other rounds).
Does that mean I have better endurance? Maybe. But I don't think thats a wrong thing, I should have a chance, as one of the competitors, at one of my strong points too, if there's gonna be multiple shorter rounds which favor others.
Obviously I've used myself as an example but looking at the comments above, this is mostly a general categorization of players, and its wrong to do away with the longer rounds that actually suit some players because others find it difficult to concentrate.
So the first point is that, as I noted in the post, rounds longer than an hour at the WSC didn't exist until this year (I'm going to ignore Zilina). I'm not sure it's a question of "doing away with them" as you put it.
DeleteThe second point is, surely you'd prefer to be "recovering" when you weren't wasting the opportunity to score points? This is the point I'm trying to make. A bad round is a bad round and you'll be losing points regardless of whether it is long or short. But you lose more points to your competition for a bad long round than you do for a bad short round. You cannot argue that long rounds and short rounds are of equal importance, because they aren't. Longer round have more points at stake!
Now, as for whether endurance should come into things, this is more important. I think most solvers are going to be swayed by their own personal experiences here, but I don't think anyone can deny that endurance is certainly an attribute that the top solvers currently have, as well as speed. The names at the top seem to be roughly the same regardless of the format of the competition.
Ok, lets forget about "doing away", lets just argue about the length as is.
DeleteYou say longer rounds have more points at stake. Taken as a single round, yes. But over the entire bunch of rounds, thats just not true. At least not for everyone. I'm not solving a long round and having a 2 hour break in between, neither am I solving a 20 minute round and having a 2 hour break in between. The breaks are random and based on meal times, so it always depends on the bunch of rounds rather than a single one. I can do with a long round where I recover in between, it won't affect me for the next round as much as a short round where I couldn't recover does.
I'll give an example of say, 3 20 minutes rounds together. I personally am far more relaxed on one 80 minute round where I know I can recover, whereas in a 20 minute round I know it might be a bit of a hit and miss towards the end, as the proportions are more defined if I don't finish a puzzle, as pointed out above by Christoph, and so I'm going all out. If I make a mistake, that puts me in a bad mindset going into the next 20 minute round, where also I feel hurried, and then another. Going into rounds and making a mistake when you're hurried is much worse than when you're relaxed.
The same mistake on a 80 minute round doesn't hit me as hard because I "feel" there's time to move to the rest and rectify things. If I lose a 50 pointer to a mistake in a 200 point round, thats 150 I'm getting max, whereas if I even make 3 50 point mistakes in a 800 point round, I'm getting 650 max. Proportionately thats way better and more important than whether or not thats true, I feel that way while solving, and I'm sure there are others like that.
So its not at all true that I'll be losing points regardless of the length of round. If I have an entire long round going bad, I have myself to blame, just like I have myself to blame for having a bunch of 20 minute rounds go bad. My point is, its different for me and different for you what we're good at, I feel much better with longer rounds, but I'm not angling for having all the rounds be 80 minutes, am I? So having both long and short rounds would be the fair way to go about it, giving everyone a chance with what they're good at.
About endurance, I don't think people are denying that its an attribute thats needed usually, the point of contention is mostly that some feel thats a necessary part of a world championship and others don't.
Of course this was all just to put things in perspective about others' preferences and how long rounds do benefit some people. I'd be fairly happy with 3-4 60 minute rounds over the 2 days too, as long as there's no crazy 15 minute, one puzzle, all-or-nothing type round. I think the WPC had it right in that when its essentially a single puzzle (even connected. I'm talking about Round 8 here as opposed to say Round 3/10 of the WSC), it should have much more people finishing to not have too much variance. Obviously with a good partial bonus structure in place.
DeleteIn terms of rounds, which was the initial thrust of the post, I do think it is valuable to have rounds of sufficient length for some solvers to finish or at least earn a majority of the points. A stunt round like the Great Wall round this time at 15 minutes could have been much better at 20 minutes, perhaps. But with longer rounds, when testing shows the round is too long, the proper approach is probably to remove some puzzles and not to make the round 2 hours of time.
ReplyDeleteI'll note the difference in mentality between this tournament and the American Crossword Puzzle Championship, which was an earlier brainchild of Will Shortz. In that tournament there are seven rounds that each feature just one puzzle. The best solvers all finish with about 75% or more of the round time remaining except on the very hardest round #5 where maybe closer to 60% of the round time might be remaining. That competition also takes a full day to complete, but much less time is spent puzzling.
I'm not recommending a 7 puzzle WSC, but I am thinking there is a tournament somewhere between the current 70+ puzzle event and that sized event which works best for solvers. I would not mind taking the concept of "my 2 cents to begin with, dont have too many rounds" and removing about 33% of the rounds from these championships, lengthening the remaining rounds by 50%, and using a scoring system that is robust to small errors by solvers who finish quite early in some of these rounds.
I would certainly be more likely to still be competing if the endurance effect was removed somewhat, considering the WSC and WPC are now consistently back to back with no rest days.
One of the more interesting experiences at the WPC this year was finishing the black and white matrix round with a decent chunk of time left, and meeting Palmer and Bram outside. Both were really on edge about having made a potential and just how badly that would screw up their championship if they'd lost the bonus. As it turned out, the 80% bonus for a small error seemed to work out quite well - and I think such a partial bonus system for small errors is essential if you are going to have these rounds where there are potentially huge bonuses at stake for the best solvers.
DeleteThe closest personal experience I have comes with Times Sudoku championship, where a preliminary round consists of 4 classics and I'll be done in 10-20 minutes of the allotted hour. There's no bonus at stake, but it does kind of suck having to wait around for so long (although granted a lot of that is being forced to sit at your desk). I'm not sure I'd be too comfortable with WSC rounds where you had people finishing with more than 50% of the time still on the clock. But as you say, there is a lot of scope for that % to come up from what it is at the moment.
Perhaps another interesting statistic to consider would be total number of puzzles at various championships. I think that one important consideration is to ensure that you have a sufficient number of puzzles so that your championship actually carries meaningful weight.
An interesting thought experiment is to consider the GP final as a WSC round. I actually think it's a good model for a round - you had some easier puzzles, some harder puzzles but nothing too extreme on either side, all with plenty of novelty and good design. Test solving had this as a ~35 minute round for good solvers, and I think Kota's winning time was closer to 32 when you discard the checking periods. There are a couple of questions to consider.
1) What time would you allot if this were a WSC round?
2) How many puzzles would you throw out if you wanted it to be a 30 minute round?
I think my answers are 1) 45-50 minutes and 2) Probably two, of "middling" difficulty.
Finally, your point about the Great Wall touches on what is being said in a previous comment. I think my observation here is that this is definitely a high variance, all-or-nothing puzzle, and one where perhaps the average/mediocre solver wasn't considered. With respects to the competition in Beijing, I'd also say the team rounds could have used an extra 10-20 minutes solving time.
See, I think endurance should be a quality a good solver should have at a WSC/WPC. When people ask me about these championships I tend to comment that it's a whole different experience to solve Sudokus and Puzzles for 2 days on end. That's the whole difference between the World Championships and any other test that you can do online. So I find it weird to say that testing endurance is a bad thing.
ReplyDeleteI think the lengths we had in Eger, Philadelphia and Beijing were nice total lengths. I think a general length of 30-60 minutes per round is good for Sudoku Championships. There's not the variance in Sudoku as in Puzzling that you can justify the 2 hour long rounds like in the WPC (which I absolutely love there).
Right - having multiple rounds on top of each other is another aspect of endurance, regardless of how long the rounds are. I think you are right to say that when you have 1-2 days worth of puzzling, endurance in this form is always going to be a major factor in who comes out on top.
DeleteI haven't done the stats for the WPC, but it's my impression there is generally more solving time at a WPC than there is at a WSC, which would mean it's less likely that a long round dominates the final classification. Which is kind of what I was discussing in the previous post.
Perhaps another angle on this issue is how long is enough for the breaks between rounds? I think the 10-15 minutes we had in Beijing was just about ideal...
I think the WPC is generally longer, considering it's usually 2 full days of puzzling followed by the final the next day, while the WSC generally has 1 and a half day of puzzling with the final in the evening of the second day.
DeleteBefore I forget -- Beijing breaks were well done. They also ran on schedule, which is commendable.
ReplyDeleteI think it's a good idea to figure out what rounds we *should* have at a WSC before we figure out how long the round should be. The Philadelphia system was terrific in this regard, with puzzles grouped according to style and occasionally difficulty.
Maybe there's something to be said for someone being good at sprints and middle-distance and endurance as well?
I actually think there's value to all of this; varying styles, varying difficulties, varying round lengths. How you match a style with a length is negotiable and maybe should be left to the discretion of the organizer. Even easy/medium/hard classics...while I think we'd all expect about 8-10 puzzles of each type and would have round length vary directly with difficulty, it would be interesting to have an endurance round of easies or a sprint-y medium round, no? Granted, the World Record puzzle in Zilina was effectively a single hard puzzle in a sprint round, but then rounds comprised of one puzzle are probably a bad idea anyhow...
So with all this in mind -- I think a decathlon-style WSC is a fair choice, tho maybe I'd formalize the different round types, perhaps to match or nearly match the Chinese idea for this year, at least when it came to variants. Varying difficulties of classics, math round, familiar variants round, odd shapes round...it seems like a good idea. Maybe a pure innovative round (like they had in Italy) is a welcome addition.
And since I've added virtually nothing to this discussion, I'd like to point out that once/if the round categories are standardized...let's make them all worth the same amount too. So not only is the classic sprint given its own round vis-a-vis the math round, but it's *value* is the same as well. Maybe 1000 pts. per round, bonus is whatever fraction of time you have left (times 1000), ten rounds, some long and some short, no fear. Sure, why not?
I think you've picked up on the line of my thinking linking these first two posts, and in particular your suggestion about weighting the rounds is probably even more applicable to the first post. It goes against the conventional wisdom that longer harder puzzles should be worth more points, but as you say it all depends on what you are valuing in your would-be champion, and it probably is worth exploring...
DeleteI'm certainly with you in terms of having clear and well-defined links and themes running through the rounds, and I've already got some good ideas and inspirations :)
Given the total length of championship is roughly fixed to two days, I think the length of each round has almost nothing to do with endurance if the balance of difficulty/points and sudoku types (classic/variant/innovative) are well-adjusted throughout the competition. Someone may feel longer rounds tend to lose his/her concentration, but it’s also true repeating short-term rounds (and intervals) will lead difficult to keep his/her stable tension.
ReplyDeleteIn the other point of view, increasing the number of rounds will generally increase a chance for competitors to be forced to use guessing or trial-and-error because these techniques are often used just before the end of a round, especially in the short-term round or over-difficult round.
Personally, I like different length rounds, although short rounds with only one puzzle to solve are a bit daunting for me, as there is a big probability that I'll end up having zero points (as happened to me with Great Wall).
ReplyDeleteThat said, more important for me is that the individual + team rounds extend to two days. As the sudokus are my main interest in these competitions, it is frustating if competing (from very personal perspective) lasts only for a day.
Just voicing my thoughts :-)
Thanks Ulla and Yuhei, your contributions are much appreciated :)
ReplyDelete