|
|
Post by jisidro on Oct 5, 2017 7:56:07 GMT
But ofc, you conduct tests and assign variables as you see fit.
|
|
wishing
Junior Strategist
Posts: 353 
|
Post by wishing on Oct 5, 2017 8:07:36 GMT
Thanks for the explanation. So saying "you are making a type 1 error" is the same as saying "you are drawing a conclusion that goes against the strongest hypothesis, but that the data doesn't support"?
Sounds a bit like saying "we need more data".
|
|
zich
Junior Strategist
Posts: 690
|
Post by zich on Oct 5, 2017 8:19:00 GMT
Part of the issue here is probably talking about error and all that, when we are not even in agreement over the hypothesis.
|
|
wishing
Junior Strategist
Posts: 353
|
Post by wishing on Oct 5, 2017 8:24:17 GMT
Yeah. I feel like the general idea of many people is that the basic assumption is that Cryx is overpowered, because that feels like common sense, like how people cannot fly. And therefore Cryx being overpowered is considered the strongest hypothesis. Where others might see it the other way round.
Which maybe shows that we can't really apply scientific methodology to a casual conversation thread. We aren't writing a paper. We are airing opinions about the results of a game tournament.
|
|
|
|
Post by jisidro on Oct 5, 2017 8:59:59 GMT
Thanks for the explanation. So saying "you are making a type 1 error" is the same as saying "you are drawing a conclusion that goes against the strongest hypothesis, but that the data doesn't support"? Sounds a bit like saying "we need more data". You can be running a test that is innapropriate for the kind of data you are analysing, you can draw the wrong conclusions, etc... etc... etc... Need more data is usually a bogus reason. You have to be really carefull about logic leaps that aren't there... for example, we can come to the conclusion that Cryx won too much (whatever too much means) in the WTC... usually you follow with it'll win too much in other tournaments but that is a different conclusion that could need different analysis than "did cryx win a lot more games in the WTC that we would expect from a perfectly balanced game?"... I do believe the trend will continue but let's see... It's a pity a few casters had to be sacrificed in the Altar of OPness for PP to decide for a little more hands off approach. Assuming they follow-through with the policy. |
|
SeBM
Junior Strategist
Posts: 102
|
Post by SeBM on Oct 5, 2017 9:15:40 GMT
I think jisidro's last post nailed it. Casters like Ossyan, Karchev, Una2, Wurmwood, High Reclaimer and Haley2 got nerfed. Most of these casters did not enjoy the level of dominance Cryx has had in the past 8 months or so. Let's hope PP can see that...
|
|
|
|
Post by gribble on Oct 5, 2017 9:33:56 GMT
you're misoperationalizng the cryx variable if you're tutting to use chi squared as a predictor for individual performance. Cryx total wins at WTC vs other Factions total wins tells us nothing about individual performance. Maybe I'm missing something, but it tells us that Cryx won more individual matchups than we would expect, everything else being equal. And the difference was statistically significant. Of course there could be a number of factors for this beyond the Power level of Cryx - as outlined by others - but we simply don't have enough information to test hypotheses on these other factors. Your tests seem to only be analyzing the effect playing Cryx had on the team standings... which honestly seems to be of much less value than analyzing the performance of Cryx in individual matchups. Again, unless I'm missing something in what you're trying to test/prove. |
|
|
|
Post by Gamingdevil on Oct 5, 2017 9:47:21 GMT
you're misoperationalizng the cryx variable if you're tutting to use chi squared as a predictor for individual performance. Cryx total wins at WTC vs other Factions total wins tells us nothing about individual performance. Maybe I'm missing something, but it tells us that Cryx won more individual matchups than we would expect, everything else being equal. And the difference was statistically significant. Of course there could be a number of factors for this beyond the Power level of Cryx - as outlined by others - but we simply don't have enough information to test hypotheses on these other factors. Your tests seem to only be analyzing the effect playing Cryx had on the team standings... which honestly seems to be of much less value than analyzing the performance of Cryx in individual matchups. Again, unless I'm missing something in what you're trying to test/prove. If I'm getting it correctly, the significance of this is that people may have thrown someone under the Cryx bus to win the other matchups. This might still indicate that Cryx is too strong, or at the very least perceived to be, but it makes the actual Cryx win rate much less significant because they would've gotten a vast majority of favourable matchups. |
|
|
|
WTC Stats
Oct 5, 2017 11:23:49 GMT
via mobile
Post by octaviusmaximus on Oct 5, 2017 11:23:49 GMT
Thank you all for the clearer explanations. Professorlust seems to be particularly averse to educating people  |
|
|
|
Post by shonkhor on Oct 5, 2017 12:58:41 GMT
Hi ProfLust: I think it is great that you are taking the time to conduct a proper statistical approach to analysing the WTC data. As I read the comments, the frustration a lot of people are having with your analysis so far is that the question most people are interested in is whether playing cryx (or any other faction) influences whether you win or lose a game in the WTC format, after controlling for factors such as player skill, team skill etc., not whether having cryx on your team once or twice helped a team win a round, or multiple rounds.
Therefore can I suggest setting the level of analysis at the individual game level. Analyse win/loss as a binary dependent variable using a glmm with a logit link function (see lme4 if you are using R). Including dummy coded faction identity (cryx vs not cryx; ret vs. not ret etc.) as fixed effects, and playerID and teamID as random effects. Use AIC to select the best supported model – this should indicate which factions had an important effect on win rate and estimate the size of the effect.
|
|
wishing
Junior Strategist
Posts: 353
|
Post by wishing on Oct 5, 2017 13:07:59 GMT
Shonkor the statistician has entered the match.
|
|
|
|
WTC Stats
Oct 5, 2017 17:17:34 GMT
via mobile
Post by professorlust on Oct 5, 2017 17:17:34 GMT
Thank you all for the clearer explanations. Professorlust seems to be particularly averse to educating people Enroll in a class if you want an education. |
|
|
|
WTC Stats
Oct 5, 2017 17:34:35 GMT
via mobile
Post by professorlust on Oct 5, 2017 17:34:35 GMT
Hi ProfLust: I think it is great that you are taking the time to conduct a proper statistical approach to analysing the WTC data. As I read the comments, the frustration a lot of people are having with your analysis so far is that the question most people are interested in is whether playing cryx (or any other faction) influences whether you win or lose a game in the WTC format, after controlling for factors such as player skill, team skill etc., not whether having cryx on your team once or twice helped a team win a round, or multiple rounds. Therefore can I suggest setting the level of analysis at the individual game level. Analyse win/loss as a binary dependent variable using a glmm with a logit link function (see lme4 if you are using R). Including dummy coded faction identity (cryx vs not cryx; ret vs. not ret etc.) as fixed effects, and playerID and teamID as random effects. Use AIC to select the best supported model – this should indicate which factions had an important effect on win rate and estimate the size of the effect. Oh I've done that. To follow along start here. fivefingers.boards.net/thread/361/world-team-championship-2017-thread?page=12What's important to note is that though yes cryx as a faction did well but simply playing cryx actually has statistically signifcant negative effect on your teams winning. Having 2 cryx players on your team helped your team winning, having 1 cryx player helped your team but not as much. So cryx can help the team but being a cryx player didn't help your team, which indicates that cryx didn't buoy players enough to overcome play errors, poor matchups, or weak teammates. Thus if you want to take away meta implications about individual tournament from the self contained meta of a large team event, go for it. It's poor operationalization and even worse analytical process. |
|
|
|
WTC Stats
Oct 5, 2017 17:43:44 GMT
via mobile
Post by professorlust on Oct 5, 2017 17:43:44 GMT
For the record, and to be more helpful than our statistician friend: Type 1 errors occur when you incorrectly reject the Null hypothesis (ie. find an effect/correlation when there is none.) Type 2 errors occur when you incorrectly fail to reject the Null hypothesis (ie. find that there is no effect/correlation when there in fact is one.) Type 1 errors are generally considered more egregious in most fields of science. However, I have some issues with Professorlust's methodology. In particular, the way he's controlled for player skill seems...off. If winrate is the variable you're using to measure power, then using 'perfect winrate' as a proxy for player skill (and removing 'highly skilled players') is naturally going to have a circular effect on your findings. Not circular when the VIF is 1.2 or Lower. Additionally 6-0 players occurred on non 6-0 teams. So if we're going to assume that cryx is over powered, we also need assume that cryx not only over performed individually in matchups that Players can arrange, which they cannot do in standard SR, we also need to investigate whether any particular individual win, helped the team. This is because since matchups can be arranged, amy good matchup should hypothetically indicate that there were overall team benefits for matchup. Assuming every matchup is 50:50 in a format where teams can seek to avoid or least mitigate the impact of bad match ups is fallacious . However, as I repeatedly lamented, we don't know anything about the matchup process for each round. However, simply because we don't know the matchup process doesn't mean we get to ignore the matchup process. Our ignorance, in my opinion, does far too much to undermine the claim that Cryx is OP as opposed to my more nuanced Cryx did well but to assign blame for Cryx's performance to Cryx being OP at design level when there a number of factors which contribute to both individual and team performances. |
|
Grimolf
Junior Strategist
Posts: 246
|
Post by Grimolf on Oct 5, 2017 17:55:33 GMT
To move away from the statistics discussion for the moment ...
I’ve been thinking about Cryx’s fantastic win-rate at the WTC, and I have started to wonder what role the new Steamroller 2017 rules might have had in helping Cryx along. I haven’t really seen many people talking about this, but it seems to me that the greater amount of terrain on the board with 2017 is great for Cryx. All of the terrain helps block LOS to Cryx’s delicate ‘jacks, protecting them from shooting so they can get up the board. Furthermore, with all of the Ghost Walk casters, it is easier for Cryx to get around the terrain (even buildings), and Dark Host ramps ghostly up to the point that terrain doesn’t affect Cryx’s infantry much, except to block LOS. Cygnar and other shooting factions are probably far less happy with all of the LOS blocking terrain, and factions that don’t have access to things like Pathfinder are probably unhappy as well. So, is SR17 in some way partly responsible for Cryx’s great success at WTC?
When it comes to Cryx’s reviled casters, the Witch Coven has been strong since the start of Mk3, but not completely overpowering. What changed was the introduction of SR17. The Coven brings a whole bunch of spells that aid in maneuverability, which is what makes them tricky to play against, and it also helps them to overcome the terrain (or make better use of it). Finally, the Coven also has a new advantage in SR17 in that it can split up to score multiple scenario elements, although I don’t know how often that happend in WTC games. So, I’m not sure that the Coven is the same problem without SR17 - again, they are strong caster, but they weren’t a major problem until SR17. Deneghra1 is perhaps another story, but part of her strength in SR17 is being a Ghost Walk caster in a world with lots of terrain.
So, again I wonder whether SR17 is partly responsible for Cryx’s success. Did Cryx do so well in part due to the fact that it has multiple ways to take advantage of and get around all of the terrain? I’d add that the Cryx themes encourage well-rounded armies (lots of infantry, solos, and then a few ‘jacks bought with battlegroup points), which also likely helps with putting on scoring pressure in SR17.
Any thoughts?
|
|
