|
Post by Trollock on Jan 12, 2018 22:03:59 GMT
Since i started a heated debate over dice math in the Madrak 1 thread, i figured id move that discussion over here, so innocent bystanders who wish to read about actual warmachine can skip the discussion in this thread So in the ancient tradition of "arguing on the internet with strangers of things that do not matter", here we go OK, so i had decided that i would drop the subject (since it way off topic by now, sorry every one...), but this is basically foul play here, quoting math and hoping no one will read or understand it But assertions with nothing to back them up are okay? Not trying to pull any fast ones, just seeing a lot of people making claims, and not really having anything to back them up. So I'm providing some links and the like. As for your excel test, it really depends. Really. I also won't play against somebody who uses any sort of computerized random number generator because they're SO easy to get wrong. Like really wrong. And not have it be obvious. Excel isn't any different, and is probably worse than usual. There have been entire scientific papers written on how to get a truly random number out of a computer. To summarize, you don't, though it's really easy to appear like you have. What you will get is something close to random, generally called pseudo-random. If i rolled 6 12/30 times on a gaming night, that would be a good reason to call me lucky, but if you use such a simplistic test to call my dice crooked you would be plain wrong. Okay, based on what please? I see you making a lot of direct assertions here, and not really providing anything to back them up. You've basically gotten one result for 1/3 of your rolls. Seems like a skew to me, and not an insignificant one. A lot of people would call over a judge at that point, and maybe ask for a bucket. So you are arguing that the random number generator in excel is to blame. It is incapable or generating a number that is "random" enough for this test? I have a rudimentary understanding of how a computer generates a random number, and i can buy that there are some advanced applications where the number is not "random enough" but this cant possibly qualify... As for your second question im not sure i understand what you mean. My point was that i used a proper random number generator that is not biased (for the record i plotted the average result of the same random number generator and it moves closer and closer to 3.5 the more times you "roll", so it should be correct enough for this discussion) and got an outlandish streak of sixes. My point is that the math in the article is solid, but doing the test with only 30 rolls of the die is simply not enough. Further down in the same article they write about an example with a d20 that is so biased that it NEVER rolls a 1 and has twice the normal chance to roll a 20. Even then, you would had to roll the die 400 times to be "sure" that it was biased. A normal die will have a bias that is WAY WAY less obvious than a d20 with 2 sides that read "20" and no side that reads "1". The whole point of my argument is that even if normal gaming dice are biased (and they are since it is impossible to manufacture an exactly cubic piece of metal with exactly even density) the bias is very small. There is simply no way that one is able to tell if some one has a bunch of dice that roll slightly high just by playing games against them. You would need to record thousands of rolls and preform statistical analysis to detect those tiny variations. Assertions like "that guy always rolls high" are confirmation bias unless the player actually uses weighted dice that are REALLY biased. Those dice exist, and there exist ppl who will use them to cheat even if i have never encountered one of them. Im talking about the minuscule bias of a dice where you tried to make a fair die.
|
|
|
Post by robbleyourworld on Jan 12, 2018 23:22:56 GMT
I couldn't resist getting involved in this. Statistics, cognitive heuristics, and arguments with strangers three of my favorite things!?
Generally speaking people are really bad at estimating probabilities. We use several different heuristics that work "good enough" in naturalistic settings but when determining truth behind probability they often fail. This gives rise to a lot of the fun biases we see surrounding die rolls.
Regarding the article that was linked, while its not quite wrong it takes substantially more than 5 observations/cell to get an adequately powered chi-square statistic. With statistical testing you really want at least 20 observations per cell at minimum. Though given the tiny effect size of biased die rolls you'd need thousands of observations to get a well powered test to reliably check if a die is really biased
|
|
|
Post by cayterpius on Jan 13, 2018 0:24:27 GMT
2 dice is 7, 3 dice is 10. The rest is luck. What are we fighting about?
|
|
|
Post by deviantcad on Jan 13, 2018 0:46:40 GMT
Glade you moved this to another thread.
There is nothing wrong with the excel RNG for something like this at least. It uses time as a seed and correct me if I'm wrong Trollock, but he probably didn't do them in 1 min intervals down the the ps or what ever place it is using.
If someone rolled 12/30 6s and you called a judge I'd make sure they dice were not rolling straight 6 then move on. That is under 2 standard deviations and a confidence interval is 27-30% ( did it twice to double check but rounded different each time it is close enough for the point at least) 30 is just to small of a sample size. flipping a far coin 4 times would give you what a 25% chance to realize that yes it is fair. I don't think it was meant to scale that low, even a as trollock pointed out with the d20 example farther down.
I apologizes I can't remember your handle, but I believe it was the member who has capsize as their avatar picture, talked about dice a while back. I really agreed with the core of his argument that I will hopefully not paraphrase to badly here. If you alter your dice to roll better you are cheating. If you cull your dice even in what is likely a math mathematically negligible way, your intent is to cheat and your cheating. I don't care if you want to put a "bad" dice off to the side or roll the "hot" ones more often go for it, if you think you're imbued with luck and it gives you the edge it doesn't hurt me anymore then you thinking you're better at math. If someone goes home and rolls dice 2000 times and picks the best ones? that is probably cheating. I can't imagine catching someone who has 5% better dice in a game, only if they always roll this one dice when they need a 6 or something else sketchy.
In a game you probably aren't going to roll enough dice to tell. If my opponent insisted on recording every dice I rolled because they didn't trust me, we both might be better of finding other people to play.
What! 3 dice is clearly 10.5 pitchforks everyone! honestly I forgot and just started rambling.
|
|
|
Post by greenjello on Jan 13, 2018 2:23:25 GMT
So you are arguing that the random number generator in excel is to blame. It is incapable or generating a number that is "random" enough for this test? Really, what I'm saying is "it depends" It's been a while since I tried to do anything with Excel, but IIRC, there are about 3-4 different ways to get different distributions of probability out of it, and that's before we start talking about the underlying processor, seed numbers, and other things. I also don't claim to be an expert on RNG, but I've read enough papers on it to know a few things. 1) Computers are not random. It's a machine designed to produce determinant results, not random ones 2) Generally computers do random via a table, and then a seed number 3) Once you have those they usually generate a number between 0 and 1, which is then used to create those D6 rolls you're interested in. Without some really solid understanding of how to get from step 3 to D6 there are a number of subtle traps involving what it really means to do modulo operations, and other mathematical operations to generate those results. One thing I can point out is that it's not really possible to divide 100 by 6 evenly, you end up with an irrational number, so that's your first problem. So I have no idea what you might have done in Excel, nor do I claim to have the understanding of Excel to know what it's doing under the hood once you have selected something. In short I don't trust it, since I do have some idea of what "random" might mean, and it's very different depending on what you're trying to accomplish. My point is that the math in the article is solid, but doing the test with only 30 rolls of the die is simply not enough. Okay, I understand what you're attempting to communicate, but I do not accept that it is true based on your saying so. I'm hoping you have something you'd like to add that backs up your assertion. If not that's fine too. Even then, you would had to roll the die 400 times to be "sure" that it was biased. Right, but that's a die with 3x more faces, you're going to need more rolls to figure out it's a problem. The whole point of my argument is that even if normal gaming dice are biased (and they are since it is impossible to manufacture an exactly cubic piece of metal with exactly even density) the bias is very small. And this is where we disagree. I've got another article, this time about Chessex and GW die that show a pretty clear and obvious bias. I can also tell you that I've seen obvious air holes in at least one of my dice. If I can find it again I'll post a picture. www.dakkadakka.com/wiki/en/That's_How_I_Roll_-_A_Scientific_Analysis_of_Dice Assertions like "that guy always rolls high" are confirmation bias unless the player actually uses weighted dice that are REALLY biased. Those dice exist, and there exist ppl who will use them to cheat even if i have never encountered one of them. Im talking about the minuscule bias of a dice where you tried to make a fair die. Not necessarily. I think we agree that confirmation bias is a thing, but think about the fool who buys GW/Chessex dice (which were shown to be biased) and has rotten luck. He's at the game store, so he get disgusted, throws away his old set, buys a new one, also from Chessex, or GW, since those are his options, and find his luck unchanged. He has now decided that his luck has followed him, instead of blaming the crappy dice, of which he keeps getting the same ones. Mean time his buddy buys a completely different set, or brings some from another game, etc. This set doesn't have the flaws of the GW/Chessex dice, and he always uses them. Suddenly he's the "lucky" guy who always rolls well. Neither is necessarily looking to have good/bad luck, they just have different equipment. Anyway, as you've pointed out a few times this is a pretty silly discussion. Personally I find the idea of random to be interesting, while I realize it's not everybody's cup of tea. In this case the best plan is to buy a set of casino dice, and use tactics that minimize probability. (ie don't field a Blitzer until they fix the random shots!)[/quote][/quote]
|
|
|
Post by macdaddy on Jan 13, 2018 4:52:01 GMT
NERDS!!!!!!!!
|
|
|
Post by Trollock on Jan 13, 2018 13:02:34 GMT
greenjelloHey thats a really nice link you got there! Assuming it is not made up (and i will assume it is not made up) that is the kind of test one needs to preform to detect dice bias. I am baffled at the size of the bias shown in that test tbh. That is something that could indeed have an effect on gaming performance. Note that they did 1000 rolls with each die (not 30) though, and that is much more along what i would expect to be reasonable. The size of the bias is WAY bigger though. Also, it only explains why some one would roll LOWER than average since all dice tested seemed to be biased in that particular way. I have been playing with the GW-typed dice for 25 years now, and i have never felt like i had worse dice than i should, but that is its own kind of confirmation bias. I expect the dice to roll average so the average rolls is what i remember I might have to take my own dice for a spin and see if i can repeat the experiment. If i am almost twice as likely to roll a 1 than i would expect, i should easily be able to detect the difference. Also, it seemed that all the GW dice were biased toward "1", so i could roll a bunch of them at the same time to save my wrist the effort
|
|
|
Post by Trollock on Jan 13, 2018 22:48:02 GMT
OK, so i have no life... I do not want to play with dice that are obviously biased, especially not if they roll 1s all the time, so i decided to test one of my GW dice that i use for gaming nights and tournaments. This is an old battered veteran die with some visible flaws from use. It has the rounded edges that the article greenjello linked talked about as very negative too. Just look at it! I rolled the die 1000 times and kept track of the results. Every 200 rolls i did a check to see the results so i could follow how the results changed over time. First we can check the average result rolled in the graph below. As you can see, the average roll started off a bit high, but as i roll more and more dice, the average goes towards 3.5, just as one would expect from a d6. That is a relief at least. Even a visibly flawed die seems to roll fairly average. Next i also kept track of how often each face came up. This graph is not super intuitive, but ill try to explain. If a die is truly random, each face should come up 1/6 of the time, or ~16.7% of the time. I added a black line in the graph at the 16.7% level. Then i plotted how often each face came up after 200, 400, 600, 800 and 1000 rolls. As you can see, "1" was not as common as it "should" have been after 200 rolls, but the more times i rolled, the closer to 16.7% the results is. After 1000 rolls it is still less than expected, but the trend is still increasing, so it is not unreasonable that if i roll 2000 times or more, it will end up close to 16.7%. "2" is slightly more common than it should be, and there is no obvious trend that says it will go down as i roll more dice. Over 1000 rolls, there was an 18.1% chance to roll a 2. This is high, but it could be random still. As you can see for the other numbers the results have not yet converged at their final value, and rolling more dice can still reveal this number to be close to 16.7%. "3" is slightly less common than expected, and if anything, the trend seems to be moving away from 16.7%. Over 1000 rolls there was a 15.1% chance to roll a 3. "4" Started off as high, but as i roll more dice, the probability goes towards 16.7% as expected. Over 1000 rolls there was a 17.6% chance of rolling a 4. "5" seems very stabile. One could have expected 5 to be less common than 16.7% since 2 (the opposite face) seemed to be more common than 16.7%, but that was not the case. Over 1000 rolls there was a 17.2% chance to roll a 5. "6" also seemed stabile over time. One could have expected 6 to be very common, since the "1" was very uncommon, but that was not the case. Over 1000 rolls there was a 16.4% chance to roll a 6. So, my conclusion here was that i was not able to reproduce the results of the posted article. There they claim that basically all dice tested had a ~25-30% chance of rolling a 1 over 1000 rolls, and i am no where near that bias. Either this particular die is of extra good quality or one of us has miscalculated badly. The average result converges at 3.5 as one would expect pretty fast. 1000 rolls does not seem to be needed to see that. After 200 rolls the result was a bit high, but after like 500 rolls we are very close to 3.5. The chance of rolling each number was not exactly 16.7% over 1000 rolls, BUT the results are still fluctuating, and one should keep rolling dice until the results no longer fluctuates to be sure, but I at least am very confident that my GW die does not have the ~30% chance to roll a 1 that was talked about in the article. After this test i do not feel a need to buy new more precise dice. The old battered ones seem to work just fine.
|
|
|
Post by greenjello on Jan 13, 2018 23:24:03 GMT
LOL, NERDS indeed.
So I believe the original article was written a while ago, about 2007ish, so let's say 11 years. Do you know how old your die is? I remember a number of people observing that GW dice (never heard about Chessex until the article) had problems. Since yours is an old battered die, could be it's a different batch. The article was also written by a prof from ASU, so it could be the dice in question are were all made in the US, while yours was made in Europe somewhere.
Anyway, this is the sort of bias that can creep into die manufacturing when looking good, not accuracy, is the most important thing. While there might be some dice that are fine, why take the chance? Personally I test out new bricks when I get them, usually averaging around the 30 roll mark. Generally that's enough to detect enough of a bias to matter.
And really that's what it comes down to, enough bias to not be lost in the noise. 30% seems like enough to be caught by a smaller number of test rolls than 1000.
It's also the reason why I use casino dice. They're obviously not perfect either, but they should be good enough, which is the only thing that matters. If one side is 5% more likely to come up, I don't care.
|
|
|
Post by greenjello on Jan 13, 2018 23:27:21 GMT
|
|
|
Post by deviantcad on Jan 14, 2018 5:00:46 GMT
I've got to give it to you Trolllock that might be the most work I've seen someone do to change their anecdotal dice math into scientifically tested.
Makes me not feel to bad about using my veteran dice.
|
|
|
Post by robbleyourworld on Jan 14, 2018 5:50:55 GMT
That die has some seen some stuff
|
|
ravenfire
Junior Strategist
Me Mulg me SMASSSSSHHHH
Posts: 244
|
Post by ravenfire on Jan 14, 2018 7:23:26 GMT
Ah the age old discussion about random generation of numbers by a computer. Believe me that is least 25 years old.
My 2 cents are: the computer is “random” enough for a game like this.
If you are designing let’s say encryption software, well then it is time to look closely at the random number generator
|
|
|
Post by greenjello on Jan 14, 2018 14:08:31 GMT
Ah the age old discussion about random generation of numbers by a computer. Believe me that is least 25 years old. My 2 cents are: the computer is “random” enough for a game like this. Maybe. My big problem is with people trying to use their own home brew dice roller. Having tried to write a few of these, I find that they almost always have subtle bugs that result in skewed rolls. Often times the skew is pretty obvious. (Maybe I'm just a bad/ignorant programmer) And that's before we get into questions of whether or not the person in question has written the code to skew the results deliberately. So when Trollock announced that his Excel macro was producing results that were off, my first impulse is to think bug. Since he didn't post how he was doing it, it doesn't seem an unreasonable conclusion. FWIW, here's an interesting discussion on RNG in Morheim when they replaced their first RNG with another one because the first while random, had a problem with streaks. I know games programmers aren't the best programmers, but they have taken some time to think about it, and still apparently got it wrong. (Or wrong enough to want to change it) steamcommunity.com/app/276810/discussions/1/485624149151076918/?ctp=3Counter point is that X-COM claims to be correctly random, yet cheats for the player to avoid cries of the RNG screwing people. Personally I never had a problem with the X-COM RNG, but there seemed to be a lot of people complaining about it. Hard to tell if it was confirmation bias, or people upset about streaks.
|
|
|
Post by Trollock on Jan 14, 2018 14:30:41 GMT
As a short follow up, i would be careful about doing small sample size testing. Not only did i actually get the excel random generator to roll 12 sixes in 30 rolls (that would make any one suspicious) and even is that is not "truly" random, it will be close enough to random enough to represent a dice. I also did some small analysis of the 1000 dice rolls i made yesterday. I pretended that instead of doing 1000 rolls, i did 5 sets of 200 rolls, and plotted the number of times each face came up in those 200 rolls. You should expect 33 rolls of each kind in 200 rolls. In the first test i would suspect that the die was broken and not rolling enough 1s. If i had only preformed the fifth test though, i could suspect it was MORE likely to roll a 1 than it was supposed to. In the first test i got 33% fewer 1s than expected. In test 5 i got 24% more than expected. Now this is the SAME die, so if it is biased, it is as biased in the first test as in the fifth, right? If i had only rolled 30 times rather than 200 i would have gotten even more varying results. Also, in the 4th test the die was 30% more likely to roll a 4 than expected. In the 5th test it was 24% LESS likely to roll a 4 than expected. The trending analysis seem to be important to see if you need to roll more dice to obtain an accurate result. Had i only dome 200 rolls i could have drawn very different conclusions than i did from 1000 rolls, but the trending analysis allows me to predict that i still have not obtained a stabile value for each face of the die. My point is simply, that rolling a die only a few times (less than 1000 or what ever) is probably not a very good way to check for bias. Rolling more times is more accurate, but also terribly annoying. As for the age of this die, i have no idea. It is super unlikely that is it from the same batch of dice used in the article, but it has the same rounded edges (allegedly terrible for randomness) and it even has some visible flaws on the surface. It may or may not contain air bubbles, though in the quoted article basically the entire bias disappeared when sharp edges was added to a die. Any air bubbles in the plastic seemed to have only minor effect on the randomness of the die. So, we have a very well written article that claims to have made an extremely ambitious experiment that shows cheap round edged dice to be wildly inaccurate. I tried to reproduce the results using one of my own dice and was unable to do so. If i was a suspicious person i could suspect the dakkadakka article to be nothing more than sponsored content by the company who was conveniently linked in the article selling casino dice, but im not that paranoid. I wont be buying new dice for myself anyway. Mine seem to work just fine. EDIT: Seems the original article linked on dakkadakka did in fact not have a link to a site selling casino dice. That makes sponsored content even less likely to be the case imo.
|
|