mm-2399 === Subject: Re: An home-made Random Generator Distribution: world This looks to be a variation on the Blum-Blum-Shub pseudorandom number generator. The reference can be found at: author = {Lenore Blum and Manuel Blum and Mike Shub}, title = {A Simple Unpredictable Pseudo-Random Number Generator.}, volume = {15}, number = {2}, year = {1986}, pages = {364-383}, bibsource = {DBLP, http://dblp.uni-trier.de} } === Subject: Re: An home-made Random Generator Distribution: world Mainly out of curiousity, when I feel the need to use a good random number generator, I usually use the Mersenne Twister. Not because I've researched the properties, but because I've seen it quoted by people as being good, it's fast, and there is no shortage of free code implementing it. I basically use random numbers for the bootstrap, simple simulations, permutation tests, etc. I do not as far as I know need a cryptographically secure random number generator. In fact, for many quick hacks I just use C's drand48(). Any comments on other random number generators I should look at. Ross-c === Subject: Re: An home-made Random Generator Distribution: world Hi Ross, I think that to make the distinction of cryptographically secure is a bit of a furphy --- Cryptographers are interested in pseudorandom number generators that are fast. Essentially, a stream cipher is a pseudorandom number generator. The important thing is that the output looks random --- that is, it is not predictable and has the statistical properties of a random distribution (but can be replicated by anyone with the key). In this case it is the unpredictability that is important. Given that, any well designed cipher or hash function will do the job, although I think stream ciphers might be easier to adapt.. I would recommend Rijndael (also known as AES) and RC6. These can be used to generate a stream of zeros and ones, which can be transformed into any number range you need. There is also optimised code available for Rijndael from NIST. One of my colleagues has written a utility for which always one to specify the range and the type of output. It is then able to generate millions of pseudorandom values in a matter of minutes. Hope this helps, Ursus === Subject: Re: Craps probability question: Clarification of Expert {SNIP} out of the equation. Agreed on the odds not changing; see my earlier reply to Timmyrocker regarding the side bets as a way to tilt the odds your way, or at least less towards the house. Paul === Subject: Re: Craps probability question: Clarification of Expert Timmyrocker, I learned early on to ignore Reef Fish. He never seems to respond to the question posted but uses it as a springboard for a rant to prove how clever and wise he is and what a dolt the poster is. I don't think I've ever gotten any useful information from him. Stereotypical Internet bully, probably borderline schizophrenic from the way he uses all caps for emphasis. I agree that in the minimalist view of the game, shooter against the house, one bet on the outcome, the odds don't change. The reason I asked is that I've observed craps being played at casinos and noticed that there were fresh bets after each roll. Taken from http://www.winneronline.com/rules/craps.htm , An Odds bet is a backup bet this I thought that perhaps the History Channel experts were hedging their original bets with odds bets, like going short or long on a foreign currency when you buy stock in a company in that country. Paul === Subject: OT: (Flame) War on Nobel Peace Prize? Haven't seen that pyramid pattern for some time. I've dropped out of the Nobel Peace Prize thread some time ago, and don't want to intrude the ongoing discussion, but I thought this group can use a bit of levity and light humor occasionally, and so I am pulling this out as a separate OT thread and hope that Google will keep it as a transient subject and not thread it into the original. what to whom, you're a much better TREE than me. Some one said, Where does the TREE come in? Well, my last name, in Chinese, is the character of a tree, or forest, consisting of two pieces of wood side by side. So, that made me interested in TREES. I think the Bleeding heart liberals are sumetime called tree huggers. But I am not absolutely sure. See e.g., http://www.commondreams.org/views05/0215-29.htm The kind of TREES I am most familiar with are TREE stumps which I think is a term attributable to Woody Allen, and I've had first hand experience with many of them in the classes I taught, at a certain University which shall remain nameless, because most of its admininstrators are BIGGER Tree Stumps. Woody met many tree stumps during his college career, I think, judging by his autobiography: I went to NYU myself, I was a philo-major there, too. I took all the abstract philosophy courses in college, like truth and beauty, advanced truth and beauty, intermediate truth, introduction to God, Death 101. I was thrown out of NYU my freshman year, I cheated on my metaphysics final in college, I looked within the soul of the boy sitting next to me. They threw me out, and my mother, who is a really sensitive woman, when I got thrown out of college, she locked herself in the bathroom and took an overdose of Mahjong tiles. Then there's the saying Can't see the Forest for the Trees for those who views on a subject is rather myopic. I've found some tree stumps in newsgroups who can't even see the TREE, let alone the forest. Then there are all kinds of trees in mathematics and statistics, such as a tree in graph theory, a tree diagram or dendrogram in cluster analysis, a minimal spanning tree, binary trees, binomial trees, k-trees, and other TREES that have little or no humor or comical content except when they are misapplied. So, I'll stop here, return you to your regular channels, go outside to take a few fresh breaths from the TREES surrounding my house, and call it a day. -- Bob. === Subject: Re: OT: (Flame) War on Nobel Peace Prize? I've always thought of tree huggers as a subset (proper or not, I'm not sure) of bleeding heart liberals. :-) Aww, give them a hug! They've earned it. Russell === Subject: how to convert covariance to something like a CV I've been struggling with this problem for weeks and am getting nowhere. I've a series of X,Y measures whose ratio should be the same. When my data is well behaved, my collegues and I take the shortcut of calculating a simplistic standard deviation of the multiple ratios and report that. We think of this in terms of error bars telling us how accurate our metaphoric meter stick is. We like Of course with ratios, there's problem with asymetry so the stdev calculated of a series of X/Y isn't the same as of Y/X and that's just plain wrong. I can make up things to be symmetric, such as using but don't know if that's kosher. An alternative is to look at linear regressions with a fixed intercept of 0. This is a fairly good way to get a metric of fit but I'm at a loss of how to convert an R value or a covariance to something like my old friend the standard deviation. Please forgive the gross abuses of terminology, it's been a long time since I did any of this in anything approaching a formal manner. --Wade Hines wade.hines@rcn.cannedmeatproduct.com === Subject: Re: how to convert covariance to something like a CV The conventional treatment of ratios is to take a log. The obvious complete solution is to start with the log of the ratios, and log at the mean and SD of that. Is there a reason not to do that? Simplicity? Are you possibly inventing a problem for yourself, or do you have real divergency? If both values are relatively far from zero, I can imagine where you would have ave(X/Y) = 0.5, 95% CI (0.4, 0.6) versus ave(Y/X) = 2.0, 95% CI (1.60, 2.40) . Now, those endpoints for +/- 20% error range do not translate into exactly the same values, but is that your need? Or do you just need to know when another set of data gives you (say) twice the interval? Regression gives you the regression coefficient, and the SE of the coefficient. http://www.pitt.edu/~wpilib/index.html === Subject: Re: how to convert covariance to something like a CV I agree with Rich that it is likely to be easier to work with logs (it's common to use logs when dealing with ratios), but I guess there's a need to be sure that it will relate sufficiently to what your requirements are. You mention that sd(Y/X) isn't the same as sd(X/Y), but it's just as true that ave(Y/X) isn't the same as 1/ave(X/Y). Does that casue you problems? [Working on the log scale will have ave(log(Y/X)) = - ave(log(X/Y)) and sd(log(Y/X)) = sd(log(X/Y)). ] You mention error bars. Are you interested in some kind of interval for the ratio (a prediction interval for say the next, yet to be observed ratio)? Are you interested in some kind of interval for the /average/ ratio (a confidence interval)? An interval for some other statistic, perhaps? Are you able to describe in more detail what properties your meter stick should have? What should it tell you? Glen === Subject: Re: Nobel Prize in Econ ...................... You mean they are not PR pieces? .................. Well, here is one doing exactly that. The government, and government officials, have NO legal responsibility to the public. States can legislate some of this for their bureaucrats, and possibly the federal government could, but it has not. Read what I have written above about fraud. A libertarian society would take everything those Enron executives had, and not even allowed them to get any public welfare; this is the libertarian punishment for deliberate actions causing financial harm to others. And it would be monetary, with compensation to the victims having first precedence, then compensation to the government, and possibly never having any stupid jail sentences. Jail as a punishment for crime is a stupid late medieval invention. There are valid purposes for jails, but punishment is not one of them, except POSSIBLY for abuse of power. Those who choose liberty do far better in getting equality than those who choose equality in getting any sort of liberty. I am in favor of the right to try, but not in any form of forced sharing. Libertarians believe in voluntary charity, but not the state. Our current federal charity budget is huge; about 1/3 of Social Security should be counted there, and all of Medicare and Medicaid. And our miseducational system destroys more minds than it helps. Nor will you ever. You are obliged not to initiate action to cause harm, but you are not obliged to give of you time or substance to help. If you are honest, even though your actions inadvertently cause harm, your liability is at most highly limited, if it is there at all. I think this should have been threatened, but before doing this, we should have blockaded the country and seized everything going in and out by sea, and bombed the land routes. Of course, warnings should be given before taking such action. As for being moral, I see nothing immoral about it. If you are attacked, you have the moral right to do what it takes to defend yourself. Nuking would probably not have been necessary. When one is dealing with a fanatic mob, such actions are necessary. The Founding Fathers were scared stiff of a democracy. The major purpose of a constitution is to hamstring the government. A democracy has NO minority rights, other than what the majority deigns to give it. In particular, they believed that the downfall of the Athenian and Roman republics was due to using the public treasury for boons. I find this questionable for Athens, but right on for Rome. The Roman Republic was done for quite some time before Caesar. ---------- ---------- The Norwegian Parliament, which is the Peace Prize agency, is strongly socialist. And both Abe and I have commented on the other Peace Prize recipients. In fact, did the Kellogg-Briand Treaty (limiting naval armaments) contribute to peace? I am much less worried about a nuclear war than about the world becoming a bunch of totalitarian states. In the past, this was not that much of a danger, as one could find a relatively unpopulated place without a strong government, and which could still accommodate a society. Now, this does not exist. When libertarians attempted to put a small settlement on the unclaimed Minerva Reef, Tonga claimed it. If the US supported liberty, it would have stayed. Similarly, it was the major powers who helped put down the Biafra secession attempt, the attempt of the capitalists to secede from Congo, and quite possibly the project by a private German company to launch cheap satellites from Congo for third world countries; Angola claimed that their rebels were based there, but they never were found. I believe that a _Time_ survey found more than 7 million libertarians in the US. This is NOT a small number. If the Alaskan ballot proposal was adopted nationally (there is no chance of this; both parties have objected to anything similar) and plurality voting for any office eliminated, libertarians would be able to vote for their candidates without electing the worst of the evils. -- This address is for information only. I do not claim that these views are those of the Statistics Department or of Purdue University. Herman Rubin, Department of Statistics, Purdue University hrubin@stat.purdue.edu Phone: (765)494-6054 FAX: (765)494-0558 === Subject: Re: Nobel Prize in Econ I didn't write that. OK, but you need to address it to wayward libertarians, not me. Hmmm, in the sense that they can not be sued for diligently doing their jobs, that is true. Among other things that prevents people from suing because weather can not be forecast perfectly. OTOH the IRS doesn't give any more accurate tax information and they don't deal with a chaotic system (dynamically speaking, although some people will claim the tax system is chaotic ;-) ). I did, and you're the first libertarian I've heard say that other than in the abstract. Did Milton Friedman say something about Enron I missed (it is entirely possible)? In other newsgroups people decried it when it happened, even though it was off topic. Did I miss the outbreak of outrage here? Maybe I've forgotten... Yes, and voluntary charity has not been sufficient for the need at any time in history of which I am aware. In my experience the libertarian reply is now something along the lines of But when libertarians run things everything will be enough better for everyone that the need will be sooo much less. Another Utopian fantasy, IMO, but YMMV. Anyway all the libertarians I know well enough to make a determination use that as an excuse to keep their money, IOW in their cases voluntary largely means none. In fact, there was an entire book written on the subject, _Cyberselfish_. Again, YMMV and for all I know Friedman gave his entire Nobel Prize to Mother Teresa for the poor, but if he did he did a much worse job of making it known than he has with his philosophy and his economic ruminations. Maybe he's modest. I don't see how libertarians would fix that. But that was exactly my point, one can not do that with certainty. Not even morally? Ahh, an amoral Utopia, what a concept. I'll pass, thank you. So, is there a line between inadvertent and negligent? But do we the right to use any means independent of the provocation? Do we have the right to kill millions of people who are noncombatants and just happen to be there? I think not. Even Curtis Lemay said that if the U.S. had lost the war with Japan, he expected to be tried as a war criminal for ordering the fire bombings of Japan. Since we can't test that, it is speculation. Some were, some were not. They were all concerned, perhaps, but scared stiff overstates it, IMO. I'd put it another way. Yes, but there have been few pure democracies, perhaps for that reason. And libertarianism, by supposedly giving people maximum freedom, also gives the members of the majority maximum freedom to join together an oppress the minority. I don't see that as a solution. We've been over that in the past, and I agree. Actually, you both have mentioned only a couple, with which you both appear to have some sore spots. Did it make it any worse? Peace isn't easy, the criteria for success must be set accordingly. Baseball players who succeed 1/3 of the time are in the Hall of Fame, and getting peace is harder to do than hitting a baseball, IMO. I'm more worried about bio-war. And IMO the majority of them are not pure libertarians. They are people looking for a justification to feel good about not giving to charities and griping about taxation. Of course, there may be no higher percentage of hypocrites among libertarians than any other religion (which is what it appears to me to be as much or more than a political movement; some people treat Rand as a god). I must say it is refreshing to interact with a libertarian who apparently really believes the whole package. You should be familiar with Arrow's Impossibility Theorem. More parties is not necessarily the answer. I don't know what the answer is, but I'm humble enough to admit it. :-) This has gone even off the off topic topic of Nobel Prizes, so I'm about ready to drop it. Russell === Subject: Re: Nobel Prize in Econ .................... The points where this has been raised come from not providing police protection. This includes not responding to 911 calls, and not acting to enforce a restraining order; this was a recent case, and this led to deaths. BTW, as for IRS information, only recently has legislation been passed exempting taxpayers from penalties for following incorrect information from IRS agents. And look at the objective press criticizing people for taking questionable deductions; of course they should take them until the real question has been decided, and there needs to be a way to get it decided without having to pay penalties and interest. There is also a difference between diligently doing their jobs, and excesses from them. The sex-abuse cases should all have been civil, and the civil judgments would often be FAR less than the costs of the criminal trials. Also, the civil suit laws need to be revamped to prevent frivolous suits; drugs have been removed from the market because of such suits, never won by the plaintiff, and a relatively cheap means of getting peak power to New York City finally was dropped after a decade of civil suits by environmentalists, none of which they won. Possibly the need is misconstrued. If you know the story about the grasshopper and the ant, this is libertarian. If you believe that the ant is obliged to support the grasshopper, you are a totalitarian left-winger. Someone does not get the right to support from a stranger because of being born, or even as the religious fanatics would have you believe, from being conceived. I have seen the problems of the world likened to lots of people adrift in a dangerous sea, with some in lifeboats which can hold 60 in a calm sea. If those in lifeboats let an unlimited number in, they will all drown. The socialist idea that we must support all will make all poor. The African nations are not poor because we do not give them enough money, but because they cannot act in a libertarian capitalist manner. And as the world is vastly overpopulated, we need competition for survival. Otherwise, we will end up as a world of marching morons. In my experience Most libertarians are more sensible than that. The idea that all can live well is the Utopian fantasy. The idea that we know how to do things well is another. Anyway all the libertarians I know well Give money to who? I would like to give much money to a good educational system, but there CANNOT be one which is controlled by the government or the present educationists. Get rid of the overhead. As for the miseducational system, libertarians would support schools and other methods which TEACH, not indoctrinate or stifle the minds. If you have seen students, you would know that their mental abilities vary greatly, and that only someone who cannot understand would put children in with their age groups. The new math failed because the TEACHERS could not understand it, and still cannot. One does not learn concepts by memorizing. The level of mathematics education is far lower than it was a half century ago, and teaching calculus does not raise it. Whose morality? I consider keeping a child from learning more because the others cannot to be HIGHLY immoral, and even has a destructive effect on thinking. But this is the current educationist morality. No Child Left Behind means No Child Gets Ahead. If a drug turns out to have bad effects which could not have been tested for, that is inadvertent, not negligent. If it would have been too expensive to test for that, and the total results of all tests were made known, this MIGHT be negligent, but also should not be actionable. The current political situation, with the demand for imposing morality, either of the right or left, precludes getting a good president elected. We need freedom, not uniformity of any kind. You seem not to have considered the actions adequate provocation. It was an act of war by the mob. When the government of Iran failed to act against it, it meant that they condoned the actions, and this made it already an act of war. To consider counteractions corresponding to the severity of the action, one needs a police force willing and able to do so. This does not exist, has not existed, and will not exist as long as people wish to impose their morality; when there is a menace, and the powers will not face it, it is necessary to face it alone, if one can. When the UN allowed U Thant to withdraw the UN troops from the Sinai border between Israel and Egypt on the demand from Egypt, the credibility of the UN was GONE. And would YOU trust the UN now, with most of its members being bad to their own people? Do we have the right to kill millions of people If they supported the mob, they were combatants. Anyone who wants to impose his morality is a fanatic. This makes Bush a fanatic, in particular. But so are Gore and Clinton and the tax-and-spenders who will take our money and spend it where their morality tells them it should be spent. It isn't always obvious, IMO. Anyway, IYO the options With past history as a guide, a libertarian society would have a very strong constitution, and an armed citizenry. It would have a government with few powers over law-abiding citizens, and also responsibility for government officials. Would there even be a coalition of the majority? And would such a coalition have the power to do so? Of course I am familiar with it; he and I had almost adjacent offices when he was doing it. He did not require consistency under randomization; if this is added, as I point out in my paper on the foundations of rational behavior, I get the results very quickly. But this is not the idea of a proportional representation government; I know the problems with that. The Alaska proposal was the single transferable vote for the election of a single person. This was rejected by the League of Women Voters as being too complicated; all one would have to do is list the candidates in order, so one could have instant runoffs. I do not know if we would have more parties, but minorities would be heard from more clearly. One might have to vote for the lesser of evils, but could declare more at the ballot box, without delaying elections. -- This address is for information only. I do not claim that these views are those of the Statistics Department or of Purdue University. Herman Rubin, Department of Statistics, Purdue University hrubin@stat.purdue.edu Phone: (765)494-6054 FAX: (765)494-0558 === Subject: Re: Nobel Prize in Econ Aha! I said in a different OT thread about the above, Russell also said in the OTHER OT thread, to what I said, and Hadn't thought of doing that, but I will, just for you! :-) -- Bob. === Subject: Re: Nobel Prize in Econ Have you actually checked the political affiliations of the current five members of the committee? === Subject: Re: How can i show statistical equivalence. first you need to figure out what you are measuring. assuming it is some kind of continuous variable, then look up hypothesis testing or confidence intervals. These are usually called tests for the difference in means (or proportions). Basically, you would need to compute the value of a function of your data and compare it to some probability distribution. unless you provide more specifics, this is how far we can help. === Subject: Re: How can i show statistical equivalence. listed above. I would also search on terms like bioequivalence and equivalence testing. Here's a short note on bioequivalence, for example: http://core.ecu.edu/psyc/wuenschk/StatHelp/Equivalence-EffectSizeCI.htm -- Bruce Weaver bweaver@lakeheadu.ca www.angelfire.com/wv/bwhomedir === Subject: Re: How can i show statistical equivalence. Bruce, Data, have very much stat knowledge. What I am trying to do is showing that 2 test devices function the same, means they come to the same results when testing samples. The samples can not be made to be 100% consistant and testing is destructive so there is some variablity from that. Still i am hopeing to show that with reasonable confidence (whatever that is) the machines come to the same results. Matt === Subject: Re: How can i show statistical equivalence. I just read the page recommended by Bruce. It seems quite simple and straightforward to me, and I'm no expert. What parts of it did you find difficult to understand? Ross-c === Subject: Re: Raing system problem One point I'd like to make is that I believe that any rating system likely to satisfy the requirements of the original poster is going to include some subjective decisions or parameters. E.g. if using cutoffs, what should these cutoffs be? It might be useful if the OP gave some examples of ratings and numbers of votes, and then put them into the rating order that s/he believes they should be in. Once there's a real-world example of what the OP wants, then it *might* be possible to give more specific advice as to how to achieve it. Ross-c === Subject: Re: Find a period in multiple delayed time series Precisely! Plus that the delay between the spikes of different cities, around one year. Martin === Subject: Re: Neet algorithm for clustering huge sparse matrix Well! I am THE Steve Richard but not this one, sorry!. (I should have take a trademark on my name). begin 666 ;-) Wink.png