Area 51 » Science

218

followers

Announcements

I'd like to propose a change to 'Proposed Q&A site for statistics, data analysis and data mining'. It sounds a bit trivial, but the name as it stand implies that only people who do only statistics for a living should use the site. There are people in many other professions who might have a lot to contribute to the site, but be put off if they feel like it is just for people where statistics is their profession rather than part of their profession. – Ian Turner May 27 '10 at 20:57

Statistics is not a subset of mathematics. It uses some mathematics, but so does physics, engineering, etc. Try asking any of these questions on MO and they will (rightly) be closed as off topic. – Rob Hyndman May 27 '10 at 23:40

I've modified the subheading to take account of the comments being made. Let me know if you think this is better (perhaps vote up this comment). I'm still not sure what to do with the major heading. I understand the objections, but haven't come up with anything better yet. Feel free to make some suggestions in the comments. – Rob Hyndman Jun 1 '10 at 3:55

show 14 more announcements

113 Example Questions (12 closed)

active oldest votes

up vote 21 down vote

added by Stu Thompson May 27 '10 at 6:40

"Are statistics 101 questions ok?" They should be. Take a look at SO or the other sites: newbie questions are asked all the time and generally answered with links to docs. If a newbie can't start learning from a stack exchange type site when will they ever contribute? – Good Person May 27 '10 at 10:47

I think a good approach to this kind of question is to let it be asked and answered once, then viciously destroy any duplicates. – Matt Parker May 27 '10 at 19:22

See also area51.stackexchange.com/proposals/33/…. My philosophy is that elementary questions should be allowed on an expert Q&A forum so long as they do not irritate the experts. – Anton Geraschenko May 27 '10 at 20:19

I go along with the Mathoverflow philosophy that this is too basic. – cletus May 27 '10 at 23:28

I agree with Anton that the crucial thing about very basic questions is that they not annoy the experts, I find it hard to imagine though that this question wouldn't annoy experts. – Noah Snyder Jun 1 '10 at 23:26

show 5 more comments

up vote 21 down vote

added by Anton Geraschenko May 27 '10 at 20:35

	Or, are Tufte-style presentation of data questions appropriate? Or, does looking at raw data and identifying interesting features by eye count as "statistical analysis"? – Anton Geraschenko May 27 '10 at 20:38
	I definitely think we should attract the sort of question where people look at data sets by eye. With people looking at all sorts of data sets, people might see patterns or have other observations from a completely different field to the field of the person asking questions. This would come in the art rather than the science side of statistics. – Ian Turner May 27 '10 at 21:01
	I think this makes a good contrast to this question: area51.stackexchange.com/proposals/33/… – Matt Parker May 27 '10 at 21:01

up vote 20 down vote

added by Rob Hyndman May 25 '10 at 2:15

up vote 20 down vote

added by chollida May 27 '10 at 18:49

up vote 20 down vote

added by Scott Wales Jun 2 '10 at 1:43

up vote 13 down vote

added by Rob Hyndman May 25 '10 at 5:48

up vote 11 down vote

added by Rob Hyndman May 25 '10 at 6:21

	Isn't this philosophy ? on-topic, but I wouldn't use this as a great on-topic example. – robin girard Jun 15 '10 at 6:53
	I think that this is a good statistical question; it has a correct answer, and one which should be well understood. – Shane Jun 16 '10 at 17:41

up vote 11 down vote

added by Rob Hyndman May 27 '10 at 4:34

	Similar to a previous question that I thought was too vague. With a little context, I think it is acceptable. A decent answer would mention Markov models, SVM classifiers and other possibilities, with some brief comments on their pros/cons. – Rob Hyndman May 27 '10 at 4:36
	This probably exposes a grey area, how far into specific fields should this Stack Exchange go compared with Stack Exchanges specific to Meterology or Climate Modelling. I would argue this is better placed on a specific Stack Exchange as a similarly posed question likely to have an earthquake tomorrow based on the last 100 years of earthquake data should definitely not sit on this Stack Exchange. – Ian Turner May 27 '10 at 13:29
	I think that's a valid point, Splash, but there's probably room for one such question - if someone showed up asking about earthquakes, we could refer them to the rain question. If they needed more specifics than that, then they can go to the geology SE. – Matt Parker May 27 '10 at 20:56
	of course this is a predictive model, but in case of the methods, you can use decision threes, associated rules and so on, depending on your dataset. you have to explain more about your dataset information fields – maahtu Jun 6 '10 at 13:37
	I think this is a great on topic example. It shows that we do not restric ourself to theory or toy examples. Mentionning other fields (like meteorology) or Stack exchange will be in a good answer but this good answer will also contain statistic (that may use as input meteorological forecast). This could lead to a communication between two stack exchange (through a link between two questions) and people should see it is encouraged (with a great on-topic example). – robin girard Jun 16 '10 at 6:14

up vote 8 down vote

added by Rob Hyndman May 27 '10 at 11:58

I'm not a huge fan of "what is the best way" kinds of questions (at least in a general sense), because they can be subjective; the best way can depend on the context of the question. – Shane Jun 7 '10 at 13:44

up vote 7 down vote

added by Harvey Motulsky May 25 '10 at 11:31

up vote 6 down vote

added by Ian Turner May 27 '10 at 12:03

There are actually eight different ways to compute a percentile (all are the same for 50th percentile, but can differ otherwise). So this question has some meat, at least if it asks about algorithms rather than definitions – Harvey Motulsky May 27 '10 at 13:42

up vote 6 down vote

added by Michel de Ruiter Jun 15 '10 at 20:17

up vote 5 down vote

added by Mike Grady May 27 '10 at 4:59

use "clValid" package. given the dataset and the number of clusters, the function can measure cluster validity using three measures: silhouette, connectivity and ... umm I've fogot the third. – maahtu Jun 6 '10 at 13:33

up vote 5 down vote

added by csgillespie Jun 2 '10 at 15:05

By offering them beer? – bmargulies Jun 5 '10 at 19:03

up vote 4 down vote

added by Harvey Motulsky May 25 '10 at 11:36

I think this is way too vague, so is not appropriate. – Harvey Motulsky May 25 '10 at 11:36

It would help drawing in potential users that aren't at the level of statistical analysis as you guys are, but at least are interested in the topic. And it's likely to pop up in people's Google results, which never hurts... – Ivo Flipse May 26 '10 at 21:35

In keeping with the spirit of Stack Overflow, this site should be the canonical source of statistical analysis. The answers to this question could bring a more complete understanding of p-values even for professionals who understand the use of p-values in their every day work, but don't also have a full theoretical background. This question belongs on any statistics site that is to be the canonical source of knowledge for that topic. – Jay Stevens May 27 '10 at 10:34

up vote 4 down vote

added by Anton Geraschenko May 27 '10 at 7:31

up vote 4 down vote

added by Andrew Song May 27 '10 at 21:30

up vote 4 down vote

added by Good Person May 29 '10 at 17:37

up vote 4 down vote

added by Simon Nickerson Jun 3 '10 at 19:05

up vote 3 down vote

added by Rob Hyndman May 25 '10 at 2:22

Bootstrapping time series is tricky as the standard bootstrap will destroy the serial correlation structure. Block bootstrap is one approach but there are others. A good answer would discuss the pros/cons of each. – Rob Hyndman May 25 '10 at 12:42

not to be confused with questions about bootystrap... which I am quite sure would be off topic – JD Long Jun 2 '10 at 19:59

up vote 3 down vote

added by Rob Hyndman May 25 '10 at 5:49

up vote 3 down vote

added by Rob Hyndman May 25 '10 at 6:21

up vote 2 down vote

added by Rob Hyndman May 25 '10 at 2:00

up vote 2 down vote

added by Rob Hyndman May 25 '10 at 6:22

	Best for this sort of thing to go to MathOverflow – Rob Hyndman May 25 '10 at 6:22
	Why MO? I feel that the Statistics site is a better fit for this kind of question, – Anon Jun 2 '10 at 3:09
	Questions about theory that justifies/explains statistic should no be limited to MathOverflow. Also I vote for it as a great on topic example. – robin girard Jun 16 '10 at 6:03

up vote 2 down vote

added by Harvey Motulsky May 25 '10 at 11:46

This is about data analysis, but would not be appropriate here. . The "hide the decline" phrase came from a ClimateGate email explaining how they spliced together two data sets (and tossed out some data) to create a graph of mean temperature over a long period of time. This site should be about general ideas and principles of data analysis methods, and not specifics of one paper. – Harvey Motulsky May 25 '10 at 11:51

Sounds like the answer would belong more on a blog instead. – Mike Mazur May 27 '10 at 4:47

Off-topic because it's too vague but I do think that responding to the specifics of "one paper" can be totally legitimate, especially if the paper discussed a relatively new concept, method or approach – Jay Stevens May 27 '10 at 10:37

I think this one is legitimate, if possibly not good for the site. – Charles Stewart Jun 7 '10 at 12:38

up vote 2 down vote

added by Harvey Motulsky May 25 '10 at 12:03

The traditional method is to remove that pair of values completely. The method of Pratt includes the data in the analysis. What are the pros and cons of each approach? – Harvey Motulsky May 25 '10 at 12:04

up vote 2 down vote

added by Joel Spolsky♦ May 25 '10 at 14:53

closed as off topic by Peter K., Matt Parker, Mike Grady, Ian Turner, Charles Stewart Jun 7 '10 at 12:35

This question does not relate to the topic of the proposal.
It was closed as part of an automated migration of off-topic to close votes on September 29, 2011.

Could somebody explain why this is off-topic? The only argument I can think of is "this is a Q&A site for statistical *analysis*", but the people interested in doing the analysis are exactly the same people who are interested in getting ahold of the data to analyze. – Anton Geraschenko May 27 '10 at 7:12

It is not intended to be a site that provides data, but one which is about analysing data. Of course, there will be times when people might ask such questions on the site, but I don't think we want to encourage it at this stage. – Rob Hyndman May 27 '10 at 7:37

I agree with Anton on this. I don't believe that it isn't data provision if you just direct individuals to locations where they can find the data. I believe that data acquisition is the first step for data analysis. – Waleed Al-Balooshi May 27 '10 at 12:23

I would hope that sources of data for statistical mining would be considered on topic for this site – chollida May 27 '10 at 18:46

But if you add sources of data, that's endless. You could try to data mine any kind of data. Any collection of any kind of numeric, textual, etc. That's a library cataloging function, not a "how to" question. Now if you want to create a "catalog of data site", fine, but make a specific site for it. Does this split by government/public data, literature collections, digital libraries, ...? – Mike Grady May 31 '10 at 0:27

up vote 2 down vote

added by Juan Antonio May 27 '10 at 5:40

I think this is legitimate. The real turning point in my statistics education was learning how to create random data for comparison with real data - it's really useful to be able to say, "This is what centrality measure X looks like in 1000 random graphs; this is what it looks like in the empirical graph." I hope tool selection advice will be legitimate here. – Matt Parker May 27 '10 at 21:08

up vote 2 down vote

added by Rob Hyndman May 27 '10 at 12:23

I think this would be valid if it were asking for advice in weighing the pros and cons of different packages for this in R, but not if it's asking specifically for how to code the model. – Matt Parker May 27 '10 at 19:09

Not sure. R can have a fair learning curve at times, and I find it hard to collect good (or at least consistent) examples of how to do this kind of thing. – Menno May 29 '10 at 11:21

@Matt: I agree. @Menno: Perhaps, we can have a FAQ with some resources for the major stats packages. – Anon Jun 2 '10 at 3:11

Menno - I agree, but StackOverflow has a robust R community to tackle the particulars of programming. That line is inevitably going to get smudged (especially in mixed-effects modeling, I think), but I'd rather see this stats site focus on the stats and leave the syntax picking to SO. – Matt Parker Jun 2 '10 at 6:00

up vote 2 down vote

added by chollida May 28 '10 at 16:02

If you asked this on SO I suspect you'd be told to go ask a statistician :) – therefromhere May 28 '10 at 17:43

Testing RNGs is actually really hard, see e.g. the DieHarder test battery and its RDieHarder binding. BTW, it's spelled Mersenne Twister. – Dirk Eddelbuettel May 29 '10 at 2:42