Area 51 » Science

218

followers

Announcements

I'd like to propose a change to 'Proposed Q&A site for statistics, data analysis and data mining'. It sounds a bit trivial, but the name as it stand implies that only people who do only statistics for a living should use the site. There are people in many other professions who might have a lot to contribute to the site, but be put off if they feel like it is just for people where statistics is their profession rather than part of their profession. – Ian Turner May 27 '10 at 20:57

Statistics is not a subset of mathematics. It uses some mathematics, but so does physics, engineering, etc. Try asking any of these questions on MO and they will (rightly) be closed as off topic. – Rob Hyndman May 27 '10 at 23:40

I've modified the subheading to take account of the comments being made. Let me know if you think this is better (perhaps vote up this comment). I'm still not sure what to do with the major heading. I understand the objections, but haven't come up with anything better yet. Feel free to make some suggestions in the comments. – Rob Hyndman Jun 1 '10 at 3:55

show 14 more announcements

113 Example Questions (12 closed)

active oldest votes

up vote 21 down vote

added by Stu Thompson May 27 '10 at 6:40

"Are statistics 101 questions ok?" They should be. Take a look at SO or the other sites: newbie questions are asked all the time and generally answered with links to docs. If a newbie can't start learning from a stack exchange type site when will they ever contribute? – Good Person May 27 '10 at 10:47

I think a good approach to this kind of question is to let it be asked and answered once, then viciously destroy any duplicates. – Matt Parker May 27 '10 at 19:22

See also area51.stackexchange.com/proposals/33/…. My philosophy is that elementary questions should be allowed on an expert Q&A forum so long as they do not irritate the experts. – Anton Geraschenko May 27 '10 at 20:19

I go along with the Mathoverflow philosophy that this is too basic. – cletus May 27 '10 at 23:28

I agree with Anton that the crucial thing about very basic questions is that they not annoy the experts, I find it hard to imagine though that this question wouldn't annoy experts. – Noah Snyder Jun 1 '10 at 23:26

show 5 more comments

up vote 21 down vote

added by Anton Geraschenko May 27 '10 at 20:35

	Or, are Tufte-style presentation of data questions appropriate? Or, does looking at raw data and identifying interesting features by eye count as "statistical analysis"? – Anton Geraschenko May 27 '10 at 20:38
	I definitely think we should attract the sort of question where people look at data sets by eye. With people looking at all sorts of data sets, people might see patterns or have other observations from a completely different field to the field of the person asking questions. This would come in the art rather than the science side of statistics. – Ian Turner May 27 '10 at 21:01
	I think this makes a good contrast to this question: area51.stackexchange.com/proposals/33/… – Matt Parker May 27 '10 at 21:01

up vote 20 down vote

added by Rob Hyndman May 25 '10 at 2:15

up vote 20 down vote

added by chollida May 27 '10 at 18:49

up vote 20 down vote

added by Scott Wales Jun 2 '10 at 1:43

up vote 13 down vote

added by Rob Hyndman May 25 '10 at 5:48

up vote 11 down vote

added by Rob Hyndman May 25 '10 at 6:21

	Isn't this philosophy ? on-topic, but I wouldn't use this as a great on-topic example. – robin girard Jun 15 '10 at 6:53
	I think that this is a good statistical question; it has a correct answer, and one which should be well understood. – Shane Jun 16 '10 at 17:41

up vote 11 down vote

added by Rob Hyndman May 27 '10 at 4:34

	Similar to a previous question that I thought was too vague. With a little context, I think it is acceptable. A decent answer would mention Markov models, SVM classifiers and other possibilities, with some brief comments on their pros/cons. – Rob Hyndman May 27 '10 at 4:36
	This probably exposes a grey area, how far into specific fields should this Stack Exchange go compared with Stack Exchanges specific to Meterology or Climate Modelling. I would argue this is better placed on a specific Stack Exchange as a similarly posed question likely to have an earthquake tomorrow based on the last 100 years of earthquake data should definitely not sit on this Stack Exchange. – Ian Turner May 27 '10 at 13:29
	I think that's a valid point, Splash, but there's probably room for one such question - if someone showed up asking about earthquakes, we could refer them to the rain question. If they needed more specifics than that, then they can go to the geology SE. – Matt Parker May 27 '10 at 20:56
	of course this is a predictive model, but in case of the methods, you can use decision threes, associated rules and so on, depending on your dataset. you have to explain more about your dataset information fields – maahtu Jun 6 '10 at 13:37
	I think this is a great on topic example. It shows that we do not restric ourself to theory or toy examples. Mentionning other fields (like meteorology) or Stack exchange will be in a good answer but this good answer will also contain statistic (that may use as input meteorological forecast). This could lead to a communication between two stack exchange (through a link between two questions) and people should see it is encouraged (with a great on-topic example). – robin girard Jun 16 '10 at 6:14

up vote 8 down vote

added by Rob Hyndman May 27 '10 at 11:58

I'm not a huge fan of "what is the best way" kinds of questions (at least in a general sense), because they can be subjective; the best way can depend on the context of the question. – Shane Jun 7 '10 at 13:44

up vote 7 down vote

added by Harvey Motulsky May 25 '10 at 11:31

up vote 6 down vote

added by Ian Turner May 27 '10 at 12:03

There are actually eight different ways to compute a percentile (all are the same for 50th percentile, but can differ otherwise). So this question has some meat, at least if it asks about algorithms rather than definitions – Harvey Motulsky May 27 '10 at 13:42

up vote 6 down vote

added by Michel de Ruiter Jun 15 '10 at 20:17

up vote 5 down vote

added by Mike Grady May 27 '10 at 4:59

use "clValid" package. given the dataset and the number of clusters, the function can measure cluster validity using three measures: silhouette, connectivity and ... umm I've fogot the third. – maahtu Jun 6 '10 at 13:33

up vote 5 down vote

added by csgillespie Jun 2 '10 at 15:05

By offering them beer? – bmargulies Jun 5 '10 at 19:03

up vote 4 down vote

added by Harvey Motulsky May 25 '10 at 11:36

I think this is way too vague, so is not appropriate. – Harvey Motulsky May 25 '10 at 11:36

It would help drawing in potential users that aren't at the level of statistical analysis as you guys are, but at least are interested in the topic. And it's likely to pop up in people's Google results, which never hurts... – Ivo Flipse May 26 '10 at 21:35

In keeping with the spirit of Stack Overflow, this site should be the canonical source of statistical analysis. The answers to this question could bring a more complete understanding of p-values even for professionals who understand the use of p-values in their every day work, but don't also have a full theoretical background. This question belongs on any statistics site that is to be the canonical source of knowledge for that topic. – Jay Stevens May 27 '10 at 10:34

up vote 4 down vote

added by Anton Geraschenko May 27 '10 at 7:31

up vote 4 down vote

added by Andrew Song May 27 '10 at 21:30

up vote 4 down vote

added by Good Person May 29 '10 at 17:37

up vote 4 down vote

added by Simon Nickerson Jun 3 '10 at 19:05

up vote 3 down vote

added by Rob Hyndman May 25 '10 at 2:22

Bootstrapping time series is tricky as the standard bootstrap will destroy the serial correlation structure. Block bootstrap is one approach but there are others. A good answer would discuss the pros/cons of each. – Rob Hyndman May 25 '10 at 12:42

not to be confused with questions about bootystrap... which I am quite sure would be off topic – JD Long Jun 2 '10 at 19:59

up vote 3 down vote

added by Rob Hyndman May 25 '10 at 5:49

up vote 3 down vote

added by Rob Hyndman May 25 '10 at 6:21

up vote 2 down vote

added by Rob Hyndman May 25 '10 at 2:00

up vote 2 down vote

added by Rob Hyndman May 25 '10 at 6:22

	Best for this sort of thing to go to MathOverflow – Rob Hyndman May 25 '10 at 6:22
	Why MO? I feel that the Statistics site is a better fit for this kind of question, – Anon Jun 2 '10 at 3:09
	Questions about theory that justifies/explains statistic should no be limited to MathOverflow. Also I vote for it as a great on topic example. – robin girard Jun 16 '10 at 6:03

up vote 2 down vote

added by Harvey Motulsky May 25 '10 at 11:46

This is about data analysis, but would not be appropriate here. . The "hide the decline" phrase came from a ClimateGate email explaining how they spliced together two data sets (and tossed out some data) to create a graph of mean temperature over a long period of time. This site should be about general ideas and principles of data analysis methods, and not specifics of one paper. – Harvey Motulsky May 25 '10 at 11:51

Sounds like the answer would belong more on a blog instead. – Mike Mazur May 27 '10 at 4:47

Off-topic because it's too vague but I do think that responding to the specifics of "one paper" can be totally legitimate, especially if the paper discussed a relatively new concept, method or approach – Jay Stevens May 27 '10 at 10:37

I think this one is legitimate, if possibly not good for the site. – Charles Stewart Jun 7 '10 at 12:38

up vote 2 down vote

added by Harvey Motulsky May 25 '10 at 12:03

The traditional method is to remove that pair of values completely. The method of Pratt includes the data in the analysis. What are the pros and cons of each approach? – Harvey Motulsky May 25 '10 at 12:04

up vote 2 down vote

added by Joel Spolsky♦ May 25 '10 at 14:53

closed as off topic by Peter K., Matt Parker, Mike Grady, Ian Turner, Charles Stewart Jun 7 '10 at 12:35

This question does not relate to the topic of the proposal.
It was closed as part of an automated migration of off-topic to close votes on September 29, 2011.

Could somebody explain why this is off-topic? The only argument I can think of is "this is a Q&A site for statistical *analysis*", but the people interested in doing the analysis are exactly the same people who are interested in getting ahold of the data to analyze. – Anton Geraschenko May 27 '10 at 7:12

It is not intended to be a site that provides data, but one which is about analysing data. Of course, there will be times when people might ask such questions on the site, but I don't think we want to encourage it at this stage. – Rob Hyndman May 27 '10 at 7:37

I agree with Anton on this. I don't believe that it isn't data provision if you just direct individuals to locations where they can find the data. I believe that data acquisition is the first step for data analysis. – Waleed Al-Balooshi May 27 '10 at 12:23

I would hope that sources of data for statistical mining would be considered on topic for this site – chollida May 27 '10 at 18:46

But if you add sources of data, that's endless. You could try to data mine any kind of data. Any collection of any kind of numeric, textual, etc. That's a library cataloging function, not a "how to" question. Now if you want to create a "catalog of data site", fine, but make a specific site for it. Does this split by government/public data, literature collections, digital libraries, ...? – Mike Grady May 31 '10 at 0:27

up vote 2 down vote

added by Juan A. Navarro May 27 '10 at 5:40

I think this is legitimate. The real turning point in my statistics education was learning how to create random data for comparison with real data - it's really useful to be able to say, "This is what centrality measure X looks like in 1000 random graphs; this is what it looks like in the empirical graph." I hope tool selection advice will be legitimate here. – Matt Parker May 27 '10 at 21:08

up vote 2 down vote

added by Rob Hyndman May 27 '10 at 12:23

I think this would be valid if it were asking for advice in weighing the pros and cons of different packages for this in R, but not if it's asking specifically for how to code the model. – Matt Parker May 27 '10 at 19:09

Not sure. R can have a fair learning curve at times, and I find it hard to collect good (or at least consistent) examples of how to do this kind of thing. – Menno May 29 '10 at 11:21

@Matt: I agree. @Menno: Perhaps, we can have a FAQ with some resources for the major stats packages. – Anon Jun 2 '10 at 3:11

Menno - I agree, but StackOverflow has a robust R community to tackle the particulars of programming. That line is inevitably going to get smudged (especially in mixed-effects modeling, I think), but I'd rather see this stats site focus on the stats and leave the syntax picking to SO. – Matt Parker Jun 2 '10 at 6:00

up vote 2 down vote

added by chollida May 28 '10 at 16:02

If you asked this on SO I suspect you'd be told to go ask a statistician :) – therefromhere May 28 '10 at 17:43

Testing RNGs is actually really hard, see e.g. the DieHarder test battery and its RDieHarder binding. BTW, it's spelled Mersenne Twister. – Dirk Eddelbuettel May 29 '10 at 2:42

up vote 2 down vote

added by therefromhere May 28 '10 at 16:18

up vote 2 down vote

added by Joe Mako May 29 '10 at 20:20

An example would be if you have the average occurrence of something for each state, why does an average of an average not make sense to get the country average. Or is there any value in looking at the median of a set of averages? – Joe Mako May 29 '10 at 20:22

up vote 2 down vote

added by Mehper C. Palavuzlar Jun 3 '10 at 10:32

you can search this term in r-project.org – maahtu Jun 6 '10 at 13:25

up vote 2 down vote

added by Mehper C. Palavuzlar Jun 3 '10 at 10:41

up vote 2 down vote

added by Menno Jun 5 '10 at 7:41

up vote 1 down vote

added by Rob Hyndman May 25 '10 at 2:09

up vote 1 down vote

added by Rob Hyndman May 25 '10 at 2:11

Questions like this often end up on StackOverflow, but I think this site should cover tool-based questions such as this. – Rob Hyndman May 25 '10 at 12:41

It feels rather weird allowing question on R packages, but refusing R code/implementations. As with Math Overflow, it might not be so bad to allow code snippets or these grey area topics. Especially when SO might not want this question – Ivo Flipse May 26 '10 at 21:31

I think this site has got to allow some implementation/tools discussion/answers in order to have practical relevance. Unless you're going to calculate it by hand, it doesn't do much good to know how a logistic regression works without having tools to implement. However, the trick will be in only allowing tools discussions that are on-topic. For example, how to use SAS to connect to a Sybase data warehouse is likely off-topic even if you want to run a logistic regression on the data you get. – Jay Stevens May 27 '10 at 10:29

I think there is a difference between asking for code vs asking for existence of a package. – Anon Jun 2 '10 at 3:01

(Is there am R package for SVMs? It would help me a lot!) – Larry OBrien Jun 4 '10 at 17:25

up vote 1 down vote

added by Joel Spolsky♦ May 25 '10 at 3:16

closed as off topic by Rob Hyndman, Mike Grady, Jeromy Anglim, Yacoby, dswan Jun 6 '10 at 10:47

This question does not relate to the topic of the proposal.
It was closed as part of an automated migration of off-topic to close votes on September 29, 2011.

too easy, I assume, and sounds like homework. – Joel Spolsky♦ May 25 '10 at 3:17

I'd rather not see any questions that ask others to do computations for the asker. How to compute, yes; result of computation, no. – Matt Parker May 27 '10 at 18:26

I agree with Matt! Concepts should be clarified, method on how solving it too. But no answers. No homework :P – George May 28 '10 at 4:47

No answers for simple calculations such as this - I agree with that. However, what about checking with stats professionals about complex answers or confirming the implementation of a stats routine in a new language / program? – Patrick Welch Jun 4 '10 at 20:38

Patrick - I'm all for that, and I think questions like that will set themselves apart from this one because of the complexity and because the asker will clearly have put some substantial effort into their own answer. – Matt Parker Jun 8 '10 at 1:34

up vote 1 down vote

added by Rob Hyndman May 25 '10 at 5:52

closed as off topic by Matt Parker, Mike Grady, dswan, Karsten W., Joe Mako Jun 7 '10 at 4:56

This question does not relate to the topic of the proposal.
It was closed as part of an automated migration of off-topic to close votes on September 29, 2011.

Best to leave questions about R code on StackOverflow. – Rob Hyndman May 25 '10 at 5:52

@Rob: I disagree. Just because a question is on-topic on another site doesn't mean that it's off-topic on every other site. If there were a SE site dedicated to statistics, I think it would be pretty artificial to exclude R questions. If a professional statistician would ask this question (in her capacity as a statistician), why isn't it on-topic? – Anton Geraschenko May 27 '10 at 6:42

@Anton. The trouble is, the site could quickly degenerate into general tech support for every tool being used. There are already several places to go for R tech support including SO, R-help mailing list, etc. Similarly for SAS, SPSS, etc. There is nowhere to go for good advice about statistical methodology, data analysis, etc. – Rob Hyndman May 27 '10 at 7:41

Another point to consider here is that this question is not about statistics at all, it is simply about R syntax. A question about how to do something statistical in R (e.g., how to fit a mixed-effects model) would be appropriate. To test the waters, I'll add such a question and see how it fares. – Rob Hyndman May 27 '10 at 12:23

@Rob: Thanks for clarifying. I agree with you now. – Anton Geraschenko May 27 '10 at 20:10

show 2 more comments

up vote 1 down vote

added by Anton Geraschenko May 27 '10 at 7:26