Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.java.programmer > #37699 > unrolled thread

How to identify outliers

Back to article view | Back to comp.lang.java.programmer

Contents

  How to identify outliers Daniele Futtorovic <da.futt.news@laposte.net> - 2017-11-28 15:40 +0100
    Re: How to identify outliers Eric Sosman <esosman@comcast-dot-net.invalid> - 2017-11-28 10:13 -0500
      Re: How to identify outliers Daniele Futtorovic <da.futt.news@laposte.net> - 2017-11-28 17:33 +0100
    Re: How to identify outliers Eric Douglas <e.d.programmer@gmail.com> - 2017-11-28 07:28 -0800
      Re: How to identify outliers Daniele Futtorovic <da.futt.news@laposte.net> - 2017-11-28 17:35 +0100
    Re: How to identify outliers Robert Klemme <shortcutter@googlemail.com> - 2017-11-30 18:38 +0100

#37699 — How to identify outliers

Hi folks,

I got a statistical problem, and due to a youth math trauma, I'd like a
bit of help to solve it.

I have a data series. Integer values, but that should be beside the
point. I want to identify and eliminate "outliers" from this series.
"Outliers" being values that are "too big" or "too small", in short,
that are not congruent with the rest of the series. I am well aware that
saying "outlier" begs the question, hence the quotes, but I can
fine-tune the degree of outlying for my purposes later on. For the time
being, I wonder if someone could outline some basic approaches I could
take in the matter.

-- 
DF.

[toc] | [next] | [standalone]

#37700

On 11/28/2017 9:40 AM, Daniele Futtorovic wrote:
> Hi folks,
> 
> I got a statistical problem, and due to a youth math trauma, I'd like a
> bit of help to solve it.
> 
> I have a data series. Integer values, but that should be beside the
> point. I want to identify and eliminate "outliers" from this series.
> "Outliers" being values that are "too big" or "too small", in short,
> that are not congruent with the rest of the series. I am well aware that
> saying "outlier" begs the question, hence the quotes, but I can
> fine-tune the degree of outlying for my purposes later on. For the time
> being, I wonder if someone could outline some basic approaches I could
> take in the matter.


http://newton.newhaven.edu/sross/piercescriterion.pdf

     If the table at the end of this (short) paper isn't suited to
your needs, I can also offer source code (in C) to adapted from
https://groups.google.com/d/topic/sci.math/Kypai3jqss8/discussion
to calculate the underlying distribution.  Drop me a line if you'd
like it.

-- 
esosman@comcast-dot-net.invalid
Eleven hundred forty-nine days to go.

[toc] | [prev] | [next] | [standalone]

#37702

On 2017-11-28 16:13, Eric Sosman wrote:
> On 11/28/2017 9:40 AM, Daniele Futtorovic wrote:
>> Hi folks,
>>
>> I got a statistical problem, and due to a youth math trauma, I'd like a
>> bit of help to solve it.
>>
>> I have a data series. Integer values, but that should be beside the
>> point. I want to identify and eliminate "outliers" from this series.
>> "Outliers" being values that are "too big" or "too small", in short,
>> that are not congruent with the rest of the series. I am well aware that
>> saying "outlier" begs the question, hence the quotes, but I can
>> fine-tune the degree of outlying for my purposes later on. For the time
>> being, I wonder if someone could outline some basic approaches I could
>> take in the matter.
> 
> 
> http://newton.newhaven.edu/sross/piercescriterion.pdf
> 
>     If the table at the end of this (short) paper isn't suited to
> your needs, I can also offer source code (in C) to adapted from
> https://groups.google.com/d/topic/sci.math/Kypai3jqss8/discussion
> to calculate the underlying distribution.  Drop me a line if you'd
> like it.
> 

Spiffy. Exactly what I was looking for. I'll go over it and get back to
you if needed and you don't mind.

Thanks a ton.

-- 
DF.

[toc] | [prev] | [next] | [standalone]

#37701

On Tuesday, November 28, 2017 at 9:40:41 AM UTC-5, Daniele Futtorovic wrote:
> Hi folks,
> 
> I got a statistical problem, and due to a youth math trauma, I'd like a
> bit of help to solve it.
> 
> I have a data series. Integer values, but that should be beside the
> point. I want to identify and eliminate "outliers" from this series.
> "Outliers" being values that are "too big" or "too small", in short,
> that are not congruent with the rest of the series. I am well aware that
> saying "outlier" begs the question, hence the quotes, but I can
> fine-tune the degree of outlying for my purposes later on. For the time
> being, I wonder if someone could outline some basic approaches I could
> take in the matter.
> 
> -- 
> DF.

Were you looking for help figuring out what the outliers should be, or for Java code to remove values from a list which exceed a minimum and maximum?

[toc] | [prev] | [next] | [standalone]

#37703

On 2017-11-28 16:28, Eric Douglas wrote:
> On Tuesday, November 28, 2017 at 9:40:41 AM UTC-5, Daniele Futtorovic wrote:
>> Hi folks,
>>
>> I got a statistical problem, and due to a youth math trauma, I'd like a
>> bit of help to solve it.
>>
>> I have a data series. Integer values, but that should be beside the
>> point. I want to identify and eliminate "outliers" from this series.
>> "Outliers" being values that are "too big" or "too small", in short,
>> that are not congruent with the rest of the series. I am well aware that
>> saying "outlier" begs the question, hence the quotes, but I can
>> fine-tune the degree of outlying for my purposes later on. For the time
>> being, I wonder if someone could outline some basic approaches I could
>> take in the matter.
> 
> Were you looking for help figuring out what the outliers should be, or for Java code to remove values from a list which exceed a minimum and maximum?

:-)

No, I was looking for the math. Code is (comparatively) easy.

-- 
DF.

[toc] | [prev] | [next] | [standalone]

#37706

On 28.11.2017 15:40, Daniele Futtorovic wrote:

> I got a statistical problem, and due to a youth math trauma, I'd like a
> bit of help to solve it.

:-)

> I have a data series. Integer values, but that should be beside the
> point. I want to identify and eliminate "outliers" from this series.
> "Outliers" being values that are "too big" or "too small", in short,
> that are not congruent with the rest of the series. I am well aware that
> saying "outlier" begs the question, hence the quotes, but I can
> fine-tune the degree of outlying for my purposes later on. For the time
> being, I wonder if someone could outline some basic approaches I could
> take in the matter.

Funny: just today I ran across 
https://www.reddit.com/r/java/comments/7gkry0/statistically_rigorous_java_performance_evaluation/ 
and eventually found https://en.wikipedia.org/wiki/Truncated_mean which 
is a fairly simple and straightforward method.  Depending on what you 
want to do you could also work with median or percentiles which are 
usually less affected by outliers than the median.

Kind regards

	robert

-- 
remember.guy do |as, often| as.you_can - without end
http://blog.rubybestpractices.com/

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.java.programmer

csiph-web