gnuplot removing outliers when plotting a data file

Question

I need to plot a data file of 2 colums using gnuplot, scatter plot is what I need I think. My understanding of gnuplot goes as far as :

plot "first_click" using 2:1

3 lines from head and tail of my data looks as follows:

1 612856
3 3840538
5 5240597
.
.
.
139845 1
141101 1
141584 1

I am expecting my scatter plot to show a logarithmic trend, however my data (as most data) has tons of outliers . So I need to do one of two things:

1. Automatically "zoom" to where most of the data is.
2. Automatically prune outliers.
3. Provide a predicate for each of the columns to manually prune the data, and perhaps predicates that can take both columns of an entry in scope --e.g., !column1 > x && ! column2 == 1

Precision is not a concern.

At this stage I prefer 1 and 2, but I'd like to see if option 3 is possible as well since I am a programmer and not a statistician.

Answer 1

gnuplot should automatically zoom to fit the data plotted (if not, you can use reset yrange, xrange to auto-zoom again). If the outliers are pruned prior to plotting then your first requirement would already be met.

Number two and three could be achieved by modifying your plot command as follows:

plot "first_click" using ($2 != 1 ? $2 : 1/0):($1 < x ? $1 : 1/0)

Would plot only values for which the second column is not equal to 1 and the first column is less than x. Where x is the value at which you want to start pruning outliers. 1/0 is a way of telling gnuplot the point is invalid and it won't be plotted.

Answer 2

You could also try

plot "first_click" using 2:1 smooth bezier with lines

This has the side effect of not showing most outliers.