This can be very annoying as it will often create spikes in your speed that completely mess up any maximum speed or distance statistics for that log file.

For a while, GPSLog Labs has had "position" and "time" filters that let you manually remove these "splats" but it was tedious to get right, so now a simpler "splat filter" has been added that makes the process much less painful.

First, go to the activity you want to filter and click on the Filters link in the side panel:

Then click "Add a filter" and choose "Splat filter":

On the Splat filter page, you need to drag the marker so that it covers the "door" of the building. It doesn't have to cover all of the spikes of the splat, you just need to capture the last good point before you enter the building and the first good point when you get out and start getting a good signal again.

As you can see, the marker is only covering the actual building under the splat, not the whole splat.

The Splat filter will then discard all the points while you were inside the building and interpolate over the gap (which will give you a long period of very slow or 0 speed, restoring the stats of your log back to something sane):

The results are dramatically better, the distance and maximum speed are both now much more accurate and representative.  This log also included the Static Navigation filter as the AMOD AGL 3080 logger was quite bad for logging at walking speeds.

Enjoy!

• There's another new button on the edit Log File tab exposing a feature that was previously a little too well hidden, this time to select which segments should be marked as "junk":
  
  

  That brings up a popup where you can quickly choose which segments are junk (and therefore shouldn't be displayed in the rest of the site) and which have the activity you're interested in:

  

• The colour of the junk segments on that page has been changed subtly from red/orange to a more neutral gray too.

• The GPS Data Filters page has been completely overhauled too so it should hopefully be a bit easier to use and faster to add certain type of filters too.

  Click on the Filters side panel of an activity you want to filter:

  
  
  The GPS Data Filters page now has clearer menu options and a bigger preview maps and graphs to show the results of the filters you have applied:
  
  
  And the individual filter pages now have more space to show their options and view the maps and graphs:
  
• The Time filter now has a preview map to show you the segment that is being discarded.

• Most filters now have a "Copy to tag" menu option that will let you copy the filter and it's parameters to a tag.  This means that once you find a good combination of filters to clean up your activity, you can have them automatically applied to all data you upload from now on tagged with that tag.
  

• The log file chopping logic has been tweaked a little to hopefully avoid creating as many unwanted "unnamed places":
  - Auto created places must now be further away from known places before they'll be created.
  - Auto created places won't be used to chop logs between known places. If you do want the auto created place, after you name it, if you reprocess the log then it will be cut properly.
  - There were also a few bug fixes that should make detection of places more accurate.

• There's better support for editing and filtering of large log files.

• The little milestone markers now have a small arrow attached to indicate the approximate direction of travel at that point:
  

• There's now a filter on the logs tab for a place that lets you narrow down the activity by tag:
  

• The log file editing page has been tweaked to make finding some of the commands easier, the menu now includes the "Add a Place" and "Manually create segments" commands that were previously a little hard for some users to find:
  

  
  It's now possible to change the Route or Tags directly from the edit page for an activity after they have been set:

  

  
  

• The "Select tags" popup now allows quick selection of sets of tags from those used previously on other activity on the same route:
  

  
  

• Speed vs time graphs when you're editing and filtering log files now have a "scale" selection, that will allow you to see detail in log files which was previously hidden by large speed spikes caused by GPS errors:
  
  
  In this example, the actual activity is completely obscured by the 5,000 km/h errors... Selecting a maximum of 50 km/h on the scale means you can see the activity properly and edit it as required:
  

• There's a new "Inverse position filter" available, that is the opposite of the Position filter. i.e. it will discard points outside the given circle.
  

• Florian is doing a great job translating the site into German, danke schön!
  

• The Summary tab on a tag detail page will show a table breaking down the activity for that tag by other tags where appropriate:
  
  

• The Distance vs Time graph on a tag detail page now shows a line through the mouse position that let's you see at a glance which logs are faster, slower or the same speed:
  
  

• Table rows are now highlighted as you hover the mouse over them.

• GPS Data Filter editing has been improved to make it more obvious that the items can be dragged to re-arrange the order that they are applied:
  
  

  -- Tom

On a run the other day, my GPS logger (an AMOD AGL3080) recorded a really noisy signal and the resulting trace was inaccurate for the first 10 minutes. I have no idea why it did this, the second 10 minutes were fine.

The end result was a log which was almost unusable, the distance recorded was twice what it should be and the average speed was way off.

By experimenting with the Speed Filter I was able to find a set of filters that cleaned the log up remarkably, meaning I could include it in my training stats after all.  The following graph and map show the discarded points and the resulting "cleaned" data:

The green points are discarded by the Speed Filter and show how bad the signal was for the first 10 minutes.

The speed is smoothed with a 10 second Speed Median filter too, but the discard filters do the bulk of the work.

I had to experiment with the Speed Filter parameters by varying the cut-off speed until the log's distance was correct (based on previous logs along the same route).  When the cut-off was too high, the resulting distance was still too long, and when it was too low, too many points were discarded and it was too short.  Once the distance was right, the resulting average speeds were reasonable enough for me to be able to use them.  As the table of before and after stats shows, the results were pretty dramatic:

This is the most successful filtering of a log I've seen so far, generally it's removing much smaller amounts of noise, and it's great to know it can be this powerful.  I hope some other GPSLog Labs users can get similar results too as it's very frustrating when a log of your activity doesn't record well and is unusable.

-- Tom

Thankfully, the real power of the filters is that they can be attached to tags, and then when that tag is assigned to a log the filters will be copied across automatically.

This means you can configure a set of filters for all your logs tagged with Run, and you can have a different set of filters for Bike Ride.  For example, you may want to tweak the sample window size on the Median filter as the average speed will vary less when running than when bike riding so the filter can be a bit more aggressive.

The process for adding filters to a tag is the same as for a log, the difference is that you can't see what is being matched by the filter, so I suggest you get the filter working on a log then copy the details across to the tag so it is automatically applied to other logs.

Note that the filters are only automatically copied across to the log when the tag is assigned to the log and you have not modified the log filters.  You can click the Reset button on the log filters page to restore it to the defaults for the tag.

As always, let me know if you have any difficulties or suggestions, and I hope you enjoy this new functionality.

-- Tom

Manual Filter

This filter lets you discard logged data between two times. E.g. if you spent some time inside a building and got spurious signals as a result, you can "zero out" the data between when you entered the building and when you left.

Select the time range by highlighting the portion of the log on the graph, or adjusting the slider, or keying the times into the text boxes:

The data between those two times will be discarded and GPSLog Labs will interpolate the log between the remaining points:

You can use as many manual discard filters as necessary to clean up your log.

Radius Filter

This filter lets you discard logged data in a specified area. E.g. if you have travelled between buildings or through an underpass where you get bad signals you can zero all points within that area. 

Select the area to remove by moving the marker on the map and adjusting the radius:

The data between within the selected area will be discarded and GPSLog Labs will interpolate the log along a straight line between the remaining points.  This won't be perfect, but will be better than the original noisy data.  Like Places, the discard filter is "sticky" so that if the track leaves the radius briefly (less than 60 seconds) it will discard everything between the first point inside the radius and the last.  This should mean you don't have to make the radius too large and will still be able to capture all the bad points.

You can use as many radius discard filters as necessary to clean up your log.

-- Tom

Speed Filter

This filter is the easiest to understand: It will discard any points where the speed is above a certain threshold.

It has two parameters, the cutoff speed (e.g. 20 km/h for running) and the number of points to discard either side of the "bad" point.

After the points have been discarded, the log will be interpolated between the remaining points, hopefully giving you a more accurate trace of your activity.

Static Navigation

The static navigation filter will only be useful if you have a GPS logger with a chipset that includes this "feature" and you can't disable it (e.g. a logger using the SiRF Star III chipset such as the AMOD AGL 3080.)  More details on static navigation can be found by reading http://www.etenblog.com/2007/11/20/static-navigation-explained/ and http://www.gpspassion.com/forumsen/topic.asp?TOPIC_ID=25575.

Basically, in this mode (which is optimized for driving, not walking or running), when your speed is below 5km/h the logger will "clamp" your position on the assumption it's getting some kind of signal "drift".  When it decides you really have moved the logged position will "jump" giving you a log that looks something like the following:

Applying a simple median filter to this data doesn't really help (see the detail below) as there are still lots of samples at the same location:

The static navigation filter will detect these points and remove not just the point, but as many of the "clamped" points as it can, giving a much better result:

(Note that this still needs a median filter to get the best results.)

The next post will describe the "manual" discard filters.

-- Tom

There is a new menu option on the log detail page that will let you edit the filters applied to that log:

The following graph comes from a log recorded while running and shows the kind of noisy data that is often present.  The incorrect speeds mess up the maximum speed, average speed and distance stats and often correspond to a bad trace on the map too.

Discard filters

The first filters you can choose from will discard invalid points from your log and then interpolate the speed and course between the remaining points.

After applying those filters, the speed vs time graph looks a lot better (discarded points are colour-coded to match up with the chosen filters):

Post filters

The second filtering stage applies an average function to the newly cleaned up data.  Smoothing the speed graph will give you a more reliable average and maximum speed statistic and also improve the distance and time split data, but won't clean up the trace on the map or affect the total distance.

The original speeds are shown on the graph below as a light blue line, and after applying a median filter, the thick blue line has been smoothed (no discard filters were applied for this graph). 

This graph shows the final results when both discard and post filters are applied:

Existing logs

The new filters will only take affect for new uploads, existing logs will retain the data from the old filter that GPSLog Labs used to apply until they are reprocessed.

Also, the discarded points from the old filter will no longer be shown on the Speed vs Time graph on the log detail other graphs tab.  This information is only available now for logs using the new filter system from the Filters menu item.

Try it out

I suggest using a simple Speed Median filter initially, as that will clean out a lot of noise, setting the window time parameter to a larger value will smooth the speed more.  I'll be describing the other filters in future posts.

As always, let me know if you have any difficulties or suggestions, and I hope this new feature helps you get more out of the site and your GPS logger!

-- Tom