Filters Overview
SUMMARY
- Filters let you edit or remove data before it is written to the report data files
- Filters are linked to Profiles and are applied during processing
- One Filter can be used by many Profiles, one Profile can use many Filters
- Filters run in a specific order, and can be applied at 3 different Levels during processing
- Changes made to a Filter are not applied to existing report data
- You must "update & reprocess" for new Filter settings to be applied to report data
- Angelfish includes a handful of predefined Filters - you can use / edit / delete them as needed
- Filters are most effective when Regular Expressions (RegEx) are used in the Match Pattern
Filter Levels
The Filter Level determine when the Filter will be run, and the Field Names that are available in the Filter. As an example, Field Names that correspond to Visit-level Dimensions are only available to Session-Level Filters. (e.g. Search Usage or External Source)
The three Filter Levels are:
RAW
- Applied to each hit after it's ingested but before any Page or Visit logic is applied
- The Raw Level is the only Filter Level that affects data in IT Reports
PAGE
- Applied to hits evaluated as Pages, after all fields in the hit are parsed
- Best level for filtering Page and Search Dimensions
SESSION
- Applied at the end of processing, after Pages have been identified
- Best level for filtering Visit Dimensions
Filter Types
Angelfish has 5 Filter Types. Each Filter Type can be used at each Filter Level.
Each Filter is configured with at least one Field Name and Match Pattern: the Field Name is used to evaluate the Match Pattern. If a match is found, the Filter is applied.
If multiple Field Names are used in a Filter, the Filter uses AND logic for each Field Name & Match Pattern. This means all Conditions must be met for the Filter logic to be applied.
The five Filter Types are:
INCLUDE
- Include Filters use the Match Pattern to identify data that is allowed to remain in processing.
- Another way to look at Include Filters is they exclude anything that doesn't match the Match Pattern.
EXCLUDE
- Exclude Filters use the Match Pattern to identify data to remove from processing.
ADVANCED
- Advanced Filters let you overwrite, combine, or remove data if specific conditions are met.
- RegEx is frequently used in Advanced Filters.
- Advanced Filters have two sections: Conditions and Output.
- Conditions: contains the Match Patterns that need to be met for the Filter to be applied.
- Output: contains fields to be updated when the Conditions are met
SEARCH & REPLACE
- Search & Replace Filters perform a simple overwrite: when a string is matched, it's replaced with a specific value.
- The "Remove Duplicate Slashes" Filter is an example of this.
- When a string of 2 or more slashes are matched, the string is replaced by a single slash.
LOOKUP TABLE
- A Lookup Table (LT) Filter provide a way to use multiple fields with multiple values in a Filter.
- LT Filters use a List to populate the Match Pattern
- LT Filters can be applied 3 ways: Overwrite, Include, Exclude.
Overwrite:
- the first field in the List is used as the Match Pattern
- when a match is found, the remaining fields in the List are overwritten with the remaining values in the matched row
Include | Exclude:
- the first field in the List is used as the Match Pattern
- when a match is found, the include | exclude logic is applied