Datasources Overview
A Datasource lets you specify the data that will be processed to create reports.
This data can be contained in web server access logs, or in SaaS-based applications that Angelfish integrates with.
NOTES
- To process data, a Profile needs to be linked to at least one Datasource.
- A Datasource and Profile can only be linked if they're in the same Collection
- One Datasource can be used by multiple Profiles
- One Profile can use multiple Datasources.
- Each Datasource needs a Name, a Datasource Type, a Location, and relevant settings
- The Reprocessing Utility can only be used if the File Naming Convention field is populated.
- Use a Service Account to store authentication credentials for remote Datasources
DATASOURCE TYPES
The Datasource Type lets you specify how a Datasource accesses data.
Datasources can read data on local storage, remote instances, or on SaaS / CDN instances.
The selected Datasource Type also affects the fields that appear in the Authentication and Log Details sections. Not all Datasource Types require Authentication.
Here's what these sections look like when the Datasource Type is SharePoint Online:
Current Datasource Types include:
Local Filesystem
Angelfish accesses the local file system of the OS instance on which it's installed.
FTP
Uses FTP to access files on a remote server.
SFTP
Uses SFTP to access files on a remote server.
UNC
Uses UNC to access a shared folder on a remote server. This includes admin shares.
SharePoint Online
Angelfish connects to your SPO environment, downloads usage data, and creates logs which are stored locally.
Azure Blob Storage
Angelfish access files in an existing Blob Storage container
Azure CDN
Angelfish connects to your Azure environment, downloads data, and creates logs which are stored locally.
AWS S3 / CloudFront
Angelfish connects to your AWS S3 environment and reads access logs.
LOCATION FIELD
For most Datasource Types, the Location field stores the path & filename of data to be processed. The Location field is in the Log Details section (see screenshot above).
RegEx, Date Substitution Statements, and Wildcards can be used in the Location field. This lets you match existing logs and new logs as they're created in the future.
The Browse button to the right of the Location field opens a modal which allows you to verify your authentication details, view contents of remote directories, and see which files are matched by the contents of the Location field.
Syntax Examples
Local Filesystem
Windows:
C:\inetpub\logs\LogFiles\W3SVC3512\u_exYYMM.*Linux:
/usr/local/logs/website-name/access.YYYY.*SFTP
/usr/local/logs/website-name/access.YYYYMM.*UNC
\\server-name\LogFiles\W3SVC3512\u_exYYMM.*\\10.1.1.55\LogFiles\W3SVC3512\u_exYYMM.*The SharePoint Online and Azure CDN Datasources Types create logs from downloaded data. For both of these, the Location field contains the path where data will be saved (as compressed log files).
Help Article: RegEx
Help Article: Wildcards & Date Substitution
Help Article: Service Accounts
DataImage92.png
DataImage97.png
| Files | ||
|---|---|---|
|
|
DataImage92.png
82 KB
|
|
|
|
DataImage97.png
32 KB
|