FETCH
gtsConfiguration parameters
-
warp.aes.metasets
-
ingress.activity.window
-
ingress.activity.update
-
ingress.activity.meta
-
warpscript.maxfetch
-
warpscript.maxfetch.hard
The FETCH
function interacts with the Warp 10 Storage Engine to retrieve data according to given criteria.
FETCH
either accepts a list of 5 parameters or a map with the parameters. Those are defining the read access rights, the concerned Geo Time Series and the time window. Other parameters in the map input are optional.
The list input is here to keep backward compatibility with old WarpScripts. The map input allows a great flexibility, as described below.
Access rights
A valid read token is needed to read data with fetch. If you use a metaset and also specify a token, the token included in the metaset will be ignored.
Concerned Geo Time Series
FETCH
selects Geo Time Series according to:
- The
selectors
parameter, which is a list of selector. - If
selectors
is not found,FETCH
uses theselector
parameter which is a single selector. - If
selector
is not found,FETCH
uses bothclass
andlabels
parameters.
Time window
FETCH
begins from the newest value and stop when the oldest value is collected. Thus, end
must be defined in your request and defines the newest included value in your time window. If end
is anterior to your oldest value, the result will be empty (no Geo Time Series). The span of the time window ending at end
is then defined according to:
- The
timespan
parameter. - If
timespan
is not defined,FETCH
collects a maximum ofcount
point. - If
count
is not defined,FETCH
determinestimespan
withstart
. Ifstart
is more recent thanend
,end
andstart
are permuted internally. Bothstart
andend
are included in the time window.
Boundaries
Since version 2.3.0, you can define boundaries around the requested data. When fetching data based on a time range, both a pre and post boundaries can be specified. When fetching by count, only a pre boundary can be requested.
Boundaries are a number of datapoints which are either before (pre boundary) or after (post boundary) the fetched data. Fetching boundaries is very useful when storing only the changes of values as they enable you to always fetch datapoints even if the requested time range does not contain any. They can also be used to fetch the first N
datapoints after a given timestamp.
Note that fetching post boundaries is less efficient than fetching the requested range or a pre boundary as the data has to be scanned in reverse order which has an impact on I/Os and ultimately on performance.
Sampling options
- Since version 2.3.0, you can randomly sample points with the
sample
parameter. Ifsample
equals 0.1,FETCH
will randomly return one point out of ten. This implementation is the fastest way to resample data. - Since version 2.3.0, you can
skip
the N newest datapoints in the time window specified.
Query without looking into directory
The fetch time to read one gts among millions of gts recorded with the same classname, the same application and the same owner can take time. Use the gts
parameter to select if you exactly know the classname and all the labels.
If you want to read only Geo Time Series attributes or labels, using FIND is more efficient than fetching the last value.
FETCH availability
You cannot use FETCH within macro unit tests. When macro are loaded, directory and storage are not ready yet, so you will have a null pointer exception during macro loading. To build your unit tests, WRAP your data and use UNWRAP from a string instead of fetch.