FETCH

gts

Configuration parameters

```
warp.aes.metasets
```
```
ingress.activity.window
```
```
ingress.activity.update
```
```
ingress.activity.meta
```
```
warpscript.maxfetch
```
```
warpscript.maxfetch.hard
```

The FETCH function interacts with the Warp 10 Storage Engine to retrieve data according to given criteria.

FETCH either accepts a list of 5 parameters or a map with the parameters. Those are defining the read access rights, the concerned Geo Time Series and the time window. Other parameters in the map input are optional.

The list input is here to keep backward compatibility with old WarpScripts. The map input allows a great flexibility, as described below.

Access rights

A valid read token is needed to read data with fetch. If you use a metaset and also specify a token, the token included in the metaset will be ignored.

Concerned Geo Time Series

FETCH selects Geo Time Series according to:

The selectors parameter, which is a list of selector.
If selectors is not found, FETCH uses the selector parameter which is a single selector.
If selector is not found, FETCH uses both class and labels parameters.

Time window

FETCH begins from the newest value and stop when the oldest value is collected. Thus, end must be defined in your request and defines the newest included value in your time window. If end is anterior to your oldest value, the result will be empty (no Geo Time Series). The span of the time window ending at end is then defined according to:

The timespan parameter.
If timespan is not defined, FETCH collects a maximum of count point.
If count is not defined, FETCH determines timespan with start. If start is more recent than end, end and start are permuted internally. Both start and end are included in the time window.

Boundaries

Since version 2.3.0, you can define boundaries around the requested data. When fetching data based on a time range, both a pre and post boundaries can be specified. When fetching by count, only a pre boundary can be requested. Boundaries are a number of datapoints which are either before (pre boundary) or after (post boundary) the fetched data. Fetching boundaries is very useful when storing only the changes of values as they enable you to always fetch datapoints even if the requested time range does not contain any. They can also be used to fetch the first N datapoints after a given timestamp. Note that fetching post boundaries is less efficient than fetching the requested range or a pre boundary as the data has to be scanned in reverse order which has an impact on I/Os and ultimately on performance.

Sampling options

Since version 2.3.0, you can randomly sample points with the sample parameter. If sample equals 0.1, FETCH will randomly return one point out of ten. This implementation is the fastest way to resample data.
Since version 2.3.0, you can skip the N newest datapoints in the time window specified.

Query without looking into directory

The fetch time to read one gts among millions of gts recorded with the same classname, the same application and the same owner can take time. Use the gts parameter to select if you exactly know the classname and all the labels.

If you want to read only Geo Time Series attributes or labels, using FIND is more efficient than fetching the last value.

FETCH availability

You cannot use FETCH within macro unit tests. When macro are loaded, directory and storage are not ready yet, so you will have a null pointer exception during macro loading. To build your unit tests, WRAP your data and use UNWRAP from a string instead of fetch.

FETCH is available since version 1.0.0.

Examples

//list parameter, fetch class ending with speed, with label id containing paris. //fetch from now the 1000 last values. [ $READTOKEN '~.*speed' { 'id' '~.*paris.*' } NOW -1000 ] FETCH 0 GET SIZE //take first GTS in the result list. size is 1000 //list parameter, fetch class ending with speed, with label id containing paris. //fetch values between two dates. (period is 100ms) [ $READTOKEN '~.*speed' { 'id' '~.*paris.*' } '2013-07-04T19:00:00Z' '2013-07-04T19:31:29Z' ] FETCH 0 GET SIZE //take first GTS in the result list. size is 14679 //list parameter, fetch class ending with speed, with label id containing paris. //fetch values 10 s before 2013-07-04 19h [ $READTOKEN '~.*speed' { 'id' '~.*paris.*' } [ 2013 07 04 19 ] TSELEMENTS-> 10 s ] FETCH 0 GET SIZE //take first GTS in the result list. size is 99 (period was not exactly 100ms)

// same examples, with a map parameter, { 'token' $READTOKEN 'class' '~.*speed' 'labels' { 'id' '~.*paris.*' } 'end' '2013-07-04T19:00:00Z' //look backward until this date 'count' 1000 //take 1000 points before this date } FETCH 0 GET SIZE //1000 values { 'token' $READTOKEN 'class' '~.*speed' 'labels' { 'id' '~.*paris.*' } 'end' '2013-07-04T19:00:00Z' //look backward until this date 'start' '2013-07-04T19:31:29Z' //no problem, end and start will be permuted internaly } FETCH 0 GET SIZE //14679 values { 'token' $READTOKEN 'class' '~.*speed' 'labels' { 'id' '~.*paris.*' } 'end' '2013-07-04T19:00:00Z' //look backward until this date 'timespan' 10 s //take 10 seconds of data before this date } FETCH 0 GET SIZE //99 values (period was not exactly 100ms)

// fetch two GTS without asking directory. You must know exactly all the labels of the GTS you want. // 'gts' can be the output of FIND function. { 'token' $READTOKEN 'gts' [ NEWGTS 'temperature' RENAME { 'device' $type 'serial' $id } RELABEL NEWGTS 'pressure' RENAME { 'device' $type 'serial' $id } RELABEL ] 'end' NOW 'count' 1 } FETCH