1. Stack
  2. Execution
    1. 2 + 2 Example
  3. Results
  4. Syntax
    1. Comments
  5. Types
    1. WarpScript™ base types
    2. WarpScript™ composite types
    3. WarpScript™ GTS type dedicated to Geo Time Series™
    4. WarpScript™ special types
  6. Variables
    1. Managing variables versions via contexts
  7. Functions
  8. Frameworks
    1. Frameworks working on individual Geo Time Series™
    2. Frameworks working on equivalence classes of Geo Time Series™
  9. Macros
    1. Example
  10. Flow Control
    1. Loops
    2. Conditionals
    3. Asynchronous Transfer of Control
  11. Unit Testing
  12. Processing

WarpScript

WarpScript

We created WarpScript™, an extensible stack oriented programming language dedicated to Geo Time Series™ analytics which offers more than 900 functions and several high level frameworks to ease and speed your data analysis. Simply create scripts containing your data analysis code and submit them to the platform, they will execute close to where the data resides and you will get the result of that analysis as a JSON object that you can integrate into your application.

The WarpScript™ approach is another differentiating factor of Warp 10™. Traditional time series platforms offer few manipulation options, usually only providing a SQL like query language which cannot express complex analysis, or providing a reduced set of aggregation functions. These approaches force you to produce more code on the client side thus increasing your development time and leading to massive transfers of unprocessed data from the platform to your applications. Our approach lets you focus on your business use cases, simplifying IoT and sensor data applications by taking care of a larger chunk of the data analysis in a very efficient way.

It is a stack-oriented language with solid foundations.

Stack

The stack has two fundamental operations, push and pop.

stack

The stack evolves as data is pushed and functions are called.

stack

For convenience, the stack is often displayed as below

Stack levelStack Content
Some stack level may be omitted42
[3]3.141592653589793
[2]2.718281828459045
[TOP]'foobar'

Execution

  • WarpScript™ is a concatenative programming language.
  • Expressions denote function calls.
  • Juxtaposition of Expressions denote function composition.
  • Function calls modify the stack
  • Constants can be considered special functions

Execution is very similar to Unix pipelines:

  • UNIX: ls -l | xargs md5sum | less
  • WarpScript: FUNC1 FUNC2 FUNC3

Each function call pops its parameters off the stack and pushes its results onto the stack.

WarpScript™ is executed within the Warp 10™ Analytics Engine, code submission is done via a simple HTTP Request.

WarpScript™ is like a RPN language.

2 + 2 Example

  • Push 2 on the stack
  • Push 2 on the stack
  • Call + function

+ pop the first two elements of the stack, add them together, and push back the result on the stack.

2 2 +

Results

For this WarpScript:

42 //push 42 on the stack
PI //call PI function. PI pushes 3.1415... on the stack.
2.718281828459045 //push e on the stack
'foobar' //push a string on the stack

The http answer of the exec endpoint will be :

< HTTP/1.1 200 OK
< Date: Fri, 03 Aug 2018 14:58:40 GMT
< Access-Control-Allow-Origin: *
< Access-Control-Expose-Headers: X-Warp10-Elapsed,X-Warp10-Ops,X-Warp10-Fetched
< X-Warp10-Elapsed: 307164
< X-Warp10-Ops: 5
< X-Warp10-Fetched: 0
< Vary: Accept-Encoding, User-Agent
< Content-Length: 49
< 
["foobar",2.718281828459045,3.141592653589793,42]              

Execution in the Warp 10™ Analytics Engine returns a JSON Array with one element per stack level, top first.

Stack levelStack ContentJSON output
`42`

[
    "foobar",
    2.71828182845904,
    3.14159265358979,
    42
]
        
[3]3.141592653589793
[2]2.718281828459045
[TOP]'foobar'

Syntax

White spaces are used as delimiter between instructions.

Carriage return has the same effect as the white space.

'hello' 'world' 'from Warp 10'

Comments

// single line comment # single line comment 'hello' // single line comment 'hello' # single line comment /* multi line comment */

Types

WarpScript™ base types

  • Numerical types LONG and DOUBLE, 64 bits signed
  • BOOLEAN type, values true and false
  • STRING type, %-URL encoded in UTF-8 'Caf%C3%A9'
  • NULL value

Strings

'hello' // single line string "hello I'm Warp 10" // single line string <' multi line string '> 'Caf%C3%A9'

Numeric values

1701 // LONG 3.141592653589793 // DOUBLE

Boolean values

true false T F

Related references

WarpScript™ composite types

  • Lists of values [ value1 value2 … valueN ]
  • Maps of key/values { key1 value1 key2 value2 }
  • Sets of values ( value1 value2 value3 value4 )

Sets are not displayable in JSON. You will see null on the stack output. Use SET-> to convert a set into a list.

[ 1 2 3 'r' ] { 'a' 'A' 'b' 'B' 'c' 3 } ( 'a' 'b' 'd' 'd' ) SET-> // a set is not JSON compliant

Composite types can be nested.

WarpScript™ GTS type dedicated to Geo Time Series™

gts

GTS is a special WarpScript object that contains data. It has his own text format for input, described in GTS input format.

  • Class name is the name of the GTS.
  • Labels describes the GTS.
  • A unique GTS is defined by its name and its labels. It means you cannot change labels without creating a new GTS in the storage layer.
  • Attributes are sticky notes attached to each GTS. You can store any key value here, and update or remove them in the storage.
  • Values can be long, double, strings, boolean. The first value stored fix the GTS value type.
  • Latitude, Longitude, Elevation are optional.

A tutorial describes GTS operations.

WarpScript™ special types

Some special types are created by WarpScript™ functions: GeoXPShape, PGraphics, byte arrays, matrices, vectors, ...

Any Java type can be produced by a function, ideally types should be extended to be Snapshotable.

Variables

Variables are used to store values beyond the stack. It increase readability by reducing stack mental gymnastics.

Store values via value 'symbol' STORE

Push back value on the stack via $symbol or !$symbol

Can also use the syntax 'symbol' LOAD

'Spock' 'myvariable' STORE //store a string in myvariable 'Hello ' $myvariable + //contatenate both strings

Related references

Managing variables versions via contexts

By default, variables are global. A context is a snapshot of all defined symbols and can be pushed on the stack and later restored. It is useful to manage nested scopes.

This is easily done via the SAVE ... RESTORE pattern.

Related references

Functions

WarpScript™ has over 900 functions organized in categories.

Frameworks

Frameworks working on individual Geo Time Series™

  • BUCKETIZE - Make ticks evenly spaced
  • MAP - Apply a function on a sliding window around each tick

Frameworks working on equivalence classes of Geo Time Series™

Geo Time Series™ are first partitioned by selected labels

  • REDUCE - Apply a function tick by tick to multiple Geo Time Series™
  • FILTER - Select Geo Time Series™ based on some criteria
  • APPLY - Apply an N-ary function to lists of Geo Time Series™, tick by tick

Macros

Macros allow for code factorization, code fragments which can be invoked.

They can be deployed:

  • Server side in Warp 10™ Analytics Engine.
  • Via jar files for use with WarpScript™ library
  • Locally with VSCode plugin automatic substitution.
<%
  <'
Macro documentation
  '>
  DOC

  SAVE 'context' STORE

  // Code of the actual macro

  $context RESTORE
%>
'macro' STORE
// Unit tests
$macro

To evaluate a function you can use $name EVAL or @name.

Related references

Example

This example show context saving used in macro.

<% <' This macro repeats n times a string It takes a number on the top of the stack, and a string below. '> DOC //this DOC part is not mandatory, just a best practice. //it allows automatic documentation generation. SAVE 'context' STORE //store all the variables in 'context' object 'n' STORE //store number argument 's' STORE //store string argument in s '' //push empty string on the top of the stack 1 $n //for 1 to n <% DROP //drop the for loop index $s + //concatenate with the top of the stack %> FOR $context RESTORE //restore all the variables from 'context' %> 'stringrepeat' STORE 42 's' STORE //store 42 in 's' variable 'Warpy! ' 5 @stringrepeat $s // 's' variable was not erased by the macro. //Now, try to comment SAVE and RESTORE line in the macro : // s will be replaced by the macro code.

Flow Control

Loops

FunctionSample
FOR$from $to <% /* ACTION */ %> FOR
FORSTEP$from $to <% /* STEP */ %> <% /* ACTION */ %> FORSTEP
WHILE<% /* COND */ %> <% /* ACTION */ %> WHILE
UNTIL<% /* ACTION */ %> <% /* COND */ %> UNTIL

Conditionals

If-Then: IFT

<% /* COND */ %>
<% /* ACTION */ %>
IFT

If-Then-else: IFTE

<% /* COND */ %>
<% /* ACTION */ %>
<% /* ALT */ %> 
IFTE

Switch: SWITCH

<% /* COND 1 */ %> <% /* ACTION 1 */ %>
<% /* COND 2 */ %> <% /* ACTION 2 */ %>
<% /* DEFAULT ACTION */ %> 2 SWITCH

Asynchronous Transfer of Control

BREAK, CONTINUE, FAIL, MSGFAIL, NRETURN, RETHROW, RETURN, STOP

Try/catch TRY

<% /* ACTION */ %>
<% /* CATCH CLAUSE */ %>
<% /* FINALLY CLAUSE */ %>
TRY

Unit Testing

ASSERT, ASSERTMSG

$result 'expected' == ASSERT
$result 'expected' 'Message' ASSERTMSG

Processing

Warp 10 includes a large subset of the Processing drawing library. You can create and manipulate images, then push a base64 encoded png onto the stack.

  • Step 1: create a processing graphic object with PGraphics function.
  • Step 2: manipulate this object with Processing functions.
  • Step 3: render the processing graphic with Pencode function.

To view the results, you can use VSCode plugin for WarpScript.

//simple example 200 150 '2D3' PGraphics //new image instance, 200x150 pixels 0xffffff6c Pbackground //yellow background 3 PstrokeWeight //stroke width 3pixels 0xff0000ff Pstroke //blue stroke (ARGB color) 0x7fff0000 Pfill //semi transparent red fill 10 30 100 50 Prect //draw a rectangle, left corner 10 30, size 100 50 20 PtextSize //set text size to 20pt 0xffff0000 Pfill //red fill "Hello Warp 10" 10 100 Ptext //text, bottom left at 10 100. Pencode //render the image in a base64 format on the stack