Collections

As any modern language, WarpScript implements a few collections types. In this tutorial, you will learn how to manage lists, maps, and sets.

Lists

Under the hood, this is a java ArrayList. This kind of resizable list allow any kind of elements, including NULL. Order is always preserved. REVERSE reverse the order of the collection.

To build a LIST, you can use [] to put an empty list on the stack, or you can put a MARK on the stack with [, put on the stack the elements of the list, then call the ] function. The ] function will take every element on the stack, put them in a list, and stops when it finds a MARK.

[ 1 2 3 ] MARK //less readable, prefer [ when you know you're starting a list. 1 2 3 ] [ 1 3 <% %> FOR //empty for loop. the loop index remains on the stack. ]

This is a mutable object. It means adding or removing an element do not return a copy of the list.

[ 1 2 3 4 5 ] 'mylist' STORE $mylist 6 +! 'mynewlist' STORE //add 6 to mylist, store the result. //push results on the stack $mylist $mynewlist

In the upper example, you can see $mylist is modified by the +! operation. This is why you can often DROP the result of +!.

[ 1 2 3 4 5 ] 'mylist' STORE $mylist 6 +! DROP //add 6 to mylist, drop the result. //push results on the stack $mylist

APPEND has the same behavior as +!, except it takes a list as input.

[ 1 2 3 4 5 ] 'mylist' STORE $mylist [ 6 7 ] APPEND 'mynewlist' STORE //add 6 and 7 to mylist, store the result. //push results on the stack $mylist $mynewlist

Lists are mutable. Saving a list variable in a new variable do not copy the list. If you do so, you will just have two variable pointing to the same object.

[ 1 2 3 4 5 ] 'mylist' STORE $mylist 'newlist' STORE $newlist REVERSE DROP //push results on the stack. Both contains 5 4 3 2 1. $mylist $newlist

If you want to make a copy of the list, you need to CLONE it.

[ 1 2 3 4 5 ] 'mylist' STORE $mylist CLONE 'newlist' STORE DROP //CLONE push on stack the copy, then the original. DROP removes the original from the stack. $newlist REVERSE DROP //push results on the stack $mylist $newlist

CLONEREVERSE clones and reverse the list order in the same operation.

LSORT allow you to sort lists of homogeneous elements.

[ "aa" "bb" "AA" "42" ] LSORT [ T F T F T F ] LSORT [ 5 8 4 -1 4 ] LSORT [ 5.0 8.0 4.0 -1.0 PI ] LSORT

A LIST can contain heterogeneous objects. Of course, you cannot compare different objects, so the resulting list won't be sortable.

[ ] 'mylist' STORE $mylist SIZE // push size on the stack (0) $mylist NEWGTS +! //put a GTS in the list "42" +! //put a string NULL +! //put a null [ ] +! DROP //put an empty string $mylist SIZE // push size on the stack (4) // $mylist LSORT //uncomment this line... It will fail.

You can create your own sort criteria and use SORTBY. In the following example, sorting lists by their length is really easy.

[ [ 1 2 3 ] [] [ 'a' 'b' ] [] ] 'mylist' STORE //list of lists $mylist <% SIZE %> SORTBY

See also GROUPBY for advanced grouping of lists.

GET, SET, REMOVE allow you to manipulate LIST elements. To check whether an element is present in a list, use CONTAINS. The first element of a LIST has index zero.

[ 1 2 3 ] 'mylist' STORE $mylist 0 GET //returns 1 $mylist 42 1 SET DROP //replace index $mylist 0 REMOVE DROP DROP //remove the first element $mylist //contains [ 42 3 ]

To iterate on list elements, you can use FOREACH or LMAP functions. FOREACH has no output format, you manage the stack the way you want. LMAP will build an output list with the elements you let on top of the stack.

[ 1 2 3 ] 'mylist' STORE $mylist <% 2 ** //square the element %> FOREACH $mylist <% DROP //drop the index 2 ** //square the element %> LMAP

SUBLIST allow you to extract a sublist from a LIST.

You can recursively FLATTEN a list of list of elements. LFLATMAP could be used for a non recursive flattening of a LIST.

[ [ 1 2 3 ] [ [ 1 2 3 ] [ 1 2 3 ] ] ] 'mylist' STORE $mylist FLATTEN //return 9 elements $mylist <% DROP %> //just drop the element index LFLATMAP //returns 5 elements: 1 2 3 and two lists

Flattening is limited to 16 levels of sublist. The following example will raise an exception:

[ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ 'Warp 10' ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] FLATTEN

ZIP is a way to merge several lists in a new list of list. The first output list will contain the first elements of each input list, the second output list will contain the second elements of each input list, and so on. If an input list is a singleton, its unique value will be considered for every output list.

[ [ 1 2 3 ] [ 'a' 'b' 'c' ] [ 42 ] //singleton ] ZIP

Explicit Warning: LIST manipulations are not synchronized. If you plan to parallelize macros that modify the same list, you need to use SYNC. See CEVAL for more information.

Maps

Maps are collections of key value pairs. Each key is unique. The simplest example is a dictionary: Every word is unique (the key), and has a definition (which could be the same for two different words). The key and the values could be any kind of objects, even NULL.

Under the hood, this is a java LinkedHashMap. It means the order of elements is guaranteed. Iteration functions will iterate in the order in which the entries were put into the map. In Python, it will be an ordered dictionary.

To build a MAP, you can use {} to put an empty map on the stack, or you can put a MARK on the stack with {, put on the stack the key values pairs of the map, then call the } function. The } function will take every pair of elements on the stack, put them in a map, and stops when it finds a MARK.

{ 'a' 1 'b' 5 // 'a' 4 //uncomment this line and try again } MARK //less readable, prefer using { mark 'a' 1 'b' 5 } { 1 5 <% //let the index on the stack, it will be the key RAND //put a random number on the stack, it will be the value %> FOR }

A MAP is a mutable object. It means adding or removing an element do not return a copy of the list. Storing a map variable into another variable just copy a reference to the same object in memory, except if you CLONE it:

{ 'a' 1 'b' 5 } 'map' STORE $map //push it on the stack $map 'newmap' STORE $newmap 8 'c' PUT DROP $newmap //push it on the stack $newmap CLONE 'clonemap' STORE DROP $clonemap 12 'cloned' PUT DROP $clonemap

When you build a map from stack elements, you cannot have key duplicates. When you PUT a new key value pair, or APPEND a new MAP to an existing one:

• If the key does not exist, the new key value is inserted at the end of the MAP.
• If the key exists in the MAP, its value is updated with the new one

When you retrieve a value from its key with GET:

• If the key exists, GET returns the value
• If the key does not exist, GET returns NULL.

To make the difference between a NULL value and a non-existing key, you can use CONTAINSKEY.

It means building counters of occurrence is really easy. In this example, I count the automotive brand occurrence in a list:

{ 'audi' 150 'volvo' 120 'dacia' 310 } 'brandcount' STORE //vehicle count per brand [ 'dacia' 'audi' 'dacia' 'dacia' 'porsche' 'volvo' 'tesla' ] //new inputs <% //enter the foreach loop, next list value is put on the top of the stack 'brand' STORE //store the brand <% $brandcount $brand CONTAINSKEY %> //contains key let the map on the stack <% //key exists, retrieve and increment the value $brandcount $brand GET 1 + //retrieve the existing value, increment it $brand PUT DROP //update the $brand key of the map . DROP the resulting MAP. %> <% //key do not exists, start a new counter 1 $brand PUT DROP %> IFTE %> FOREACH $brandcount //display the updated brand list

If your keys are homogeneous and sortable, you can sort the map up to its keys with MSORT.

Sets

A SET is a collection of unique elements. Under the hood, this is a java HashSet. It means the iteration order is not guaranteed at all.

The SET type cannot be represented on in the JSON stack output. You will see a "null" output. The only way to visualize it is to turn it into a LIST with SET->.

Building a set is very similar to maps or lists building. To build a SET, you can use () to put an empty set on the stack, or you can put a MARK on the stack with (, put on the stack the elements of the set, then call the ) function. The ) function will take every element on the stack, put them in a set, and stops when it finds a MARK.

You can also build it from a list with ->SET.

() SET-> //convert to list for visualization ( 45 46 47 48 -1 ) SET-> //order is NOT preserved ( 42 42 42 42 ) SET-> //you can put duplicates

Pro Tips

Some WarpScript functions sometimes take arguments in a LIST. For example, when you FETCH data. The result is a list of GTS on top of the stack. You need to BUCKETIZE these data as soon as you get them. You can use SWAP to put a MARK below the top of the stack:

[ 'TOKEN' 'className' { 'label0' '=value0' 'label1' '~val.' } start timespan ] FETCH [ //put a mark on the stack SWAP //here is the trick ! you swap the FETCH result and put it into your new list. bucketizer.max lastbucket bucketspan bucketcount ] BUCKETIZE