Reference architectures
At any scale
Edge
Designed for IoT, you can embed Warp 10 on your device.
- Millions of series
- 100s million of datapoints
- ~10k datapoints / s
Standalone
Meet the power of Warp 10 on your computer or on a single server.
- 10s million of series
- 100s of Billion of datapoints
- ~100k datapoints / s
Distributed
Scale the power of Warp 10 on your datacenter.
- Billions of series
- Trillions of datapoints
- Millions of datapoints / s
SAAS
No data center? No problem, we can host your data and you can access Warp 10 in the cloud.
The Warp 10 platform is available in two versions, standalone and distributed. Both versions provide the same level of functionality except for some minor differences, the complete WarpScript language is available in both versions. They differ mainly by the way the Storage Engine is implemented.
Standalone
The Standalone version is designed to be deployed on a single server whose size can range from a Raspberry Pi to a multi CPU box. It uses LevelDB as its storage layer or an in-memory datastore for cache setups.
All features (storage, analysis) are provided by a single process, hence the name standalone.
Multiple Standalone instances can be made to work together to provide High Availability to your deployment. This is provided via a replication mechanism called Datalog.
Distributed
The Distributed version coordinates multiple processes on multiple servers. The Storage layer uses Apache HBase for data persistence. Communication between processes is done through Kafka and ZooKeeper. This version is suitable for heavy workloads and giant datasets. Scalability comes with a price, the added complexity of the architecture.
Analytics Engine
Warp 10 can be deployed without its Storage Engine, simply as an Analytics Engine. This engine will provide a WarpScript execution environment and with custom extensions can fetch data from any Time Series datasource.
Performance
Deployments of the Standalone version have been demonstrated with datasets of several million Geo Time Series and hundreds of billions of datapoints.
The Distributed version has been used with over 500M Geo Time Series and several trillion datapoints in a rather modest setup.
Ingestion performance of LevelDB based Standalone version is in the 100k datapoints per second on a standard server (8 cores, 16GB RAM, SSD). Occasionally as your dataset grows and LevelDB compactions kick in, performance may degrade momentarily. Ingestion rates of close to 10k datapoints per second have been observed on a Raspberry Pi with SD storage. Ingestion performance for the in-memory data store has demonstrated at up to 120M datapoints per second.
Ingestion performance of the Distributed version of Warp 10 can scale by adding additional ingress processes. A couple of ingress daemons can absorb several million datapoints per second.