Kinetica Database Now on Azure

October 12, 2021 – Database company Kinetica  is now accessible as a service on the Microsoft Azure cloud platform, designed to give organizations real-time contextual analysis and location intelligence on massive data sets with reduced computing infrastructure and lower costs.

Kinetica’s vectorized database is used to analyze data from sensors and machines in real time. For instance, one of the largest global retailers uses Kinetica to deliver dynamic, real-time inventory replenishment across its entire supply chain; several of the largest global telco’s use Kinetica to optimize network planning with coverage visualizations; and the Defense Department, uses Kinetica to monitor airborne threats over North America.

“Vectorization is ideal for IoT use cases where streaming geospatial and time-series data gets fused with other static or steaming data at speed. It historically required exotic hardware and specialized–and scarce–skills, putting it out of reach to all but the largest and most well-funded organizations or government entities,” said Nima Negahban, founder and CEO of Kinetica. “With Kinetica’s vectorized database now available as-a-service on Microsoft Azure, that has all changed. Any organization can harness the power of Kinetica for IoT initiatives — and it can be deployed in minutes.”

Kinetica on Microsoft Azure is fully managed by Kinetica, integrated with Microsoft Azure monitoring, and equipped with a modern user interface for ease of use. It provisions in minutes, streamlines data ingestion, and delivers seamless analysis to provide exceptional time-to-value. Its consumption-based pricing lets users pay as they go, choosing between vectorized CPU pricing and GPU pricing.

IoT data is forecasted to reach 73 zettabytes by 2025, according to IDC, while a recent study by Deloitte estimates that 40% of IoT devices will be capable of sharing location in 2025, up from 10% in 2020. This makes data with both a time-series and spatial component the fastest growing category of big data this decade. Adoption of geospatial data is starting to become more widespread across the business sector. Emerging high value use cases that leverage continuous readings over time with geospatial coordinates include proximity-based marketing, smart grid operations management, environmental remediation, contact tracing, spatial determination of health outcomes, connected car services, fleet optimization, and others.

Data across time and space presents organizations with three fundamental challenges:

  • Sensor and machine data is extremely large and fast moving when compared to the first generation of big data sets that were mostly human generated interactions with the web
  • The value from this data comes from fusing data together for context through geo and temporal joins versus traditional primary and foreign key relationships
  • Insights come from machine learning with advanced capabilities for geospatial and time series analytics

The current generation of massively parallel processing (MPP) databases for big data analytics, such as BigQuery, Cassandra, and Snowflake, simply weren’t designed to handle the speed, unique data integration requirements, and advanced spatial and temporal analytics on data across time and space. Past approaches to harnessing value from this kind of data have fallen short, resulting in decisions not being made fast enough, a lack of critical context, and sub optimized insights.  On top of that, costs are excessive due to inefficiencies with respect to both development efforts and compute costs.

Data-level parallelism, or vectorization, accelerates analytics exponentially by performing the same operation on different sets of data simultaneously, for maximum performance and efficiency. Data level parallelism is particularly adept at functions required to perform advanced calculations across time and space, such as windowing functions, predicate joins, graph solving and others. Vectorization underpins AI initiatives and high-performance computing, but these vectorized computing processors have yet to enter the mainstream for analytics projects such as route optimizations, real-time risk assessment, and visual mapping.

Kinetica’s new offering changes that. By harnessing the built-in vectorization capabilities of the latest generation of chips from Nvidia and Intel in the cloud, and making it available as a service, Kinetica now delivers orders of magnitude faster processing compared with traditional cloud databases.

At a top financial services firm, for instance, a 700-node Spark cluster running queries in hours took seconds on 16 nodes of Kinetica. At a top retailer, 100 nodes of Cassandra and Spark were consolidated into eight Kinetica nodes. A big pharmaceutical company achieved identical performance between an 88-node SQL on Hadoop cluster and a 6-node Kinetica cluster in Microsoft Azure.

“We’re using it today,” said NORAD and U.S. Northern Command General Glen VanHerck at the Air Force Association’s virtual Air Warfare Symposium. “It’s out in our fields and our sectors right now. Historically our sectors were very manually driven — phone calls to pass data, etc. Today we fuse all that data together, and we’re seeing the picture much more real-time and much more in an automatic type of digital environment.”

“Kinetica’s fully-vectorized database on Microsoft Azure Marketplace significantly outperforms traditional cloud databases for big data analytics,” says Jeremy Rader, GM, Enterprise Strategy & Solutions for the Data Platforms Group at Intel, “and now does so at the same speed as a GPU but at a fraction of the cost by harnessing data-level parallelism using the built-in Advanced Vector Extensions (AVX-512) of our latest 3rd Gen Intel® Xeon® Scalable processors.”

“We’re pleased to offer Kinetica as a fully vectorized database as-a-Service on Microsoft Azure,” says Ramnik Gulati,Director Product Marketing Databases at Microsoft. “Kinetica is the database for space and time, and is now available to more customers and markets with their release into the growing Azure Marketplace ecosystem.”

“Accelerated computing is key to breakthroughs in machine learning, data science, visualization, simulation, and computer-aided design,” said Scott McClellan, senior director at NVIDIA. “Kinetica’s new as-a-service offering on Microsoft Azure enables enterprises to easily speed up the databases that power their work with NVIDIA GPUs.”

Kinetica is available immediately on the Microsoft Azure Marketplace, with as-a-service in AWS available later this year.