Amazon Web Services introduced Kinesis, a fully managed service for real-time processing of high-volume, streaming data.
The idea with Amazon Kinesis is to direct the flow of real-time data into cloud-hosted analytic applications and then display desired real-time intelligent via online dashboards.
The new service interoperates with other AWS big data services, such as Amazon Simple Storage Service (Amazon S3), Amazon Elastic Map Reduce (Amazon EMR) or Amazon Redshift. It scales elastically to support applications that process data streams of nearly any size, while replicating all data across multiple Availability Zones to help ensure high durability and availability. Amazon said Kinesis can allow a customer to store and process terabytes of data an hour from hundreds of thousands of real-time data sources, such as web site click-streams, marketing and financial transactions, social media feeds, logs and metering data and location-tracking events.
“Database and MapReduce technologies are good at handling large volumes of data,” said Terry Hanold, Vice President of New Business Initiatives, AWS. “But they are fundamentally batch-based, and struggle with enabling real-time decisions on a never-ending—and never fully complete—stream of data. Amazon Kinesis aims to fill this gap, removing many of the cost, effort and expertise barriers customers encounter with streaming data solutions, while providing the performance, durability and scale required for the largest, most advanced implementations.”
“Exchange order data peaks at tens to hundreds of megabytes per second, and we have to accurately record and process every single transaction,” said Ann Neidenbach, Senior Vice President of Global Software Development, NASDAQ OMX. “Amazon Kinesis is designed to handle market order data in real-time, durably and reliably, without data loss, enabling continuous analysis and auditing of this stream of market data. This lets us focus on building the next generation of innovative financial tools and services.”
