The vast majority of Big Data databases and architectures are still based on the concepts and commodity x86 hardware available when the Google Big Table paper was published in 2006. Since then, there have been tremendous advances in networking, server, and storage technologies that remain largely underutilized. Levyx has chosen to re-factor the lowest levels of the Big Data software stack to more fully exploit these hardware advances in what we call a “Scale-in, then Scale-out” approach.  This software stack can operate on a Cray, Power, X86, ARM platforms using NVME, IP, IB/RDMA, FC protocols and leveraging internal or externally connected flash storage and RAM SANs such as Kove. It is designed to eliminate or minimize the disruption to the rest of Big Data application software stack, while providing accelerated processing and storing of Big Data sets.
 
In addition, we have introducing a concept we call “persistent computing”, the ability to use newer storage technologies such as Intel Optane and NAND flash more directly in compute architectures as a partial substitute for memory while providing persistence.  Two versions of this technology are available-1) a massively scalable, low latency Key Value Storage which incorporates a compact index to supports point and range queries. 2) a Scale-out distributed data set and query/JITC off load platform that is integrated with Apache Spark.
 
Use cases in HPC include hi-speed, low-latency indexes and state managers, hi-speed streaming ingest capture and analytics, machine learning parameter servers, NoSQL DB accelerators, and hi-performance, persistent compute-ready data caches.
 
 We’ll also discuss how these offerings can facilitate workflow and processing of large amounts of data and reduce wall clock time spent loading and unloading large amounts of intermediate data during batch, multi-stage, and concurrent analytics or processing operations.
 
Bernie Wu has a BS/MS Eng from UC Berkeley and an MBA from UCLA.
http://www.levyx.com