Alluxio boosts scalability and adds multi-tenant support

Alluxio boosts scalability and adds multi-tenant support

Posted on

Alluxio Inc., the developer of an open-source distributed filesystem for use in data-intensive applications, today announced version 2.9 of its Data Orchestration Platform with a new scale-out, multi-tenant architecture, cross-environment synchronization, improved manageability and better support for the Kubernetes orchestrator for software containers. The new release also boosts security and performance improvements through the use of application program interfaces for Amazon Web Services Inc. S3 and Posix storage.

Alluxio manages data across an assortment of on-premises and cloud infrastructure, using caching to minimize latency and moving data close to the point of processing. The company targets large-scale analytics workloads using frameworks like Apache Spark, Presto and Hive.

It has become “the de-facto choice for large internet companies to accelerate the development of their data analytics and AI applications,” said Peng Chen, a big data engineer manager at Tencent Holdings Inc., in a prepared statement.

“Customers generally have no plans of moving away from on-premises environments entirely,” said Adit Madan, Alluxio’s director of product. “They have a highly heterogeneous platform and use Alluxio for portability.”

Metadata synchronization

The scale-out architecture uses metadata synchronization across multiple environments (pictured). This enables multiple Alluxio clusters to be deployed based on workload capacity. Clusters are aware of each other and can automatically synchronize metadata between them.

This feature is particularly useful for satellite architectures in which clusters are segregated across team-level tenants for isolation. It allows the platform to scale out and add new clusters without creating a central resource bottleneck. Tenant-level isolation is enabled through metadata synchronization.

Improved Kubernetes support makes the data stack more portable. It’s enabled through an operator that uses Kubernetes custom resource definitions to configure deployment, connections to underlying storage configuration updates and uninstallation.

Authentication and access policies have been centralized across compute engines and storage via the S3 API, which is “a core element of hybrid and multi-cloud where data is spread across environments,” Madan said. “This is a way to access data from any source with authentication done in a consistent way.”

Alluxio connects directly to Ping Identity Corp.’s PingFederate identity and access management platform and can also work with open-source governance platforms like Apache Ranger and “anything that conforms to OpenID,” which is a decentralized authentication protocol, Madan said.

Free downloads of the Alluxio 2.9 open-source community edition and a trial version of Alluxio enterprise edition are available here.

Image: Alluxio

Show your support for our mission by joining our Cube Club and Cube Event Community of experts. Join the community that includes Amazon Web Services and CEO Andy Jassy, Dell Technologies founder and CEO Michael Dell, Intel CEO Pat Gelsinger and many more luminaries and experts.

Source link

Leave a Reply

Your email address will not be published. Required fields are marked *