Like an everflowing stream: New tech promises remote S3 nearline disk performance

www.theregister.co.uk | 7/6/2018 | Staff
rach-rach (Posted by) Level 3
Click For Photo: https://regmedia.co.uk/2018/07/06/foreststream.jpg?x=1200&y=794

Analysis You can't store files in Amazon's public cloud, access them on-premises, and expect local disk access performance.

You can store them in a sync-and-share facility like Box and Dropbox but then they have to be downloaded completely. It's not so good for large files, large data sets and production environments.

Storage - Nasuni - Panzura - Complexity

You could also use a cloud storage gateway, like Nasuni or Panzura, which works fine but adds complexity and may not scale.

Startup LucidLink claims it uses local metadata caching, parallel TCP/IP streaming, pre-fetching and caching to make public cloud-stored files usable for on-premises primary data storage.

People - CEO - Peter - Thompson - CTO

It was founded by two ex-DataCore people, CEO Peter Thompson and CTO George Dochev. That background is relevant because DataCore uses parallelised IO in its record-breaking SPC-1 v1 benchmark results.

Dochev was DataCore's director of software engineering until June 2015. LucidLink was founded in January 2016, took in $1.6m in seed funding in December that year and has just had a second seed round, $5.5m, this year.

Way - Files - Object - Stores

What they appear to have come up with is a faster way of streaming files from remote object stores.

Let's start by having files stored as objects in S3 buckets, Amazon being their first supported cloud.

Things - Way - NFS - CIFS - SMB

The things that get in the way of being able to use NFS, CIFS or SMB to stream data from them for on-premises use are time and latency. TCP/IP, for example, is a chatty protocol, with many metadata message sequences as well as data transfer sequences.

Specialist suppliers, such as Bridgeworks in the UK, speed things up by parallelising TCP/IP streams and so cut the transfer time. That's part of what LucidLink's technology does.

Architecture - Diagram - LucidLink - Store - Amazon

An architecture diagram shows a LucidLink store in Amazon S3 and a LucidLink app (or agent) in the customer's server. This stores synced metadata from the Amazon-resident LucidLink store and presents the LucidLink files as...
(Excerpt) Read more at: www.theregister.co.uk
Wake Up To Breaking News!
Sign In or Register to comment.

Welcome to Long Room!

Where The World Finds Its News!