Stardog 5.3 includes 5x cluster performance improvement and S3 backups. Read on for the details.
We’re happy to announce the latest release of Stardog, which brings new capabilities: S3 backups, cluster performance improvements, and a new query evaluation extension point.
Up to 5x Cluster Perf Increase
Stardog’s cluster is a strongly consistent system based on synchronous replication. Essentially we delegate commits to an elected coordinator which waits for successful replication of each transaction to other members of the cluster. Despite performing replication to independent nodes in parallel, this was still the biggest bottleneck in use cases with small transactions. The smaller the transaction, the more overhead the old replication protocol imposed.
After analyzing further, we identified two potential improvements: reducing the size of the messages and reducing the number of messages. This analysis spanned many different components in the system ranging from the HTTP client to the authentication layer to the cluster commit protocol. We were able to significantly reduce both the size and number of messages resulting in optimized cluster performance for small transactions, i.e., those composed of a few thousand triples or less. This increase is significant: up to 5x in some cases. Systems with larger transactions should also experience a noticeable performance increase, which tails off as the transaction size approaches 100k.
No code or configuration changes are necessary to take advantage of this speed-up. Test your cluster with Stardog 5.3 to experience it for yourself.
S3 Backup & Restore
Backups are indispensable and even more so for critical data. We all know it. But putting a disaster recovery plan into place isn’t the most exciting way to spend time. This support includes any S3 implementation. With it you can reduce the maintenance burden of disaster recovery processes by using S3 as a safe place to store backups.
With Stardog 5.3, we are introducing native support for direct S3 backup. This eliminates any room for error in using shell scripts or other glue code or manual processes. This seamless integration works by specifying an S3 URL instead of a filename:
stardog-admin db backup \ --to "s3:///mybucket/backup/prefix?region=us-east-1&AWS_ACCESS_KEY_ID=accessKey&AWS_SECRET_ACCESS_KEY=secret" \ myDb
The same works with the
db restore command.
S3 provides a reliable off-site storage service which is perfect for backups. Don’t shirk the responsibility of keeping your data safe.
Customizing SPARQL’s DESCRIBE
DESCRIBE query form is used to collect data related
to a graph node. For example, to see what we know about Tom Hanks:
:Tom_Hanks a :Person ; :actedIn :Forest_Gump .
In general, SPARQL is a declarative query language with formally
specified semantics. The
DESCRIBE query form, however, is not
specified and the details are left to be provided by the
implementation. Stardog’s default implementation prior to 5.3 returns
a set of statements where the resource being described is the
subject. With Stardog 5.3, it’s now possible to create your own
DESCRIBE and choose between implementations for
In 5.3 implementations of
DESCRIBE are called strategies. They
are written in Java by implementing the
interface. Implementations are registered using the Java service
loader by declaring the implementation of this interface. Strategies
are identified by a name and are chosen at runtime with the new
describe.strategy query hint like so:
#pragma describe.startegy bidirectional DESCRIBE <theResource>
The default describe strategy can also be set per database using the
We’ve included two new describe strategy implementations:
CBD - Concise Bounded Description
strategy is a well-known specification for the
description of a resource. The bidirectional strategy augments the default
strategy by adding triples where the resource being described is the
object of the triple.
This new facility provides a very specific way to extend Stardog which can remove the need to compute descriptions externally. Customers benefit by keeping business logic centralized and consistent.
We aim in all things to provide value to customers who are building knowledge graphs. Sometimes this requires low-level systems engineering to meet performance targets and other times it requires devops tools necessary to run a reliable deployment with a minimum amount of intervention. Or we can extend graph query evaluation with new features like GraphQL and PATH queries and now with DESCRIBE strategies.
Check out Stardog 5.3 today–including the full Release Notes–and let us know what you think.