0.9.5 Changes ------------- * FIXED: Name of index copy target is validated so you cannot copy a database and give it an invalid name. * FIXED: Printing result of ASK queries in the CLI. * FIXED: Format parsing for construct queries on the CLI where unrecognized format types were resulting in NullPointerExceptions. * FIXED: Databases which fail to initialize are no longer added to the server. * FIXED: Better cleanup of databases which failed to be created. * FIXED: Query explain over HTTP POST works and no longer returns a NOT_ACCEPTABLE response. * MODIFIED: Increased performance of database copy. * MODIFIED: Clearer error message when the server sub-command is mistyped. * MODIFIED: Improved access to disk indexes resulting in significant query performance improvements in many workloads. * MODIFIED: Clearer output messages from the 'drop' CLI command. 0.9.4 Changes ------------- * FIXED: Order by will correctly use secondary, tertiary, etc. orderings. * FIXED: Null values in sorting are correctly sorted as the lowest value. * FIXED: Resolved issue related to re-using a variable multiple times in the same BGP, correct results are now produced. * FIXED: Changed handling of Channel close events in database server to avoid endless propagation back to the client, particularly on Windows. * FIXED: Database cleanup on close events to avoid corrupting metadata. * FIXED: Handling of type errors in regex filter. * MODIFIED: New algorithm for handling ?x rdf:type ?o and :inst rdf:type ?o BGP's in queries with reasoning resulting in significant performance improvements. * MODIFIED: ChannelFactory is shared amongst all clients in the same JVM to avoid OOM errors due to an issue with Netty (NETTY-424). * MODIFIED: Login process to obtain a connection to a database changed to reduce connection overhead. * MODIFIED: Shiro session timeout handling to avoid erroneous timeouts during long processes, such as bulk loads. 0.9.3 Changes ------------- * MODIFIED: Server start script gracefully handles the situation when STARDOG_HOME does not exist, is not a directory, or is not writable * MODIFIED: Connection string parameters are now validated, errors are thrown when unknown parameters are provided. * MODIFIED: Return result of empty query results in SPARQL-JSON format when accessing the HTTP query endpoint directly. Returning an empty binding set instead of empty JSON. * MODIFIED: Checking of Accept headers for the query HTTP endpoint to better handle the case when multiple Accept headers are sent to the endpoint, but only one is relevant for the type of results to be returned. * MODIFIED: Reporting of query execution time of select queries in CLI to better reflect total execution time. * FIXED: Resolved issue in query generation arising from a variable being used in a BGP, context statement, and within an optional causing no results to be returned. * FIXED: Bug in query planner where a join variable was also used in a filter with an equality test. * FIXED: Bug in ICV when datatype restrictions are used in conjunction with a datatype property * FIXED: Dropping database on Windows now works; Windows was holding files open and we could not delete them. * FIXED: NPE when retrieving an unknown option for a database * FIXED: Inferences were missed because hasValue restrictions were incorrectly filtered in RL & EL profiles 0.9.2.1 Changes --------------- * FIXED: NPE when retrieving user permissions 0.9.2 Changes ------------- * MODIFIED: Security layer implementation changed to provide better performance when manipulating user, role, and permission information * MODIFIED: Reasoning & ICV interfaces to be more consistent along consistency checking & validation * MODIFIED: Major improvements to query re-writing performance. Re-writes are up to 3x faster, resulting in a noticeable improvement in executing queries with reasoning * MODIFIED: Improved CLI help * MODIFIED: Friendlier error messages when invalid SPARQL queries are provided in the command line * MODIFIED: More verbose startup & shutdown logging information * MODIFIED: Better output for ICV validation when the database is trivially valid * MODIFIED: A solution to quickly find nearly optimal query plans in case plan optimization takes too long * MODIFIED: Improved hash join algorithm for faster processing in case there are multiple alternative join keys * FIXED: Bug in query plan caching that could result in incorrect plans being used for similar queries * FIXED: Fixed cardinality estimations for some named graph scans * FIXED: Significantly better cardinality estimations for large plans, particularly involving multiple unions. Can result in improvements of up to 50% when bad joins can be avoided. * FIXED: Bug in ICV when getting constraints for a new database that does not yet have any constraints * FIXED: NPE that can arise during query evaluation after loading a large dataset * FIXED: Searchable configuration property can now be changed after creation * FIXED: Database load errors no longer prevent kernel from starting * FIXED: Auto-completion for local files in Stardog Shell works regardless of other parameters used in the command 0.9.1 Changes ------------- * ADDED: Stardog Shell * MODIFIED: Auto-update of statistics now lockless * MODIFIED: Made visible extension point for custom SPARQL functions. * FIXED: Cardinality estimation for queries across named graphs 0.9 Changes ----------- * ADDED: new native protocol, SNARL, based on Netty and Google's Protocol Buffers * ADDED: new system catalog for all system info, metadata, etc. * ADDED: new search system over RDF literals, integrated with SPARQL query eval * ADDED: new server management: online, offline databases; better server startup, shutdown, and locking * ADDED: CLI now supports remote db administration, many new subcommands * ADDED: new Quartz-based job scheduler with cron expression scheduling * ADDED: new, optimized index support for named graphs and triples-only databases * ADDED: new, differential write indexes with customizable merge thresholds * ADDED: new customizable TBOX (i.e., reasoning schema or ontology) extraction & management * ADDED: new Stardog configuration system, including database creation templates * ADDED: new reasoning services (explanation, satisfiability, consistency checking) available programmatically * ADDED: reasoning extended to named graphs * ADDED: ICV validation extended to named graphs * ADDED: new or improved support for NQUADS, Trig, TSV, and CSV * ADDED: improved concurrency in data loading * ADDED: consistent use of mmap() for better performance * ADDED: load GZIP-compressed files directly, including ZIP files with multiple RDF * MODIFIED: new security system, including user management, granular roles and permissions model * MODIFIED: new statistics computation & recomputation algorithms * MODIFIED: updated documentation * FIXED: many (many!) known bugs fixed NOTE: 0.9 introduces backwards incompatible changes in the on-disk format of Stardog databases. The migrate command in the CLI will not be able to migrate 0.7.3 or earlier databases to the current format. You will need to recreate the databases if you have an existing Stardog installation. 0.7.3 Changes ------------- * ADDED: New SNARL Connection view, ReasoningConnection which exposes the functions provided by the Stardog reasoner. * ADDED: Support in ReasoningConnection for obtaining the explanation for an inference. * ADDED: Explanations of IC violations as the expected and/or missing statements that make up the violation * ADDED: New example for using explanation facilities, updating ICV example. * MODIFIED: Improved error messages when CLI fails to connect to a database. * FIXED: Bug in the handling of reflexive properties when they involve TOP * FIXED: Bug in initialization of TOP operator with bound objects, resolves NPE in execution * FIXED: Bug in bnode handling in TBox extraction 0.7.2 Changes ------------- * ADDED: RDFS & DL reasoning types. DL reasoning type supports schema-only queries. * ADDED: Execution optimizations for when TOP appears in a query due to reasoning * MODIFIED: Sesame Repository/Sail implementation to use a SNARL Connection per Sail/RepositoryConnection rather than per Repository. * MODIFIED: Better formatting of query results in CLI query command including new formatting options. * FIXED: Retrieving ranges of data properties in reasoner. * FIXED: Read transaction isolation bug for mulitple connections against the same database. * FIXED: Bug related to incorrect re-association of Shiro auth credentials with pooled connections. 0.7.1 Changes ------------- * ADDED: Canonicalization of literal values on input, either bulk loading or normal add operations. Canonlicalization can be disabled when creating the database if it is not desired, but is on by default. * MODIFIED: CLI connection strings no longer assume you are connecting to an embedded store when no protocol is supplied. You must now specify a protocol. * FIXED: ExpressionFactory is no longer stripped from the core jar file * FIXED: Multi-line queries work with CLI now * FIXED: Bug in loading IC's from file, occasionally constraints could be malformed 0.7 Changes ----------- * ADDED: Bulk insert/removal performance on existing databases greatly improved via concurrent application of changes to the index and the introduction of a differential index. * ADDED: Stardog reasoner now supports OWL2 EL and RL reasoning types. These can be specified at connection time and EL & RL reasoning is performed at query time the same as QL. * ADDED: Support for 'hybrid' and TBox queries for all reasoning types. * ADDED: Support for Integrity Constraints. * MODIFIED: Propogation of reasoner state amongst open connection to the reasoner, state is better shared across connections requiring less overhead to maintain the up to date state. * MODIFIED: 'delete' was renamed to 'drop' in the command line. * FIXED: Bug in two-phase commits for changes to a reasoner * NOTE: The version of Sesame changes with this release from 2.3.3 to 2.3.4, which is not an official release. There was a bug [1] in the serialization of Sesame result sets for select queries using their binary format which we discovered while working on Stardog 0.7. The fix for SES-852 was included in their 2.6 branch, so we had to backport the change to Sesame 2.3.3; we called that version Sesame 2.3.4. [1] http://www.openrdf.org/issues/browse/SES-852 0.6.10 Changes -------------- * FIXED: Fixed an issue with not always consuming the HTTP response which could lead to not releasing HTTP connections to the pool and it its eventual exhaustion. * MODIFIED: Improved algebraic rewriting of query plans which avoid multiple evaluation of the same part of the query, e.g., the operator in both parts of a UNION. * MODIFIED: A better handling of the UNION operator by sorting its output which facilitates joins with other operators in the query plan. * MODIFIED: A better handling of deep chains of nested joins by maintaining a certain order of intermediate results produced by hash joins and avoiding page loads. * MODIFIED: An improved, thread-safe implementation of the query plan cache, which is now concurrently accessible from all connections. This allows the server to re-use a previously optimized query plan for a structurally equivalent query executed by another client. * ADDED: an init startup script for a Stardog server so that it can automatically start with other system services at the boot time. 0.6.9 Changes ------------- * FIXED: Fixed an issue pertaining to QL re-writes with optionals that use a filter which was causing incorrect results for some QL queries (ticket #44). * FIXED: Fixed query evaluation where the presence of a particular combintation of union, optional, and distinct can lead to non-distinct result sets (ticket #48). * FIXED: Fixed the passing of the disableSecurity flag into the HTTP server * FIXED: Fixed a bug in the rendering of SPARQL queries using isBlank with remote endpoints * FIXED: Fixed a bug in query evaluation arising from a certain combination of order by, limit, and optional where the sort key is on a variable bound by an optional pattern. * FIXED: User specified query bindings specified via Query.parameter are correctly serialized and sent to the remote server. * MODIFIED: Query rewriting now fails when a query has BGP's that are not supported by the reasoner; this is strict mode. Strict mode can be disabled and the reasoner will re-write only the BGP's it supports. This introduces a JVM flag 'strictReasoning' for controlling this behavior. * MODIFIED: HTTP connection will now switch over to HTTP POST if the SPARQL query is very long to avoid any potential URL overflow issues. * ADDED: CLI now includes an 'export' function to dump the contents of part or all of a Stardog database to RDF. 0.6.8 Changes ------------- * ADDED: Updated POM to include Avro * ADDED: Avro schema included in distribution. * FIXED: Fixed a bug in the caching of the current list of databases, could still access deleted databases in some circumstances. (ticket #42) * FIXED: Fixed an bug in the handling of unions in the QL reasoner. Nested unions were occasionally getting duplicated leading to incorrect results without the distinct modifier. (ticket #43, #46) * MODIFIED: Better handling of constant values in construct queries decreasing the time it takes to create and return the Statements constructed by the query. * MODIFIED: Increased the speed of some index scans by avoiding some page reads * MODIFIED: Better execution plan re-use & caching, similar to Prepared Statements, in structurally equivalent plans. * MODIFIED: Significantly improved join-join cardinality estimations. * MODIFIED: Query optimizer does a better job with join selection and can often pick a better join for the execution. * MODIFIED: Avro & HTTP connections are eagerly validated by the driver so invalid connections will fail on connect rather than on first action. * MODIFIED: Transaction ID generation changed to reduce chance of ID collision * MODIFIED: Internal connection management, both with raw database connections and HTTP based connections to address deadlock issues in highly concurrent environments 0.6.7 Changes ------------- * FIXED: HTTP error in construct queries via the Jena bindings * ADDED: New CLI command 'server' which can start any of the installed Stardog servers * ADDED: New Avro-based RPC protocol for Stardog Connections * ADDED: Maven POM file included in distribution 0.6.6 Changes ------------- * FIXED: incorrect query evaluation results in some corner cases involving OPTIONAL * MODIFIED: Improved query evaluation performance with a new query planner * MODIFIED: command-line interface to accept global args: --home, --change-buffer-size, --disable-security * MODIFIED: command-line add, remove, create commands to include --strict-parsing option 0.6.5 Changes ------------- * FIXED: Bug in query evaluation where we were missing results when the first entry in an iterator in a disk index occurs on a page boundry * ADDED: Support for full-text search via our semantic search engine Waldo. 0.6.2 Changes ------------- * FIXED: Typo in HTTP protocol, Trig & Trix mimetypes were switched. * FIXED: Bug in left outer joins when the right hand operator was initially empty causing the first solution set to be missed. (ticket #39) * FIXED: Resolved IllegalArgumentException which was incorrectly thrown from the Jena integration when a query w/ optionals was executed against a remote Stardog database returning null values in the response. * FIXED: Bug in detection of special BNode syntax in queries for ensuring BNode stability in queries. 0.6.1 Changes ------------- * FIXED: Debugging exception thrown when using log4j bindings for SLF4j with the logging level set to debug. (ticket #36) * FIXED: Issue with calcuation of optionals in joins when the optional binding is at the beginning or end of an iteration. (ticket #34) * FIXED: Resolved synchronization issue for connections to an index that are fetched from the internal connection pool. Resolves getting already closed connections from the pool. (ticket #37) * FIXED: Datatyped literals used in filters were getting munged during QL re-writes resulting in an exception. (ticket #35) * MODIFIED: Iteration over disk indices modified to more efficiently calcuate which pages are actually required to answer the query and avoid loading intermediate pages. 0.6 Changes ----------- * FIXED: Bug pertaining to loading datasets which use literals larger than 8k. * FIXED: Bug in the writer locks that could allow multiple writers access to an index in a multithreaded environment, which usually lead to corruptted indexes (ticket #29). * FIXED: Issue with the CLI's database create command, multiple calls in rapid succession to create databases caused some of the new databases to be lost (ticket #32). * FIXED: QL query rewriter was mangling the variables used within the query in certain situations causing query results to be incorrect (ticket #30). * FIXED: QL query rewriter was not correctly respecting variable substitutions with datatyped literals and was subsequently losing the variable binding specified by the user (ticket #31). * FIXED: FROM/FROM Named clauses were incorrectly set to the HTTP server * MODIFIED: Improved index page load speed. * MODIFIED: Reduced memory requirements in the indexes wrt to cache overhead * MODIFIED: Persistence of memory indexes to disk is now done in parallel to the greatest extent possible yield a significant increase in save (and load) performance. This required changing the on disk format of memory indexes. Old versions of memory indexes are unfortunately, backwards incompatible with these updates. We apologize for the inconvenience. * MODIFIED: On format of disk-based indexes. This helps us resolve some issues we had with the previous format, and increases load speed, but the index format is backwards incompatible with the previous indexes. However, we provide a migration utility in the command line program for changing old disk indexes to the new format. See the CLI for more information on migrating old indexes (or ping the mailing list). * MODIFIED: Decreased memory usage in the query engine * MODIFIED: Added a change buffer in the layer between the SNARL API and the underlying index it is connected to. This allows most changes within a transaction to be pushed into the index at once increasing the performance of commits as the changes can mostly be applied in bulk. 0.5.3 Changes ------------- * ADDED: Internal buffer for changes via the Sesame API, triples are no longer added to the Stardog database one at a time via the SAIL API. They are buffered until the buffer is full, a read operation occurs, or the transaction is committed. * FIXED: Internal bookkeeping for the state of the system information, timestamp for last update time is correctly updated and you should not get the out of sync warnings printed to the console. * FIXED: Incorrect return code for parse errors in HTTP protocol. * FIXED: NPE thrown by TBox extractor when encountering non object/data properties. * FIXED: Correct behavior for SPARQL queries w/ QL reasoning that uses the 'GRAPH' keyword. BGP's in graph contexts are not rewritten as inferences are put into the default, no context, graph. Hybrid queries that include some patterns against the default graph and others against patterns in graph contexts are correctly partially rewritten. (ticket #27) 0.5.2 Changes ------------- * MODIFIED: Add, Remove and Clear Contexts now take an optional query parameter, "graph-uri" which specfies the named graph the operation should be executed against. This deprecates the use of this parameter *in* the actual URL. See HTTP_PROTOCOL.txt for additional details. * MODIFIED: Temp files used to track changes during transactions and to buffer results coming into HTTP server are more aggresively removed. * FIXED: Specifying 'null' as the context to remove via the Connection API now actually deletes the default context rather than the entire contents of the database. * FIXED: Cast exception caused during QL query execution when the usage of the property in the query does not match the type (object v data) of the property in the kb. (ticket #24) * FIXED: NPE during evaluation of a query with QL reasoning enabled when the query has a optionals nested inside of a graph statement. (ticket #26) * FIXED: Could get NPE's during filter evaluation (compare expressions only) if one of the variables in the expression was not bound to any value in a BGP. * FIXED: Malformed literals in inserts could come back and cause errors when being removed. Errors are now thrown when a malformed literal (invalid language tag or inconsistent xsd datatype) is inserted into the database. (ticket #22 & #25) * FIXED: Resolving BGP's across contexts using wildcards (e.g. graph ?g { ... }) was losing information between the contexts leading to incorrect results and/or null values in the results. (ticket #20 & #21) * ADDED: Gremlin integration via our Sesame Sail implementation and the Blueprints API. See the README for more details on the integration. * ADDED: statements(subject, predicate, object, context) to the Remover API and deprecated statement(...); the latter only accepted non-null values, while the former does accept nulls as wildcards, so it is safe to replace all usages of one with the other. statement(...) is marked as deprecated and schedule for removal from the API no earler than 0.6. 0.5.1 Changes ------------- * FIXED: Deleting statements with wildcards via a StardogRepository. * ADDED: Support for the Sesame RepositoryFactory & Config APIs so Stardog can be used via the OpenRDF shell and Workbench. * ADDED: Native connection pools for Stardog connections. See the examples for code demonstrating how to configure and use them. 0.5 Changes ----------- * FIXED: Query objects reference to database was not refeshed after commit or rollback of transaction. (ticket #15) * FIXED: Bug in queries with graph contexts using unbound variables via an HTTP connection; the variable was getting mangled. * MODIFIED: Introduced new Flex Indexes into the Database structure. * ADDED: Shiro based security layer 0.4.10 Changes -------------- * FIXED: Bug in the concurrent calcuation of statistics where a latch could get stuck when certain types of exceptions occured during processing * FIXED: Issue in Remove CLI where description did not match behavior; Specifying only a context now deletes the context as stated in the documentation. (ticket #17) * FIXED: Bug in removing a context within a durable transaction that caused the transaction record to get corrupted. (ticket #16) * MODIFIED: Changed Add CLI to print an error message if no files are specified to add to the database. * MODIFIED: Removed default parse format of RDF/XML from add & remove operations. If a specific format is specified, that is used, otherwise the format based on the file extension will be used. This brings the behavior of add & remove inline with create. RDF/XML is still used as the fallback when no format is specified and there is a non-standard extension on the file. * MODIFIED: Modified how some filters are evaluated greatly increasing the speed in which basic (and/or/not & compares) filters are executed. 0.4.9 Changes ------------- * ADDED: Handling of the TOP concept in the OWL 2 QL reasoner * ADDED: New query operator which provides more efficient evaluation of queries where order by and limit are used. * ADDED: System information regarding which databases are present will be reloaded from disk when it is changed by an external source. * ADDED: JVM flag to disable strict mode of parsing. * FIXED: Encoding of graph URIs in remove context calls to HTTP server. * FIXED: Typo in route which effectively hid the remove context endpoint in the HTTP server. * FIXED: Mangling of ASK queries when sending them to the HTTP server. (ticket #13) * FIXED: Typo in help information in the add/remove data commands in the CLI. * FIXED: BOM markers at the start of files will be gracefully handled instead of failing to parse. * FIXED: If copy of system info on disk is more recent than cached copy in memory, it will not be overwritten on shutdown. (ticket #11) * FIXED: UnsupportedOperationException thrown when clearing a context within a transaction. (ticket #12) * FIXED: IllegalArgumentException thrown when literals are used in query atoms going through rewriting. (ticket #9) * FIXED: ClassCastException when non-QL axioms are used. (ticket #8) * FIXED: Issue with deleting directories when STARDOG_HOME contains relative paths. (ticket #6) * FIXED: NullPointerException when top level scope contains an empty set of query atoms. (ticket #7) * FIXED: Closing temporary files in the data directory used for some file operations. * FIXED: Concurrent modification exception from an iterator when persisting memory indexes to disk asynchronously. * FIXED: HTTP protocol was dropping context information from queries. * MODIFIED: Queueing of jobs during concurrent loading to avoid running out of memory when processing large datasets. This also reduces the amount of memory used during loading. * MODIFIED: Altered algorithm to get better estimates of join cardinalities. * MODIFIED: Memory databases to use new statistics algorithm. 0.4.8 Changes ------------- * ADDED: Creation of db indices is done in parallel. It is an experimental feature which can be disabled via the JVM flag (disableConcurrent); note, that concurrent loading requires more RAM than serial loading. Either give Stardog more RAM (i.e., as much as you cann spare) on startup or pass the disableConcurrent flag. * FIXED: Query atoms of the form { ?a ?a ?b } now return correct results * FIXED: Bug in multi-key joins * FIXED: Bug in QL reasoning optimizer for detecting empty joins * FIXED: File exhaustion bug. When Sesame integration was used with the HTTP server, the server did not always close the temporary files used to track the incoming edits causing some systems to report too many files were open. * FIXED: Bug in query evaluation where the named graph bound in a pattern could cause an exception during execution. * FIXED: Cardinality estimates in index statistics * FIXED: bug in stardog.sh script so that it can run anywhere (thanks Alin Dreghiciu for this fix) * MODIFIED: The name of the file being added/removed from a database in the command line is now printed. 0.4.7 Changes ------------- * Fixed an NPE in the HTTP server (reported by Brian Sletten) * make the QL axioms parser more strictly respect the OWL spec, especially w/r/t to the set of datatypes that QL supports