diff --git a/Documentation/6.0/Raven.Documentation.Pages/server/configuration/indexing-configuration.markdown b/Documentation/6.0/Raven.Documentation.Pages/server/configuration/indexing-configuration.markdown index da975eeebf..1ffae9d881 100644 --- a/Documentation/6.0/Raven.Documentation.Pages/server/configuration/indexing-configuration.markdown +++ b/Documentation/6.0/Raven.Documentation.Pages/server/configuration/indexing-configuration.markdown @@ -210,9 +210,12 @@ EXPERT ONLY: {PANEL: Indexing.Auto.ArchivedDataProcessingBehavior} -Set the default deployment mode for static indexes. +The default processing behavior for archived documents in auto-indexes. -- **Type**: `enum IndexDeploymentModeArchivedDataProcessingBehavior` (`ExcludeArchived`, `IncludeArchived`, `ArchivedOnly`) +- **Type**: `enum ArchivedDataProcessingBehavior`: + * `ExcludeArchived`: only non-archived documents are processed by the index. + * `IncludeArchived`: both archived and non-archived documents are processed by the index. + * `ArchivedOnly`: only archived documents are processed by the index. - **Default**: `ExcludeArchived` - **Scope**: Server-wide, or per database @@ -351,9 +354,16 @@ Allow to open an index without checking if current Database ID matched the one f {PANEL: Indexing.Static.ArchivedDataProcessingBehavior} -Set the default deployment mode for static indexes. +* Set the default processing behavior for archived documents in static indexes. +* This setting applies only to static indexes that use _Documents_ as their data source. + It does not apply to indexes based on _Time Series_ or _Counters_, which default to `IncludeArchived`. + +--- -- **Type**: `enum IndexDeploymentModeArchivedDataProcessingBehavior` (`ExcludeArchived`, `IncludeArchived`, `ArchivedOnly`) +- **Type**: `enum ArchivedDataProcessingBehavior`: + * `ExcludeArchived`: only non-archived documents are processed by the index. + * `IncludeArchived`: both archived and non-archived documents are processed by the index. + * `ArchivedOnly`: only archived documents are processed by the index. - **Default**: `ExcludeArchived` - **Scope**: Server-wide, or per database diff --git a/Documentation/6.0/Raven.Documentation.Pages/server/extensions/.docs.json b/Documentation/6.0/Raven.Documentation.Pages/server/extensions/.docs.json index 5c306d031c..1d490b6e10 100644 --- a/Documentation/6.0/Raven.Documentation.Pages/server/extensions/.docs.json +++ b/Documentation/6.0/Raven.Documentation.Pages/server/extensions/.docs.json @@ -20,6 +20,12 @@ "Path": "archival.markdown", "Name": "Data Archival", "DiscussionId": "b976732a-3da7-48e5-a454-4f34b8f716f9", - "Mappings": [] + "Mappings": [ + { + "Version": 6.2, + "Key": "data-archival/overview" + } + ], + "LastSupportedVersion": "6.0" } ] diff --git a/Documentation/6.0/Raven.Documentation.Pages/server/kb/javascript-engine.markdown b/Documentation/6.0/Raven.Documentation.Pages/server/kb/javascript-engine.markdown new file mode 100644 index 0000000000..1c4ac603b1 --- /dev/null +++ b/Documentation/6.0/Raven.Documentation.Pages/server/kb/javascript-engine.markdown @@ -0,0 +1,160 @@ +# Knowledge Base: JavaScript Engine +--- + +{NOTE: } + +* RavenDB integrates **JavaScript scripting** across various features, including: + * [RQL projections](../../indexes/querying/projections) + * [Subscriptions](../../client-api/data-subscriptions/creation/examples#create-subscription-with-filtering-and-projection) + * [ETL](../../server/ongoing-tasks/etl/basics) + * [Smuggler (data import/export)](../../client-api/smuggler/what-is-smuggler#transformscript) + * [Single](../../client-api/operations/patching/single-document) or [Set based](../../client-api/operations/patching/set-based) document patches + * [Time Series](../../document-extensions/timeseries/client-api/javascript-support) + and + [Incremental Time Series](../../document-extensions/timeseries/incremental-time-series/client-api/javascript-support) + +* To execute JavaScript code, + RavenDB uses [Jint](https://github.com/sebastienros/jint), an open source JavaScript interpreter supporting ECMAScript 5.1. + +* In this page: + * [How RavenDB uses Jint](../../server/kb/javascript-engine#how-ravendb-uses-jint) + * [Predefined JavaScript functions](../../server/kb/javascript-engine#predefined-javascript-functions) + + +{NOTE/} + +--- + +{PANEL: How RavenDB uses Jint} + +* **Execution context**: + Jint executes a JavaScript function on a single document at a time, with each execution running in isolation. + Its processing context is limited to a single document, with no persistent execution state - + even in patch operations, where it might appear to maintain continuity. + +* **Performance considerations**: + Since initializing the Jint engine is resource-intensive, + RavenDB caches Jint instances based on user-defined scripts to reuse them and enhance performance. + +* **Execution limitations**: + * RavenDB limits the amount of statements that can be performed for each document processing. + The default value is **10,000** and it can be set using the [Patching.MaxStepsForScript](../../server/configuration/patching-configuration#patching.maxstepsforscript) configuration. + * RavenDB limits the amount of cached Jint engines. + The default value is **2,048** and it can be set using the [Patching.MaxNumberOfCachedScripts](../../server/configuration/patching-configuration#patching.maxstepsforscript) configuration. + * Recursive calls within scripts are limited to a depth of **64**, a constant value that cannot be modified. + +{PANEL/} + +{PANEL: Predefined JavaScript functions} + +In addition to Jint's ECMAScript 5.1 implementation, +RavenDB provides the following set of predefined functions: + +--- + +#### **Document operations**: + +| Method Signature | Return type | Description | +|-------------------------------------------------------------|--------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| **id(document)** | `string` | Returns the ID of the specified document[[ex]](../../client-api/operations/patching/set-based#updating-by-document-id). | +| **load(documentId)** | `object` | Returns the document with the given ID.
Used in [patching](../../client-api/operations/patching/single-document#loading-documents-in-a-script) or [ETL scripts](../../server/ongoing-tasks/etl/basics#transform). | +| **load(documentId, collectionName)** | `object` | Returns the document with the given ID.
Used in [JavaScript indexes](../../indexes/javascript-indexes). | +| **loadPath(document, pathString)** | `object` | Returns document(s) based on IDs found within the specified `pathString` in the given document.
The `pathString` can be in a simple _Foo.Bar_ form, in which case a single document is returned. A path like _Foo.Bars[].Buzz_ can return an array of documents. | +| **getMetadata(document)** | `object` | Returns the metadata of the specified document, including properties like `ChangeVector`, `ID`, and `LastModified`. | +| **lastModified(document)** | `number` | Returns the number of milliseconds elapsed since the last modification time (UTC) of the specified document. | +| **include(documentId)** | `Task` | Used in RQL [queries](../../client-api/session/querying/what-is-rql) to include the document with the specified ID with the results. | +| **put(documentId, document, [optional]changeVectorString)** | `Task` | Creates or updates a document with the specified ID.
To generate a new document ID, you can use the _"collectionPrefix/[Server-Side ID](../../server/kb/document-identifier-generation#strategy--2)"_ notation[[ex]](../../client-api/operations/patching/single-document#add-document).
This function can also clone an existing document.
Note: attachments & counters will not be included in the clone[[ex]](../../client-api/operations/patching/single-document#clone-document). Used in patching. | +| **del(documentId)** | `void` | Deletes the document with the specified ID.
Used in [patching](../../client-api/operations/patching/set-based#updating-a-collection-name). | +| **archived.archiveAt(document, dateString)** | `void` | Schedules the specified document to be archived at the specified `dateString`.
Used in [patching](../../data-archival/schedule-document-archiving#schedule-multiple-documents-for-archiving---from-the-client-api). | +| **archived.unarchive(document)** | `void` | Unarchives the specified document.
Used in [patching](../../data-archival/unarchiving-documents#unarchive-all-documents-in-a-collection---from-the-client-api). | + +--- + +#### **Counter operations**: + +| Method Signature | Return type | Description | +|------------------------------------------------------|--------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| **counter(documentId, counterName)** | `number` | Returns the value of the specified counter for the given document ID[[ex]](../../client-api/operations/patching/single-document#get-counter). | +| **counter(document, counterName)** | `number` | Returns the value of the specified counter for the given document[[ex]](../../client-api/operations/patching/single-document#get-counter). | +| **incrementCounter(documentId, counterName, value)** | `void` | Increments the specified counter for the given document ID.
If the counter does not exist, it is implicitly created with the provided `value`. Counter values can be negative, allowing both increment and decrement operations[[ex]](../../client-api/operations/patching/single-document#increment-counter). | +| **incrementCounter(document, counterName, value)** | `void` | Increments the specified counter for the given document.
If the counter does not exist, it is implicitly created with the provided `value`. Counter values can be negative, allowing both increment and decrement operations[[ex]](../../client-api/operations/patching/single-document#increment-counter). | +| **deleteCounter(documentId, counterName)** | `void` | Delete the specified counter from the given document ID[[ex]](../../client-api/operations/patching/single-document#delete-counter). | +| **deleteCounter(document, counterName)** | `void` | Delete the specified counter from the given document[[ex]](../../client-api/operations/patching/single-document#delete-counter). | +| **counterRaw(documentId, counterName)** | `object` | Returns a dictionary containing the counter value for each database node. The overall counter value is the sum of all node values. | +| **counterRaw(document, counterName)** | `object` | Returns a dictionary containing the counter value for each database node. The overall counter value is the sum of all node values. | + +--- + +#### **Compare-exchange**: + +| Method Signature | Return type | Description | +|---------------------------------|--------------|-------------------------------------------------------------------------------------------------------------------------------------| +| **cmpxchg(compareExchangeKey)** | `object` | Returns the value stored in a [Compare Exchange](../../client-api/operations/compare-exchange/overview) item for the specified key. | + +--- + +#### **String manipulation**: + +| Method Signature | Return type | Description | +|---------------------------------------------------------|--------------|----------------------------------------------------------------------------------------------------------------------------------------| +| **String.prototype.startsWith(searchString, position)** | `boolean` | Returns _true_ if the specified string starts with `searchString` at the given `position`. `position` is optional and defaults to `0`. | +| **String.prototype.endsWith(searchString, position)** | `boolean` | Returns _true_ if the specified string end with `searchString` at the given `position`. `position` is optional and defaults to `0`. | +| **String.prototype.padStart(targetLength, padString)** | `string` | Pads the string from the start with `padString`
(or whitespace by default) until it reaches `targetLength`. | +| **String.prototype.padEnd(targetLength, padString)** | `string` | Pads the string from the end with `padString`
(or whitespace by default) until it reaches `targetLength`. | +| **String.prototype.format(arg1, arg2, arg3 ...)** | `string` | Formats the string by replacing occurrences of `{[number]}` with the corresponding argument based on a zero-based index. | +| **startsWith(inputString, prefix)** | `boolean` | Returns _true_ if `inputString` starts with the specified `prefix`. | +| **endsWith(inputString, suffix)** | `boolean` | Returns _true_ if `inputString` ends with the specified `suffix`. | +| **regex(inputString, regex)** | `boolean` | Returns _true_ if `inputString` matches the specified `regex` pattern. | + +--- + +#### **Arrays & objects**: + +| Method Signature | Return type | Description | +|------------------------------------------------------|-----------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| **Array.prototype.find(function callback)** | Array's element | Returns the first element in the array for which the `callback` function returns _true_. | +| **Object.map(input, function mapFunction, context)** | `Array` | Returns an array containing the results of `mapFunction` applied to all properties of `input` (or items, if input is an array). The `mapFunction` signature is `function(itemValue, itemKey)`. | + +--- + +#### **Mathematical operations**: + +| Method Signature | Return type | Description | +|---------------------------|--------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| **Raven_Min(num1, num2)** | `number` | Returns the smaller of `num1` and `num2`. If both params are of the same type (both numbers or both strings), a standard comparison is performed.
If they are of mixed types (one number and one string), the string is parsed as a double for comparison.
`LazyNumberValue` params resulting from method `scalarToRawString` are not supported. | +| **Raven_Max(num1, num2)** | `number` | Returns the larger of `num1` and `num2`. If both params are of the same type (both numbers or both strings), a standard comparison is performed.
If they are of mixed types (one number and one string), the string is parsed as a double for comparison.
`LazyNumberValue` params resulting from method `scalarToRawString` are not supported. | + +--- + +#### **Conversion operations**: + +| Method Signature | Return type | Description | +|------------------------------------------------|-----------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| **scalarToRawString(document, lambdaToField)** | Raw field value.
`LazyStringValue` for strings,
`LazyNumberValue` for floating point numbers. | Returns the raw representation of a field. Useful for handling numbers that exceed the numeric or accuracy range of `double` (See [Numbers in Jint](../../server/kb/numbers-in-ravendb#numbers-in-javascript-engine)), or for optimizing memory consumption when projecting large string values.
The returned value is immutable. | +| **convertJsTimeToTimeSpanString(ticksNumber)** | `string` | Returns a human-readable `TimeSpan` representation of the specified `ticksNumber`. | + + +--- + +#### **Debugging**: + +| Method Signature | Return | Description | +|-------------------------------------------------|----------|--------------------------------------------------------------------------------------------------------------------------------------------| +| **output(message)** or **console.log(message)** | `void` | Prints message to the debug output.
Used for debugging [single document patches](../../client-api/operations/patching/single-document). | + +{PANEL/} + +## Related Articles + +### Patching + +- [How to Perform Single Document Patch Operations](../../client-api/operations/patching/single-document) +- [How to Perform Set Based Operations on Documents](../../client-api/operations/patching/set-based) + +### Querying + +- [Projections](../../indexes/querying/projections) + +### Knowledge Base + +- [Numbers in RavenDB](../../server/kb/numbers-in-ravendb) diff --git a/Documentation/6.2/Raven.Documentation.Pages/.docs.json b/Documentation/6.2/Raven.Documentation.Pages/.docs.json index 0f85c81b6f..9fdec8a1fc 100644 --- a/Documentation/6.2/Raven.Documentation.Pages/.docs.json +++ b/Documentation/6.2/Raven.Documentation.Pages/.docs.json @@ -19,6 +19,11 @@ "Name": "Document Extensions", "Mappings": [] }, + { + "Path": "/data-archival", + "Name": "Data Archival", + "Mappings": [] + }, { "Path": "/server", "Name": "Server", diff --git a/Documentation/6.2/Raven.Documentation.Pages/client-api/operations/server-wide/create-database.dotnet.markdown b/Documentation/6.2/Raven.Documentation.Pages/client-api/operations/server-wide/create-database.dotnet.markdown index d923a88b92..688dacbdab 100644 --- a/Documentation/6.2/Raven.Documentation.Pages/client-api/operations/server-wide/create-database.dotnet.markdown +++ b/Documentation/6.2/Raven.Documentation.Pages/client-api/operations/server-wide/create-database.dotnet.markdown @@ -108,7 +108,7 @@ The `DatabaseRecord` is a collection of database configurations: | **AutoIndexes** | `Dictionary` | Auto-index definitions for the database. | | **Client** | `ClientConfiguration` | [Client behavior](../../../studio/server/client-configuration) configuration. | | **ConflictSolverConfig** | `ConflictSolver` | Define the strategy used to resolve [Replication conflicts](../../../server/clustering/replication/replication-conflicts). | -| **DataArchival** | `DataArchivalConfiguration` | [Data Archival](../../../server/extensions/archival) configuration for the database. | +| **DataArchival** | `DataArchivalConfiguration` | [Data Archival](../../../data-archival/overview) configuration for the database. | | **DatabaseName** | `string` | The database name. | | **Disabled** | `bool` | Set the database initial state.
`true` - disable the database.
`false` - (default) the database will be enabled.

This can be modified later via [ToggleDatabasesStateOperation](../../../client-api/operations/server-wide/toggle-databases-state). | | **DocumentsCompression** | `DocumentsCompressionConfiguration` | Configuration settings for [Compressing documents](../../../server/storage/documents-compression). | diff --git a/Documentation/6.2/Raven.Documentation.Pages/client-api/operations/server-wide/create-database.java.markdown b/Documentation/6.2/Raven.Documentation.Pages/client-api/operations/server-wide/create-database.java.markdown index 130011c8fd..a3ad1346b4 100644 --- a/Documentation/6.2/Raven.Documentation.Pages/client-api/operations/server-wide/create-database.java.markdown +++ b/Documentation/6.2/Raven.Documentation.Pages/client-api/operations/server-wide/create-database.java.markdown @@ -76,7 +76,7 @@ | **autoIndexes** | `Map` | Auto-index definitions for the database. | | **client** | `ClientConfiguration` | [Client behavior](../../../studio/server/client-configuration) configuration. | | **conflictSolverConfig** | `ConflictSolver` | Define the strategy used to resolve [Replication conflicts](../../../server/clustering/replication/replication-conflicts). | -| **dataArchival** | `DataArchivalConfiguration` | [Data Archival](../../../server/extensions/archival) configuration for the database. | +| **dataArchival** | `DataArchivalConfiguration` | [Data Archival](../../../data-archival/overview) configuration for the database. | | **databaseName** | `String` | The database name. | | **disabled** | `boolean` (default: false) | Set the database initial state.
`true` - disable the database.
`false` - (default) the database will be enabled.

This can be modified later via [ToggleDatabasesStateOperation](../../../client-api/operations/server-wide/toggle-databases-state). | | **documentsCompression** | `DocumentsCompressionConfiguration` | Configuration settings for [Compressing documents](../../../server/storage/documents-compression). | diff --git a/Documentation/6.2/Raven.Documentation.Pages/client-api/operations/what-are-operations.dotnet.markdown b/Documentation/6.2/Raven.Documentation.Pages/client-api/operations/what-are-operations.dotnet.markdown index 87d176bb3c..3a82ead25a 100644 --- a/Documentation/6.2/Raven.Documentation.Pages/client-api/operations/what-are-operations.dotnet.markdown +++ b/Documentation/6.2/Raven.Documentation.Pages/client-api/operations/what-are-operations.dotnet.markdown @@ -270,6 +270,7 @@ * **Misc**:        ConfigureExpirationOperation        ConfigureRefreshOperation +        [ConfigureDataArchivalOperation](../../data-archival/enable-data-archiving#enable-archiving---from-the-client-api)        UpdateDocumentsCompressionConfigurationOperation        DatabaseHealthCheckOperation        GetOperationStateOperation diff --git a/Documentation/6.2/Raven.Documentation.Pages/client-api/operations/what-are-operations.js.markdown b/Documentation/6.2/Raven.Documentation.Pages/client-api/operations/what-are-operations.js.markdown index 5c9e29cd07..6f25e7770e 100644 --- a/Documentation/6.2/Raven.Documentation.Pages/client-api/operations/what-are-operations.js.markdown +++ b/Documentation/6.2/Raven.Documentation.Pages/client-api/operations/what-are-operations.js.markdown @@ -271,6 +271,7 @@ __Send syntax__: * __Misc__:        ConfigureExpirationOperation        ConfigureRefreshOperation +        ConfigureDataArchivalOperation        UpdateDocumentsCompressionConfigurationOperation        DatabaseHealthCheckOperation        GetOperationStateOperation diff --git a/Documentation/6.2/Raven.Documentation.Pages/client-api/operations/what-are-operations.php.markdown b/Documentation/6.2/Raven.Documentation.Pages/client-api/operations/what-are-operations.php.markdown index a9c759f0f2..1621edb065 100644 --- a/Documentation/6.2/Raven.Documentation.Pages/client-api/operations/what-are-operations.php.markdown +++ b/Documentation/6.2/Raven.Documentation.Pages/client-api/operations/what-are-operations.php.markdown @@ -256,6 +256,7 @@ * **Misc**:        ConfigureExpirationOperation        ConfigureRefreshOperation +        ConfigureDataArchivalOperation        UpdateDocumentsCompressionConfigurationOperation        DatabaseHealthCheckOperation        GetOperationStateOperation diff --git a/Documentation/6.2/Raven.Documentation.Pages/client-api/operations/what-are-operations.python.markdown b/Documentation/6.2/Raven.Documentation.Pages/client-api/operations/what-are-operations.python.markdown index a957a40807..eb978d1bf2 100644 --- a/Documentation/6.2/Raven.Documentation.Pages/client-api/operations/what-are-operations.python.markdown +++ b/Documentation/6.2/Raven.Documentation.Pages/client-api/operations/what-are-operations.python.markdown @@ -257,6 +257,7 @@ * **Misc**:        ConfigureExpirationOperation        ConfigureRefreshOperation +        ConfigureDataArchivalOperation        UpdateDocumentsCompressionConfigurationOperation        DatabaseHealthCheckOperation        GetOperationStateOperation diff --git a/Documentation/6.2/Raven.Documentation.Pages/data-archival/.docs.json b/Documentation/6.2/Raven.Documentation.Pages/data-archival/.docs.json new file mode 100644 index 0000000000..8e3012a214 --- /dev/null +++ b/Documentation/6.2/Raven.Documentation.Pages/data-archival/.docs.json @@ -0,0 +1,57 @@ +[ + { + "Path": "overview.markdown", + "Name": "Overview", + "DiscussionId": "28bb68b0-3af7-46b3-ad4c-e34b7e81565d", + "Mappings": [ + { + "Version": 6.0, + "Key": "server/extensions/archival" + } + ] + }, + { + "Path": "enable-data-archiving.markdown", + "Name": "Enable Data Archiving", + "DiscussionId": "e5eeaaa7-bb98-4ae7-9cd1-75cb300738e7", + "Mappings": [ + { + "Version": 6.0, + "Key": "server/extensions/archival" + } + ] + }, + { + "Path": "schedule-document-archiving.markdown", + "Name": "Schedule Document Archiving", + "DiscussionId": "ec35fa9a-834a-48b0-96c3-d67e1648cbae", + "Mappings": [ + { + "Version": 6.0, + "Key": "server/extensions/archival" + } + ] + }, + { + "Path": "archived-documents-and-other-features.markdown", + "Name": "Archived Documents and Other Features", + "DiscussionId": "977c8625-ad1d-4dd9-8b29-eb92037957f8", + "Mappings": [ + { + "Version": 6.0, + "Key": "server/extensions/archival" + } + ] + }, + { + "Path": "unarchiving-documents.markdown", + "Name": "Unarchiving Documents", + "DiscussionId": "a01cdd59-257e-4a0e-8ec6-07d7d7d3d239", + "Mappings": [ + { + "Version": 6.0, + "Key": "server/extensions/archival" + } + ] + } +] diff --git a/Documentation/6.2/Raven.Documentation.Pages/data-archival/archived-documents-and-other-features.dotnet.markdown b/Documentation/6.2/Raven.Documentation.Pages/data-archival/archived-documents-and-other-features.dotnet.markdown new file mode 100644 index 0000000000..fd4a151ae9 --- /dev/null +++ b/Documentation/6.2/Raven.Documentation.Pages/data-archival/archived-documents-and-other-features.dotnet.markdown @@ -0,0 +1,389 @@ +# Archived Documents and Other Features +--- + +{NOTE: } + +* Once you have archived documents in your database (see how to [enable](../data-archival/enable-data-archiving) and [schedule](../data-archival/schedule-document-archiving) document archiving), + RavenDB features can detect these documents and handle them in different ways. + +* Some features, like indexes and data subscriptions, provide native support for configuring whether to: + * **Exclude** archived documents from processing, reducing index size and improving query relevance. + * **Include** only archived documents, for tasks that target archived data specifically. + * **Process both** archived and non-archived documents when needed. + +* Other features can manage archived documents differently based on their purpose. For example: + * ETL tasks can skip or selectively process archived documents. + * Archived documents can be included or excluded when exporting or importing data. + +* Limiting processing to either archived or non-archived documents may improve performance by reducing workload and transfer volume. + +* Learn more below about how various RavenDB features interact with archived documents. + +--- + +* In this article: + * [Archived documents and indexing](../data-archival/archived-documents-and-other-features#archived-documents-and-indexing) + * [Archived documents and querying](../data-archival/archived-documents-and-other-features#archived-documents-and-querying) + * [Archived documents and data subscriptions](../data-archival/archived-documents-and-other-features#archived-documents-and-subscriptions) + * [Archived documents and document extensions](../data-archival/archived-documents-and-other-features#archived-documents-and-document-extensions) + * [Archived documents and smuggler (export/import)](../data-archival/archived-documents-and-other-features#archived-documents-and-smuggler-(export/import)) + * [Archived documents and expiration](../data-archival/archived-documents-and-other-features#archived-documents-and-expiration) + * [Archived documents and ETL](../data-archival/archived-documents-and-other-features#archived-documents-and-etl) + * [Archived documents and backup](../data-archival/archived-documents-and-other-features#archived-documents-and-backup) + * [Archived documents and replication](../data-archival/archived-documents-and-other-features#archived-documents-and-replication) + * [Archived documents and patching](../data-archival/archived-documents-and-other-features#archived-documents-and-patching) + +{NOTE/} + +--- + +{PANEL: Archived documents and indexing} + +* Indexing performance may decline as the database grows, since a larger number of documents increases indexing load, expands index size, and can eventually reduce query speed. +* Archiving documents and excluding them from indexing can be an effective way to maintain performance. + By removing low-priority or infrequently accessed documents from the indexing process, RavenDB can create smaller, faster indexes focused on current or high-value data. + This also improves the relevance and responsiveness of queries, as they execute over a smaller and more meaningful dataset. + +--- + +* **Configuring indexing behavior - Static indexes**: + * **At the database level or server-wide**: + To control whether static indexes process archived documents from the source collection, + set the [Indexing.Static.ArchivedDataProcessingBehavior](../server/configuration/indexing-configuration#indexing.static.archiveddataprocessingbehavior) + configuration key at either the database level or server-wide (default: `ExcludeArchived`). + * Note that this setting applies only to static-indexes that are using _Documents_ as their data source. + This global configuration does Not apply to static-indexes based on _Time Series_ or _Counters_, which default to `IncludeArchived`. + * **Per index**: + You can override this global behavior per-index directly in the index definition, using the Client API from the Studio + (see the examples below). + +* **Configuring indexing behavior - Auto indexes:** + * **At the database level or server-wide**: + To control whether auto-indexes process archived documents at the database level or server-wide, + set the [Indexing.Auto.ArchivedDataProcessingBehavior](../server/configuration/indexing-configuration#indexing.auto.archiveddataprocessingbehavior) configuration key (default `ExcludeArchived`). + * **Per index**: + Unlike static indexes, you cannot configure this behavior per auto-index, + because dynamic queries (which trigger auto-index creation) do not provide a way to control this setting. + +--- + +* The available configuration options are: + * `ExcludeArchived`: only non-archived documents are processed by the index. + * `IncludeArchived`: both archived and non-archived documents are processed by the index. + * `ArchivedOnly`: only archived documents are processed by the index. + +--- + +##### Configuring archived document processing for a static index - from the Client API + +You can configure how a static index handles archived documents when creating the index using the Client API. +This setting will **override** the global configuration defined by the [Indexing.Static.ArchivedDataProcessingBehavior](../server/configuration/indexing-configuration#indexing.static.archiveddataprocessingbehavior) configuration key. + +{CONTENT-FRAME: } + +Example: + + {CODE-TABS} + {CODE-TAB:csharp:LINQ_index index_1@DataArchival\ArchivedDocsIntegration.cs /} + {CODE-TAB:csharp:JS_index index_2@DataArchival\ArchivedDocsIntegration.cs /} + {CODE-TAB:csharp:IndexDefinitionBuilder index_3@DataArchival\ArchivedDocsIntegration.cs /} + {CODE-TABS/} + +{CONTENT-FRAME/} +{CONTENT-FRAME: } + +When a static-index is configured to include **both** archived and non-archived documents in its processing, +you can also apply custom logic based on the presence of the `@archived` metadata property. + +For example: + + {CODE-TABS} + {CODE-TAB:csharp:LINQ_index index_4@DataArchival\ArchivedDocsIntegration.cs /} + {CODE-TAB:csharp:JS_index index_5@DataArchival\ArchivedDocsIntegration.cs /} + {CODE-TAB:csharp:IndexDefinition index_6@DataArchival\ArchivedDocsIntegration.cs /} + {CODE-TABS/} + +{CONTENT-FRAME/} + +--- + +##### Configuring archived document processing for a static index - from the Studio + +You can configure how a static index handles archived documents directly from the Studio. +This setting will **override** the global configuration defined by the [Indexing.Static.ArchivedDataProcessingBehavior](../server/configuration/indexing-configuration#indexing.static.archiveddataprocessingbehavior) configuration key. + +![Configure index](images/configure-static-index.png) + +1. Open the [Indexes list view](../studio/database/indexes/indexes-list-view) and select the index you want to configure, + or create a new index. +2. Scroll down and open the **Archived Data** tab. +3. Click to select how this index should process archived documents: + * **Default**: The index will use the behavior set by the global configuration. + * **Exclude Archived**: Index only non-archived documents. + * **Include Archived**: Index both archived and non-archived documents. + * **Archived Only**: Index only archived documents. + +![Processing options](images/processing-options.png "Configure the static index") + +{PANEL/} + +{PANEL: Archived documents and querying} + +* **Full collection queries**: + * Queries that scan an entire collection without any filtering condition (e.g. `from Orders`) will include archived documents. + * These queries are not influenced by indexing configuration related to archived documents because they do not use indexes. + * Learn more about full collection queries in [Full collection query](../client-api/session/querying/how-to-query#collectionQuery). + +* **Dynamic queries (auto-indexes)**: + * When making a dynamic query, RavenDB creates an auto-index to serve it. + Whether that index processes archived documents depends on the value of the [Indexing.Auto.ArchivedDataProcessingBehavior](../server/configuration/indexing-configuration#indexing.auto.archiveddataprocessingbehavior) configuration key at the time the query is made. + * Once created, the auto-index retains that behavior. + Query results will continue to reflect the configuration that was in effect when the index was first built - even if the setting is changed later. + * Learn more about dynamic queries in [Query a collection - with filtering](../client-api/session/querying/how-to-query#dynamicQuery). + +* **Querying static-indexes**: + * When querying a static-index, the results will include, exclude, or consist solely of archived documents depending on how the static-index was configured. + The index behavior is determined by: + * the value of the [Indexing.Static.ArchivedDataProcessingBehavior](../server/configuration/indexing-configuration#indexing.static.archiveddataprocessingbehavior) configuration key at the time the static-index was created, or - + * the explicit setting in the index definition, which overrides the global configuration key. + * The index's archived data processing behavior can be modified after its creation using the Studio or the Client API. + +{PANEL/} + +{PANEL: Archived documents and subscriptions} + +* Processing large volumes of documents in data subscriptions increases the workload on both the server and subscription workers. +* You can reduce this load by defining the subscription query to exclude archived documents, include only archived documents, or process both archived and non-archived data. + This gives you control over which documents are sent to workers - helping you focus on the most relevant data and reduce unnecessary processing. + +--- + +* **Configuring the subscription task behavior**: + * **At the database level or server-wide**: + To control whether queries in data subscription tasks process archived documents, + set the [Subscriptions.ArchivedDataProcessingBehavior](../todo..) configuration key at either the database level or server-wide + (default: `ExcludeArchived`). + * **Per task**: + You can override this global behavior per data subscription task directly in the task definition, + using the Client API or from the Studio (see the examples below). + +--- + +* The available configuration options are: + * `ExcludeArchived`: only non-archived documents are processed by the subscription query. + * `IncludeArchived`: both archived and non-archived documents are processed by the subscription query. + * `ArchivedOnly`: only archived documents are processed by the subscription query. + +--- + +##### Configuring archived document processing for a data subscription task - from the Client API + +You can configure how a subscription task handles archived documents when creating the subscription using the Client API. +This setting will **override** the global configuration defined by the [Subscriptions.ArchivedDataProcessingBehavior](../server/configuration/subscription-configuration#subscriptions.archiveddataprocessingbehavior) configuration key. + +{CONTENT-FRAME: } + +Example: + +{CODE-TABS} +{CODE-TAB:csharp:Generic-syntax subscription_task_generic_syntax@DataArchival\ArchivedDocsIntegration.cs /} +{CODE-TAB:csharp:RQL-syntax subscription_task_rql_syntax@DataArchival\ArchivedDocsIntegration.cs /} +{CODE-TABS/} + +{CONTENT-FRAME/} + +--- + +##### Configuring archived document processing for a data subscription task - from the Studio + +You can configure how a subscription task handles archived documents directly from the Studio. +This setting will **override** the global configuration defined by the [Subscriptions.ArchivedDataProcessingBehavior](../server/configuration/subscription-configuration#subscriptions.archiveddataprocessingbehavior) configuration key. + +![Configure subscription](images/configure-subscription.png) + +1. Open the [Ongoing tasks list view](../studio/database/tasks/ongoing-tasks/general-info) and select the subscription task you want to configure, + or create a new subscription. +2. Click to select how the subscription query should process archived documents: + * **Default**: The subscription will use the behavior set by the global configuration. + * **Exclude Archived**: Process only non-archived documents. + * **Include Archived**: Process both archived and non-archived documents. + * **Archived Only**: Process only archived documents. + +{PANEL/} + +{PANEL: Archived documents and document extensions} + +* **Attachments**: + * Attachments are Not archived (compressed), even if the document they belong to is archived. + +* **Counters**: + * Counters are Not archived (compressed), even if the document they belong to is archived. + * Unlike indexes whose source data is _Documents_ - which default to `ExcludeArchived` - + indexes whose source data is _Counters_ do process archived documents by default (`IncludeArchived`). + This behavior can be modified in the index definition. + +* **Time series**: + * Time series are Not archived (compressed), even if the document they belong to is archived. + * Unlike indexes whose source data is _Documents_ - which default to `ExcludeArchived` - + indexes whose source data is _Time series_ do process archived documents by default (`IncludeArchived`). + This behavior can be modified in the index definition. + +* **Revisions**: + * No revision is created at the time the server archives a document, even if the Revisions feature is enabled. + * However, if you modify an archived document (when Revisions are enabled), a revision is created for that document - and that revision is archived as well. + +{PANEL/} + +{PANEL: Archived documents and smuggler (export/import)} + +You can control whether archived documents are included when exporting or importing a database. + +--- + +##### Export/Import archived documents - from the Client API + +[Smuggler](../client-api/smuggler/what-is-smuggler), RavenDB’s tool for database export and import, can be configured to include or exclude archived documents. +By default, archived documents are **included** in the operation. + +{CONTENT-FRAME: } + +In this example, exported data **excludes** archived documents: + +{CODE export@DataArchival\ArchivedDocsIntegration.cs /} + +{CONTENT-FRAME/} + +{CONTENT-FRAME: } + +In this example, imported data **includes** archived documents: + +{CODE import@DataArchival\ArchivedDocsIntegration.cs /} + +{CONTENT-FRAME/} + +--- + +##### Export archived documents - from the Studio + +![Export archived documents](images/export-archived-documents.png "Export archived documents") + +1. Go to **Tasks > Export Database**. +2. Toggle the **Include archived documents** option to control whether archived documents are included in the database export. + +--- + +##### Import archived documents - from the Studio + +![Import archived documents](images/import-archived-documents.png "Import archived documents") + +1. Go to **Tasks > Import Data**. +2. Toggle the **Include archived documents** option to control whether archived documents are included in the import. + +{PANEL/} + +{PANEL: Archived documents and expiration} + +* Archiving can be used alongside other features, such as [Document expiration](../server/extensions/expiration). + +* For example, a document can be scheduled to be archived after six months and expired after one year. + This allows you to keep recent documents active and quickly accessible, move older documents to archival storage where slower access is acceptable, + and eventually remove documents that are no longer needed. + +* In the following example, both the `@archive-at` and the `@expires` metadata properties have been added to document `companies/90-A` + to schedule it for archiving and expiration, respectively: + + {CODE-BLOCK:json} +{ + "Name": "Wilman Kala", + "Phone": "90-224 8858", + ... + "@metadata": { + "@archive-at": "2026-01-06T22:45:30.018Z", + "@expires": "2026-07-06T22:45:30.018Z", + "@collection": "Companies", + ... + } +} + {CODE-BLOCK/} + +{PANEL/} + +{PANEL: Archived documents and ETL} + +* An ETL transformation script can examine each source document’s [metadata](../server/ongoing-tasks/etl/raven#accessing-metadata) + for the existence of the `@archived: true` property, which indicates that the document is archived. + Based on this check, the script can decide how to handle the document - for example, skip it entirely or send only selected fields. + +* With [RavenDB ETL](../../../server/ongoing-tasks/etl/raven), documents that are archived in the source database and sent to the target + are not archived in the destination database. + +* In the following example, the ETL script checks whether the document is archived, and skips it if it is: + + {CODE-BLOCK: JavaScript} +var isArchived = this['@metadata']['@archived']; + +if (isArchived === true) { + return; // Do not process archived documents +} + +// Transfer only non-archived documents to the target +loadToOrders(this); + {CODE-BLOCK/} + +{PANEL/} + +{PANEL: Archived documents and backup} + +* Archived documents are included in database backups (both _logical backups_ and _snapshots_), + no special configuration is required. + +* When restoring a database from a backup, archived documents are restored as well, + and their archived status is preserved. + +{PANEL/} + +{PANEL: Archived documents and replication} + +Archived documents are included in [Internal](../server/clustering/replication/replication#internal-replication) replication, +[External](../server/clustering/replication/replication#external-replication) replication, and [Hub/Sink](../server/clustering/replication/replication#hubsink-replication) replication - +no special configuration is required. + +{PANEL/} + +{PANEL: Archived documents and patching} + +* Patching can be used to **schedule** multiple documents for archiving. See the dedicated sections: + [Schedule multiple documents for archiving - from the Studio](../data-archival/schedule-document-archiving#schedule-multiple-documents-for-archiving---from-the-studio). + [Schedule multiple documents for archiving - from the Client API](../data-archival/schedule-document-archiving#schedule-multiple-documents-for-archiving---from-the-client-api). + +* Patching is used to **unarchive** documents. + See the dedicated article [Unarchiving documents](../data-archival/unarchiving-documents). + +* When **cloning** an archived document using the `put` method within a patching script + (see method details in this [Document operations](../server/kb/javascript-engine#document-operations) section) the cloned document will Not be archived, + and the `@archived: true` property will be removed from the cloned document. + +{PANEL/} + +## Related Articles + +### Document Archival +- [Overview](../data-archival/overview) +- [Enable data archiving](../data-archival/enable-data-archiving) +- [Schedule document archiving](../data-archival/schedule-document-archiving) +- [Unarchiving documents](../data-archival/unarchiving-documents) + +### Configuration +- [Overview](../server/configuration/configuration-options#settings.json) +- [Database Settings](../studio/database/settings/database-settings#view-database-settings) + +### Tasks +- [Smuggler (Import/Export)](../client-api/smuggler/what-is-smuggler) +- [ETL Basics](../server/ongoing-tasks/etl/basics) +- [Replication](../../server/clustering/replication/replication) + +### Extensions +- [Document Expiration](../../server/extensions/expiration) + +### Patching +- [Patch By Query](../client-api/rest-api/queries/patch-by-query) diff --git a/Documentation/6.2/Raven.Documentation.Pages/data-archival/enable-data-archiving.dotnet.markdown b/Documentation/6.2/Raven.Documentation.Pages/data-archival/enable-data-archiving.dotnet.markdown new file mode 100644 index 0000000000..c01ebcafb0 --- /dev/null +++ b/Documentation/6.2/Raven.Documentation.Pages/data-archival/enable-data-archiving.dotnet.markdown @@ -0,0 +1,82 @@ +# Enable Data Archiving +--- + +{NOTE: } + +* By default, data archiving is disabled. + To use the archiving feature, you must first **enable** it. + +* When configuring the feature, + you can also set the **frequency** at which RavenDB scans the database for documents scheduled for archiving. + +* Once enabled, the archiving task runs periodically at the configured frequency, + scanning the database for documents that have been scheduled for archival. + Learn how to schedule documents for archival in [Schedule document archiving](../data-archival/schedule-document-archiving). + +* In this article: + * [Enable archiving - from the Client API](../data-archival/enable-data-archiving#enable-archiving---from-the-client-api) + * [Enable archiving - from the Studio](../data-archival/enable-data-archiving#enable-archiving---from-the-studio) + +{NOTE/} + +--- + +{PANEL: Enable archiving - from the Client API} + +Use `ConfigureDataArchivalOperation` to enable archiving for the database and configure the archiving task. + +--- + +**Example**: + +{CODE-TABS} +{CODE-TAB:csharp:Enable_archiving enable_archiving@DataArchival\EnableArchiving.cs /} +{CODE-TAB:csharp:Enable_archiving_async enable_archiving_async@DataArchival\EnableArchiving.cs /} +{CODE-TABS/} + +--- + +**Syntax**: + +{CODE syntax_1@DataArchival\EnableArchiving.cs /} + +{CODE syntax_2@DataArchival\EnableArchiving.cs /} + +| Parameter | Type | Description | +|---------------------------|---------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| **Disabled** | `bool` | `true` - archiving is disabled for the entire database (default).
`false` - archiving is enabled for the database. | +| **ArchiveFrequencyInSec** | `long?` | Frequency (in seconds) at which the server scans for documents scheduled for archiving. Default: `60` | +| **MaxItemsToProcess** | `long?` | The maximum number of documents the archiving task will process in a single run (i.e., each time it is triggered by the configured frequency). Default: `int.MaxValue` | + +{PANEL/} + +{PANEL: Enable archiving - from the Studio} + +![Enable archiving](images/enable-archiving.png "Enable archiving") + +1. Go to **Settings > Data Archival**. +2. Toggle on to enable data archival. +3. Toggle on to customize the frequency at which the server scans for documents scheduled for archiving. + Default is 60 seconds. +4. Toggle on to customize the maximum number of documents the archiving task will process in a single run. +5. Click Save to apply your settings. + +{PANEL/} + +## Related Articles + +### Document Archival +- [Overview](../data-archival/overview) +- [Schedule document archiving](../data-archival/schedule-document-archiving) +- [Archived documents and other features](../data-archival/archived-documents-and-other-features) +- [Unarchiving documents](../data-archival/unarchiving-documents) + +### Configuration +- [Overview](../server/configuration/configuration-options#settings.json) +- [Database Settings](../studio/database/settings/database-settings#view-database-settings) + +### Extensions +- [Document Expiration](../server/extensions/expiration) + +### Patching +- [Patch By Query](../client-api/rest-api/queries/patch-by-query) diff --git a/Documentation/6.2/Raven.Documentation.Pages/data-archival/images/an-archived-document.png b/Documentation/6.2/Raven.Documentation.Pages/data-archival/images/an-archived-document.png new file mode 100644 index 0000000000..f5f9972251 Binary files /dev/null and b/Documentation/6.2/Raven.Documentation.Pages/data-archival/images/an-archived-document.png differ diff --git a/Documentation/6.2/Raven.Documentation.Pages/data-archival/images/archived-document-in-list-view.png b/Documentation/6.2/Raven.Documentation.Pages/data-archival/images/archived-document-in-list-view.png new file mode 100644 index 0000000000..295c8ce9f9 Binary files /dev/null and b/Documentation/6.2/Raven.Documentation.Pages/data-archival/images/archived-document-in-list-view.png differ diff --git a/Documentation/6.2/Raven.Documentation.Pages/data-archival/images/configure-static-index.png b/Documentation/6.2/Raven.Documentation.Pages/data-archival/images/configure-static-index.png new file mode 100644 index 0000000000..8fb74b5f8b Binary files /dev/null and b/Documentation/6.2/Raven.Documentation.Pages/data-archival/images/configure-static-index.png differ diff --git a/Documentation/6.2/Raven.Documentation.Pages/data-archival/images/configure-subscription.png b/Documentation/6.2/Raven.Documentation.Pages/data-archival/images/configure-subscription.png new file mode 100644 index 0000000000..634e994855 Binary files /dev/null and b/Documentation/6.2/Raven.Documentation.Pages/data-archival/images/configure-subscription.png differ diff --git a/Documentation/6.2/Raven.Documentation.Pages/data-archival/images/enable-archiving.png b/Documentation/6.2/Raven.Documentation.Pages/data-archival/images/enable-archiving.png new file mode 100644 index 0000000000..2a74f754d0 Binary files /dev/null and b/Documentation/6.2/Raven.Documentation.Pages/data-archival/images/enable-archiving.png differ diff --git a/Documentation/6.2/Raven.Documentation.Pages/data-archival/images/export-archived-documents.png b/Documentation/6.2/Raven.Documentation.Pages/data-archival/images/export-archived-documents.png new file mode 100644 index 0000000000..02df88c74d Binary files /dev/null and b/Documentation/6.2/Raven.Documentation.Pages/data-archival/images/export-archived-documents.png differ diff --git a/Documentation/6.2/Raven.Documentation.Pages/data-archival/images/import-archived-documents.png b/Documentation/6.2/Raven.Documentation.Pages/data-archival/images/import-archived-documents.png new file mode 100644 index 0000000000..c038e3302a Binary files /dev/null and b/Documentation/6.2/Raven.Documentation.Pages/data-archival/images/import-archived-documents.png differ diff --git a/Documentation/6.2/Raven.Documentation.Pages/data-archival/images/processing-options.png b/Documentation/6.2/Raven.Documentation.Pages/data-archival/images/processing-options.png new file mode 100644 index 0000000000..8092219ff4 Binary files /dev/null and b/Documentation/6.2/Raven.Documentation.Pages/data-archival/images/processing-options.png differ diff --git a/Documentation/6.2/Raven.Documentation.Pages/data-archival/images/schedule-document-for-archiving.png b/Documentation/6.2/Raven.Documentation.Pages/data-archival/images/schedule-document-for-archiving.png new file mode 100644 index 0000000000..75cd0879ae Binary files /dev/null and b/Documentation/6.2/Raven.Documentation.Pages/data-archival/images/schedule-document-for-archiving.png differ diff --git a/Documentation/6.2/Raven.Documentation.Pages/data-archival/images/schedule-multiple-documents-for-archiving.png b/Documentation/6.2/Raven.Documentation.Pages/data-archival/images/schedule-multiple-documents-for-archiving.png new file mode 100644 index 0000000000..3a00911f82 Binary files /dev/null and b/Documentation/6.2/Raven.Documentation.Pages/data-archival/images/schedule-multiple-documents-for-archiving.png differ diff --git a/Documentation/6.2/Raven.Documentation.Pages/data-archival/images/snag/an-archived-document.snagx b/Documentation/6.2/Raven.Documentation.Pages/data-archival/images/snag/an-archived-document.snagx new file mode 100644 index 0000000000..7fdc420ca0 Binary files /dev/null and b/Documentation/6.2/Raven.Documentation.Pages/data-archival/images/snag/an-archived-document.snagx differ diff --git a/Documentation/6.2/Raven.Documentation.Pages/data-archival/images/snag/archived-document-in-list-view.snagx b/Documentation/6.2/Raven.Documentation.Pages/data-archival/images/snag/archived-document-in-list-view.snagx new file mode 100644 index 0000000000..39f206abca Binary files /dev/null and b/Documentation/6.2/Raven.Documentation.Pages/data-archival/images/snag/archived-document-in-list-view.snagx differ diff --git a/Documentation/6.2/Raven.Documentation.Pages/data-archival/images/snag/configure-static-index.snagx b/Documentation/6.2/Raven.Documentation.Pages/data-archival/images/snag/configure-static-index.snagx new file mode 100644 index 0000000000..9cd0ffe0d7 Binary files /dev/null and b/Documentation/6.2/Raven.Documentation.Pages/data-archival/images/snag/configure-static-index.snagx differ diff --git a/Documentation/6.2/Raven.Documentation.Pages/data-archival/images/snag/configure-subscription.snagx b/Documentation/6.2/Raven.Documentation.Pages/data-archival/images/snag/configure-subscription.snagx new file mode 100644 index 0000000000..38a9ad8506 Binary files /dev/null and b/Documentation/6.2/Raven.Documentation.Pages/data-archival/images/snag/configure-subscription.snagx differ diff --git a/Documentation/6.2/Raven.Documentation.Pages/data-archival/images/snag/enable-archiving.snagx b/Documentation/6.2/Raven.Documentation.Pages/data-archival/images/snag/enable-archiving.snagx new file mode 100644 index 0000000000..35b41541d7 Binary files /dev/null and b/Documentation/6.2/Raven.Documentation.Pages/data-archival/images/snag/enable-archiving.snagx differ diff --git a/Documentation/6.2/Raven.Documentation.Pages/data-archival/images/snag/export-archived-documents.snagx b/Documentation/6.2/Raven.Documentation.Pages/data-archival/images/snag/export-archived-documents.snagx new file mode 100644 index 0000000000..588823b852 Binary files /dev/null and b/Documentation/6.2/Raven.Documentation.Pages/data-archival/images/snag/export-archived-documents.snagx differ diff --git a/Documentation/6.2/Raven.Documentation.Pages/data-archival/images/snag/import-archived-documents.snagx b/Documentation/6.2/Raven.Documentation.Pages/data-archival/images/snag/import-archived-documents.snagx new file mode 100644 index 0000000000..0c2b28d819 Binary files /dev/null and b/Documentation/6.2/Raven.Documentation.Pages/data-archival/images/snag/import-archived-documents.snagx differ diff --git a/Documentation/6.2/Raven.Documentation.Pages/data-archival/images/snag/schedule-document-for-archiving.snagx b/Documentation/6.2/Raven.Documentation.Pages/data-archival/images/snag/schedule-document-for-archiving.snagx new file mode 100644 index 0000000000..89903d768a Binary files /dev/null and b/Documentation/6.2/Raven.Documentation.Pages/data-archival/images/snag/schedule-document-for-archiving.snagx differ diff --git a/Documentation/6.2/Raven.Documentation.Pages/data-archival/images/snag/schedule-multiple-documents-for-archiving.snagx b/Documentation/6.2/Raven.Documentation.Pages/data-archival/images/snag/schedule-multiple-documents-for-archiving.snagx new file mode 100644 index 0000000000..f85999a907 Binary files /dev/null and b/Documentation/6.2/Raven.Documentation.Pages/data-archival/images/snag/schedule-multiple-documents-for-archiving.snagx differ diff --git a/Documentation/6.2/Raven.Documentation.Pages/data-archival/images/snag/unarchive-documents.snagx b/Documentation/6.2/Raven.Documentation.Pages/data-archival/images/snag/unarchive-documents.snagx new file mode 100644 index 0000000000..aa2b379a24 Binary files /dev/null and b/Documentation/6.2/Raven.Documentation.Pages/data-archival/images/snag/unarchive-documents.snagx differ diff --git a/Documentation/6.2/Raven.Documentation.Pages/data-archival/images/unarchive-documents.png b/Documentation/6.2/Raven.Documentation.Pages/data-archival/images/unarchive-documents.png new file mode 100644 index 0000000000..7ab3118786 Binary files /dev/null and b/Documentation/6.2/Raven.Documentation.Pages/data-archival/images/unarchive-documents.png differ diff --git a/Documentation/6.2/Raven.Documentation.Pages/data-archival/overview.markdown b/Documentation/6.2/Raven.Documentation.Pages/data-archival/overview.markdown new file mode 100644 index 0000000000..b2d0a14a9c --- /dev/null +++ b/Documentation/6.2/Raven.Documentation.Pages/data-archival/overview.markdown @@ -0,0 +1,127 @@ +# Data Archival Overview +--- + +{NOTE: } + +* As a database grows, basic functions like indexing can slow down. + To address this, RavenDB offers the ability to **archive selected documents**. + Documents that are rarely accessed or no longer relevant for active use, but still need to be kept for compliance or historical purposes, can be archived. + +* RavenDB features can detect archived documents and handle them in different ways. + For example, indexing and data subscriptions can be configured to exclude, include, or handle only archived documents during processing. + Limiting processing to either archived or non-archived documents may **improve performance**. + Learn more in [Archived documents and other features](../data-archival/archived-documents-and-other-features). + +* Archived documents are stored in **compressed** form, which helps **reduce database size**. + Note that while compression saves disk space, retrieving archived documents may be slower and consume more CPU/memory than accessing regular documents. + +* To take advantage of the archiving feature, you must first **enable** it. + Once enabled, you can **schedule** documents for archiving. + Learn more in the [Overview](../data-archival/overview#overview) section below. + +--- + +* In this article: + * [Overview](../data-archival/overview#overview) + * [The archived document](../data-archival/overview#the-archived-document) + * [Licensing](../data-archival/overview#licensing) + +{NOTE/} + +--- + +{PANEL: Overview} + +* **Enable the archiving feature** + * To archive documents, you must first enable the archiving feature in RavenDB. + Learn how in the dedicated article: [Enable data archiving](../data-archival/enable-data-archiving). + +* **Schedule documents for archival** + * Once the feature is enabled, you can schedule any document to be archived at a specific future time. + This is done by adding the `@archive-at` metadata property to the document, which specifies the time at which it should be archived. + Learn more in the dedicated article: [Schedule document archiving](../data-archival/schedule-document-archiving). + * The server scans the database periodically (at an interval specified when the task is enabled), + identifies documents scheduled for archiving, and archives them at the scheduled time. + * In a cluster, the archiving task runs on the [preferred node](../client-api/configuration/load-balance/overview#the-preferred-node) (the first node in the cluster topology). + Archived documents are then propagated to the other nodes through [internal replication](../server/clustering/replication/replication#internal-replication). + +* **What is archived** + * The JSON document itself is archived. + Time series, counters, and attachments associated with the document are Not archived. + A revision that is created from an archived document is archived as well. + +* **Modifying archived documents** + * Archived documents and their extensions (time series, counters, attachments) remain accessible and can be updated + (except for revisions, which are immutable). + * Modifying an archived document or any of its extensions does not affect its archival status - + the document remains archived. + +* **Unarchiving documents** + * An archived document can be unarchived. + Learn more in the dedicated article: [Unarchiving documents](../data-archival/unarchiving-documents). + +{PANEL/} + +{PANEL: The archived document} + +A document that has been archived is compressed and marked with both a metadata property and an internal metadata flag: + +* **Metadata property**: + * When a document is archived, + the archiving task replaces the `@archive-at` metadata property with `"@archived": true`. + * This property is informational only - you **cannot** archive a document by manually adding `"@archived": true` to its metadata. + To archive a document, you must schedule it for archival. Learn more in [Schedule document archiving](../data-archival/schedule-document-archiving). + * This property allows RavenDB features, clients, and users to recognize the document’s archived status and handle it accordingly. + For example, a user-defined ETL task can use the presence of this property to skip or handle archived documents differently. + Learn more in [Archived documents and other features](../data-archival/archived-documents-and-other-features). + +* **Internal flag**: + * When a document is archived, the internal flag `"@flags": "Archived"` is added to its metadata. + * Features like indexing and data subscriptions use this flag to detect archived documents and handle them based on the configured behavior. + Learn more in [Archived documents and indexing](../data-archival/archived-documents-and-other-features#archived-documents-and-indexing) & + [Archived documents and data subscriptions](../data-archival/archived-documents-and-other-features#archived-documents-and-subscriptions). + +--- + +**An archived document**: + +![An archived document](images/an-archived-document.png "An archived document") + +1. This label in the Studio indicates that this document is archived. +2. The `@flags` metadata property contains the `Archived` flag. +3. The document’s metadata contains the `@archived: true` property. + +--- + +**An archived document in the list view**: + +![Archived document in list view](images/archived-document-in-list-view.png "Archived document in list view") + +1. This icon indicates the document is archived. + +{PANEL/} + +{PANEL: Licensing} + +The archival feature is available with the **Enterprise** license. +Learn more about licensing in [Licensing overview](../start/licensing/licensing-overview). + +{PANEL/} + +## Related Articles + +### Document Archival +- [Enable data archiving](../data-archival/enable-data-archiving) +- [Schedule document archiving](../data-archival/schedule-document-archiving) +- [Archived documents and other features](../data-archival/archived-documents-and-other-features) +- [Unarchiving documents](../data-archival/unarchiving-documents) + +### Configuration +- [Overview](../server/configuration/configuration-options#settings.json) +- [Database Settings](../studio/database/settings/database-settings#view-database-settings) + +### Extensions +- [Document Expiration](../server/extensions/expiration) + +### Patching +- [Patch By Query](../client-api/rest-api/queries/patch-by-query) diff --git a/Documentation/6.2/Raven.Documentation.Pages/data-archival/schedule-document-archiving.dotnet.markdown b/Documentation/6.2/Raven.Documentation.Pages/data-archival/schedule-document-archiving.dotnet.markdown new file mode 100644 index 0000000000..ef4cde8f85 --- /dev/null +++ b/Documentation/6.2/Raven.Documentation.Pages/data-archival/schedule-document-archiving.dotnet.markdown @@ -0,0 +1,144 @@ +# Schedule Document Archiving +--- + +{NOTE: } + +* Documents cannot be archived directly - they must be scheduled for archival. + To **schedule a document** for archival, add the `@archive-at` property to the document's metadata and set its value to the desired archival time (in UTC). + This can be done in several ways, as described in this article. + +* **Note**: + Just scheduling a document for archival does Not archive it at the specified time. + Actual archiving is performed only by a background task that runs when the archival feature is [enabled](../data-archival/enable-data-archiving). + This task periodically scans the database for documents scheduled for archival. + The scan frequency is configurable when [enabling ](../data-archival/enable-data-archiving) the archival feature (default: 60 seconds). + +* The archiving task will archive any document whose `@archive-at` time has passed at the time of the scan. + The `@archive-at` metadata property will then be replaced with `@archived: true`. + +* You can schedule documents for archival even if the archiving feature is not yet enabled. + These documents will be archived once the feature is enabled and the task runs - provided the scheduled time has already passed. + +--- + +* In this article: + * [Schedule a SINGLE document for archiving - from the Client API](../data-archival/schedule-document-archiving#schedule-a-single-document-for-archiving---from-the-client-api) + * [Schedule a SINGLE document for archiving - from the Studio](../data-archival/schedule-document-archiving#schedule-a-single-document-for-archiving---from-the-studio) + * [Schedule MULTIPLE documents for archiving - from the Client API](../data-archival/schedule-document-archiving#schedule-multiple-documents-for-archiving---from-the-client-api) + * [Schedule MULTIPLE documents for archiving - from the Studio](../data-archival/schedule-document-archiving#schedule-multiple-documents-for-archiving---from-the-studio) + +{NOTE/} + +--- + +{PANEL: Schedule a SINGLE document for archiving - from the Client API} + +To schedule a single document for archival from the Client API, +add the `@archive-at` property directly to the document metadata as follows: + +{CODE-TABS} +{CODE-TAB:csharp:Schedule_document schedule_document@DataArchival\ScheduleArchiving.cs /} +{CODE-TAB:csharp:Schedule_document_async schedule_document_async@DataArchival\ScheduleArchiving.cs /} +{CODE-TABS/} + +Learn more about modifying the metadata of a document in [Modifying Document Metadata](../client-api/session/how-to/get-and-modify-entity-metadata). + +{PANEL/} + +{PANEL: Schedule a SINGLE document for archiving - from the Studio} + +* To schedule a single document for archival from the Studio: + * Open the Document view. + * Add the `@archive-at` property to the document's metadata. + * Set its value to the desired archive time in UTC format. + * Save the document. + +![Schedule a document for archiving](images/schedule-document-for-archiving.png "Schedule a document for archiving") + +1. This is the `@archive-at` property that was added to the document's metadata. + E.g.: `"@archive-at": "2025-06-25T14:00:00.0000000Z"` +2. After saving the document, the Studio displays the time remaining until the document is archived. + +{PANEL/} + +{PANEL: Schedule MULTIPLE documents for archiving - from the Client API} + +* Use the `PatchByQueryOperation` to schedule multiple documents for archiving. + +* In the **patch query**, you can apply any filtering condition to select only the documents you want to archive. + In the **patch script**, call the `archived.archiveAt` method to set the desired archival time (in UTC). + +* When the patch operation is executed, + the server will add the `@archive-at` property to the metadata of all documents that match the query filter. + +--- + +**Example:** + +The following example schedules all orders that were made at least a year ago for archival. +The **patch query** filters for these older orders. +Any document matching the query is then scheduled for archival by the **patch script**. + +{CODE-TABS} +{CODE-TAB:csharp:Schedule_documents schedule_documents@DataArchival\ScheduleArchiving.cs /} +{CODE-TAB:csharp:Schedule_documents_async schedule_documents_async@DataArchival\ScheduleArchiving.cs /} +{CODE-TAB:csharp:Operation_overload schedule_documents_overload@DataArchival\ScheduleArchiving.cs /} +{CODE-TAB:csharp:Operation_overload_async schedule_documents_overload_async@DataArchival\ScheduleArchiving.cs /} +{CODE-TABS/} + +--- + +**Syntax:** + +{CODE syntax@DataArchival\ScheduleArchiving.cs /} + +| Parameter | Type | Description | +|-----------------------|-------------|--------------------------------------------------------------------------------------| +| **doc** | `object` | The document to schedule for archiving. | +| **utcDateTimeString** | `string` | The UTC timestamp (as a string) that specifies when the document should be archived. | + +To learn more about the `PatchByQueryOperation`, see [Set-based patch operations](../client-api/operations/patching/set-based). + +{PANEL/} + +{PANEL: Schedule MULTIPLE documents for archiving - from the Studio} + +* To schedule multiple documents for archiving from the Studio: + * Open the Patch view. + * Enter a patch script that uses the `archived.archiveAt` method, providing the desired archive date in UTC. + * Execute the patch. + +--- + +**Example**: + +The following patch script, used directly in the Studio, +performs the same operation as the [Client API example](../data-archival/schedule-document-archiving#schedule-multiple-documents-for-archiving---from-the-client-api) shown above. + +![Schedule multiple documents for archiving](images/schedule-multiple-documents-for-archiving.png "Schedule multiple documents for archiving") + +1. Open the Patch view. +2. Enter the patch script. +3. Click to execute the patch. +4. You can test the patch on a sample document before executing the whole operation. + Learn more in [Test patch](../studio/database/documents/patch-view#test-patch). + +{PANEL/} + +## Related Articles + +### Document Archival +- [Overview](../data-archival/overview) +- [Enable data archiving](../data-archival/enable-document-archiving) +- [Archived documents and other features](../data-archival/archived-documents-and-other-features) +- [Unarchiving documents](../data-archival/unarchiving-documents) + +### Configuration +- [Overview](../server/configuration/configuration-options#settings.json) +- [Database Settings](../studio/database/settings/database-settings#view-database-settings) + +### Extensions +- [Document Expiration](../server/extensions/expiration) + +### Patching +- [Patch By Query](../client-api/rest-api/queries/patch-by-query) diff --git a/Documentation/6.2/Raven.Documentation.Pages/data-archival/unarchiving-documents.dotnet.markdown b/Documentation/6.2/Raven.Documentation.Pages/data-archival/unarchiving-documents.dotnet.markdown new file mode 100644 index 0000000000..289c3226fd --- /dev/null +++ b/Documentation/6.2/Raven.Documentation.Pages/data-archival/unarchiving-documents.dotnet.markdown @@ -0,0 +1,141 @@ +# Unarchiving Documents +--- + + + {NOTE: } + +* Archived documents can be unarchived at any time. + +* The archiving feature does Not need to be enabled to unarchive documents. + Any previously archived document can be unarchived, regardless of the feature's current state. + +* Do **not** attempt to unarchive a document by manually removing the `@archived: true` metadata property from the document. + This will not clear the internal archived status of the document. + To properly unarchive a document, use the `archived.unarchive()` method as described below. + +* In this article: + * [Unarchive documents - from the Client API](../data-archival/unarchiving-documents#unarchive-documents---from-the-client-api) + * [Unarchive documents - from the Studio](../data-archival/unarchiving-documents#unarchive-documents---from-the-studio) + * [Unarchiving documents with index-based patch queries](../data-archival/unarchiving-documents#unarchiving-documents-with-index-based-patch-queries) + +{NOTE/} + +--- + +{PANEL: Unarchive documents - from the Client API} + +* To unarchive documents from the Client API, use the `PatchByQueryOperation` operation, + which targets one or more documents based on the patch query. + +* You can apply any filtering condition within the query to target only the documents you want to unarchive. + +* Within the **patch script**, call the `archived.unarchive()` method to unarchive all documents + that match the **patch query**. + +--- + +**Example:** + +The following operation will unarchive ALL archived documents in the _Orders_ collection. + +{CODE-TABS} +{CODE-TAB:csharp:Unarchive_documents unarchiving@DataArchival\Unarchiving.cs /} +{CODE-TAB:csharp:Unarchive_documents_async unarchiving_async@DataArchival\Unarchiving.cs /} +{CODE-TAB:csharp:Operation_overload unarchiving_overload@DataArchival\Unarchiving.cs /} +{CODE-TAB:csharp:Operation_overload_async unarchiving_overload_async@DataArchival\Unarchiving.cs /} +{CODE-TABS/} + +--- + +**Syntax:** + +{CODE syntax@DataArchival\Unarchiving.cs /} + +| Parameter | Type | Description | +|------------|----------|----------------------------| +| **doc** | `object` | The document to unarchive. | + +{PANEL/} + +{PANEL: Unarchive documents - from the Studio} + +* To unarchive documents from the Studio: + * Open the Patch view. + * Enter a patch script that uses the `archived.unarchive()` method. + * Execute the patch. + +--- + +**Example**: + +The following patch script, used directly in the Studio, +performs the same operation as the [Client API example](../data-archival/unarchiving-documents#unarchive-documents---from-the-client-api) shown above. +It will unarchive all archived documents in the _Orders_ collection. + +![Unarchive documents](images/unarchive-documents.png "Unarchive documents") + +1. Open the Patch view. +2. Enter the patch script. +3. Click to execute the patch. +4. You can test the patch on a sample document before executing the whole operation. + Learn more in [Test patch](../studio/database/documents/patch-view#test-patch). + +{PANEL/} + +{PANEL: Unarchiving documents with index-based patch queries} + +* When running a patch query to unarchive documents over an index, + you need to consider the index configuration regarding archived documents. + +* If the index is configured to exclude archived documents, the query portion of the patch operation will not match any archived documents - + because those documents are not included in the index. + As a result, no documents will be unarchived by the patch operation. + +* For example, the following patch query uses a dynamic query that creates an auto-index. + If the [Indexing.Auto.ArchivedDataProcessingBehavior](../server/configuration/indexing-configuration#indexing.auto.archiveddataprocessingbehavior) configuration key is set to its default `ExcludeArchived` value, + then even if archived documents exist in the _Orders_ collection with `ShipTo.Country == 'USA'`, they will not be matched - because the auto-index does not include them - + and the patch operation will not unarchive any documents. + + {CODE unarchive_using_auto_index@DataArchival\Unarchiving.cs /} + +--- + +Two possible workarounds are: + +1. **Configure the index to include archived documents**: + This ensures archived documents are included in the index and can be matched by the patch query. + Use this option only if including archived documents in the index aligns with your indexing strategy. + + **For auto-indexes**: + Set the [Indexing.Auto.ArchivedDataProcessingBehavior](../server/configuration/indexing-configuration#indexing.auto.archiveddataprocessingbehavior) configuration key to `IncludeArchived`. + **For static-indexes**: + Set the [Indexing.Static.ArchivedDataProcessingBehavior](../server/configuration/indexing-configuration#indexing.static.archiveddataprocessingbehavior) configuration key to `IncludeArchived`, + or - configure the definition of the specific static-index to include archived documents. + See [Archived documents and indexing](../data-archival/archived-documents-and-other-features#archived-documents-and-indexing). + +2. **Use a collection query instead of an index query**: + Run a simple collection-based query that does not rely on an index. + Apply your filtering logic inside the patch script to unarchive only the relevant documents. + + For example: + {CODE unarchive_using_filter_in_script@DataArchival\Unarchiving.cs /} + +{PANEL/} + +## Related Articles + +### Document Archival +- [Overview](../data-archival/overview) +- [Enable data archiving](../data-archival/enable-data-archiving) +- [Schedule document archiving](../data-archival/schedule-document-archiving) +- [Archived documents and other features](../data-archival/archived-documents-and-other-features) + +### Configuration +- [Overview](../server/configuration/configuration-options#settings.json) +- [Database Settings](../studio/database/settings/database-settings#view-database-settings) + +### Extensions +- [Document Expiration](../server/extensions/expiration) + +### Patching +- [Patch By Query](../client-api/rest-api/queries/patch-by-query) diff --git a/Documentation/6.2/Raven.Documentation.Pages/server/configuration/indexing-configuration.markdown b/Documentation/6.2/Raven.Documentation.Pages/server/configuration/indexing-configuration.markdown index cd62d88406..2323d3a23c 100644 --- a/Documentation/6.2/Raven.Documentation.Pages/server/configuration/indexing-configuration.markdown +++ b/Documentation/6.2/Raven.Documentation.Pages/server/configuration/indexing-configuration.markdown @@ -212,9 +212,12 @@ EXPERT ONLY: {PANEL: Indexing.Auto.ArchivedDataProcessingBehavior} -Set the default deployment mode for static indexes. +The default processing behavior for archived documents in auto-indexes. -- **Type**: `enum IndexDeploymentModeArchivedDataProcessingBehavior` (`ExcludeArchived`, `IncludeArchived`, `ArchivedOnly`) +- **Type**: `enum ArchivedDataProcessingBehavior`: + * `ExcludeArchived`: only non-archived documents are processed by the index. + * `IncludeArchived`: both archived and non-archived documents are processed by the index. + * `ArchivedOnly`: only archived documents are processed by the index. - **Default**: `ExcludeArchived` - **Scope**: Server-wide, or per database @@ -353,9 +356,16 @@ Allow to open an index without checking if current Database ID matched the one f {PANEL: Indexing.Static.ArchivedDataProcessingBehavior} -Set the default deployment mode for static indexes. +* Set the default processing behavior for archived documents in static indexes. +* This setting applies only to static indexes that use _Documents_ as their data source. + It does not apply to indexes based on _Time Series_ or _Counters_, which default to `IncludeArchived`. + +--- -- **Type**: `enum IndexDeploymentModeArchivedDataProcessingBehavior` (`ExcludeArchived`, `IncludeArchived`, `ArchivedOnly`) +- **Type**: `enum ArchivedDataProcessingBehavior`: + * `ExcludeArchived`: only non-archived documents are processed by the index. + * `IncludeArchived`: both archived and non-archived documents are processed by the index. + * `ArchivedOnly`: only archived documents are processed by the index. - **Default**: `ExcludeArchived` - **Scope**: Server-wide, or per database diff --git a/Documentation/6.2/Raven.Documentation.Pages/server/configuration/subscription-configuration.markdown b/Documentation/6.2/Raven.Documentation.Pages/server/configuration/subscription-configuration.markdown new file mode 100644 index 0000000000..6665c7ac04 --- /dev/null +++ b/Documentation/6.2/Raven.Documentation.Pages/server/configuration/subscription-configuration.markdown @@ -0,0 +1,36 @@ +# Configuration: Subscriptions +--- + +{NOTE: } + +* The following configuration keys control various aspects of subscription behavior in RavenDB. + Learn more about subscriptions in [Data subscriptions](../../client-api/data-subscriptions/what-are-data-subscriptions). + +* In this article: + * [Subscriptions.ArchivedDataProcessingBehavior](../../server/configuration/subscription-configuration#subscriptions.archiveddataprocessingbehavior) + * [Subscriptions.MaxNumberOfConcurrentConnections](../../server/configuration/subscription-configuration#subscriptions.maxnumberofconcurrentconnections) + +{NOTE/} + +{PANEL: Subscriptions.ArchivedDataProcessingBehavior} + +The default processing behavior for archived documents in a subscription query. + +- **Type**: `enum ArchivedDataProcessingBehavior`: + * `ExcludeArchived`: only non-archived documents are processed by the subscription query. + * `IncludeArchived`: both archived and non-archived documents are processed by the subscription query. + * `ArchivedOnly`: only archived documents are processed by the subscription query. +- **Default**: `ExcludeArchived` +- **Scope**: Server-wide, or per database + +{PANEL/} + +{PANEL: Subscriptions.MaxNumberOfConcurrentConnections} + +The maximum number of concurrent subscription connections allowed per database. + +- **Type**: `int` +- **Default**: `1000` +- **Scope**: Server-wide or per database + +{PANEL/} diff --git a/Documentation/6.2/Raven.Documentation.Pages/server/extensions/.docs.json b/Documentation/6.2/Raven.Documentation.Pages/server/extensions/.docs.json index 5c306d031c..8066b07990 100644 --- a/Documentation/6.2/Raven.Documentation.Pages/server/extensions/.docs.json +++ b/Documentation/6.2/Raven.Documentation.Pages/server/extensions/.docs.json @@ -15,11 +15,5 @@ "Name": "Document Refresh", "DiscussionId": "b976732a-3da7-48e5-a454-4f34b8f716f9", "Mappings": [] - }, - { - "Path": "archival.markdown", - "Name": "Data Archival", - "DiscussionId": "b976732a-3da7-48e5-a454-4f34b8f716f9", - "Mappings": [] } ] diff --git a/Documentation/6.2/Raven.Documentation.Pages/studio/database/settings/.docs.json b/Documentation/6.2/Raven.Documentation.Pages/studio/database/settings/.docs.json index 5c5d6e0894..14a9b454eb 100644 --- a/Documentation/6.2/Raven.Documentation.Pages/studio/database/settings/.docs.json +++ b/Documentation/6.2/Raven.Documentation.Pages/studio/database/settings/.docs.json @@ -49,7 +49,7 @@ }, { "Path": "document-archival.markdown", - "Name": "Document Archival", + "Name": "Data Archival", "DiscussionId": "c5633531-4b5d-4816-97d4-82d351f3f6a2", "Mappings": [] }, diff --git a/Documentation/6.2/Raven.Documentation.Pages/studio/database/settings/document-archival.markdown b/Documentation/6.2/Raven.Documentation.Pages/studio/database/settings/document-archival.markdown new file mode 100644 index 0000000000..c2e2426202 --- /dev/null +++ b/Documentation/6.2/Raven.Documentation.Pages/studio/database/settings/document-archival.markdown @@ -0,0 +1,19 @@ +# Data Archival +--- + +{NOTE: } + +* As a database grows, basic functions like indexing can slow down. + To address this, RavenDB offers the ability to **archive selected documents**. + +* Documents that are rarely accessed or no longer relevant for active use, + but still need to be kept for compliance or historical purposes, can be archived. + +* This feature is covered in detail in the following articles: + * [Data archival overview](../../../data-archival/overview) + * [Enable data archiving](../../../data-archival/enable-data-archiving) + * [Schedule document archiving](../../../data-archival/schedule-document-archiving) + * [Archived documents and other features](../../../data-archival/archived-documents-and-other-features) + * [Unarchiving documents](../../../data-archival/unarchiving-documents) + +{NOTE/} diff --git a/Documentation/6.2/Samples/csharp/Raven.Documentation.Samples/DataArchival/ArchivedDocsIntegration.cs b/Documentation/6.2/Samples/csharp/Raven.Documentation.Samples/DataArchival/ArchivedDocsIntegration.cs new file mode 100644 index 0000000000..c69a577f24 --- /dev/null +++ b/Documentation/6.2/Samples/csharp/Raven.Documentation.Samples/DataArchival/ArchivedDocsIntegration.cs @@ -0,0 +1,254 @@ +using System; +using System.Collections.Generic; +using System.Linq; +using System.Threading.Tasks; +using Raven.Client.Documents; +using Raven.Client.Documents.Conventions; +using Raven.Client.Documents.DataArchival; +using Raven.Client.Documents.Indexes; +using Raven.Client.Documents.Operations.Indexes; +using Raven.Client.Documents.Smuggler; +using Raven.Client.Documents.Subscriptions; +using Raven.Documentation.Samples.Orders; + +namespace Raven.Documentation.Samples.DataArchival +{ + #region index_1 + public class Orders_ByOrderDate : + AbstractIndexCreationTask + { + public class IndexEntry + { + public DateTime OrderDate { get; set; } + } + + public Orders_ByOrderDate() + { + Map = orders => from order in orders + select new IndexEntry + { + OrderDate = order.OrderedAt + }; + + // Configure whether the index should process data from archived documents: + // ======================================================================== + ArchivedDataProcessingBehavior = + // You can set to 'ExcludeArchived', 'IncludeArchived, or 'ArchivedOnly' + Raven.Client.Documents.DataArchival.ArchivedDataProcessingBehavior.IncludeArchived; + } + } + #endregion + + #region index_2 + public class Orders_ByOrderDate_JS : AbstractJavaScriptIndexCreationTask + { + public Orders_ByOrderDate_JS() + { + Maps = new HashSet() + { + @"map('Orders', function (order) { + return { + OrderDate: order.OrderedAt + }; + })" + }; + + // Configure whether the index should process data from archived documents: + // ======================================================================== + ArchivedDataProcessingBehavior = + // Can set the to 'ExcludeArchived', 'IncludeArchived, or 'ArchivedOnly' + Raven.Client.Documents.DataArchival.ArchivedDataProcessingBehavior.IncludeArchived; + } + } + #endregion + + #region index_4 + public class Orders_ByArchivedStatus : + AbstractIndexCreationTask + { + public class IndexEntry + { + public bool? isArchived { get; set; } + public DateTime? OrderDate { get; set; } + public string ShipToCountry { get; set; } + } + + public Orders_ByArchivedStatus() + { + Map = orders => from order in orders + let metadata = MetadataFor(order) + + // Retrieve the '@archived' metadata property from the document: + let archivedProperty = + metadata.Value(Raven.Client.Constants.Documents.Metadata.Archived) + // Alternative syntax: + // let archivedProperty = + // (bool?)metadata[Raven.Client.Constants.Documents.Metadata.Archived] + + select new IndexEntry + { + // Index whether the document is archived: + isArchived = archivedProperty == true, + + // Index the order date only if the document is archived: + OrderDate = archivedProperty == true ? order.OrderedAt : null, + + // Index the destination country only if the document is not archived: + ShipToCountry = archivedProperty == null ? order.ShipTo.Country : null + }; + + ArchivedDataProcessingBehavior = + Raven.Client.Documents.DataArchival.ArchivedDataProcessingBehavior.IncludeArchived; + } + } + #endregion + + #region index_5 + public class Orders_ByArchivedStatus_JS : AbstractJavaScriptIndexCreationTask + { + public Orders_ByArchivedStatus_JS() + { + Maps = new HashSet() + { + @"map('Orders', function (order) { + var metadata = metadataFor(order); + var archivedProperty = metadata['@archived']; + + var isArchived = (archivedProperty === true); + + var orderDate = isArchived ? order.OrderedAt : null; + var shipToCountry = !isArchived ? order.ShipTo.Country : null; + + return { + IsArchived: isArchived, + OrderDate: orderDate, + ShipToCountry: shipToCountry + }; + })" + }; + + ArchivedDataProcessingBehavior = + Raven.Client.Documents.DataArchival.ArchivedDataProcessingBehavior.IncludeArchived; + } + } + #endregion + + public class ArchivedDocsIntegration + { + public async Task IndexExamples() + { + using (var store = new DocumentStore()) + { + #region index_3 + var indexDefinition = new IndexDefinitionBuilder() + { + Map = orders => from order in orders + select new { order.OrderedAt } + } + .ToIndexDefinition(new DocumentConventions()); + + indexDefinition.Name = "Orders/ByOrderDate"; + + // Configure whether the index should process data from archived documents: + // ======================================================================== + indexDefinition.ArchivedDataProcessingBehavior = + // You can set to 'ExcludeArchived', 'IncludeArchived, or 'ArchivedOnly' + ArchivedDataProcessingBehavior.IncludeArchived; + + store.Maintenance.Send(new PutIndexesOperation(indexDefinition)); + #endregion + } + + using (var store = new DocumentStore()) + { + #region index_6 + var indexDefinition = new IndexDefinition + { + Name = "Orders/ByArchivedStatus", + + Maps = new HashSet + { + @"from order in docs.Orders + let metadata = MetadataFor(order) + let archivedProperty = (bool?)metadata[""@archived""] + + select new + { + IsArchived = archivedProperty == true, + OrderDate = archivedProperty == true ? order.OrderedAt : null, + ShipToCountry = archivedProperty == null ? order.ShipTo.Country : null + }" + }, + + ArchivedDataProcessingBehavior = ArchivedDataProcessingBehavior.IncludeArchived + }; + + store.Maintenance.Send(new PutIndexesOperation(indexDefinition)); + #endregion + } + } + + public async Task SubscriptionExamples() + { + using (var store = new DocumentStore()) + { + #region subscription_task_generic_syntax + var subscriptionName = store.Subscriptions + .Create(new SubscriptionCreationOptions() + { + Name = "ArchivedOrdersSubscription", + // Workers that will subscribe to this subscription task + // will receive only archived documents from the 'Orders' collection. + ArchivedDataProcessingBehavior = ArchivedDataProcessingBehavior.ArchivedOnly + + // You can set the behavior to 'ExcludeArchived', 'IncludeArchived, or 'ArchivedOnly' + }); + #endregion + } + + using (var store = new DocumentStore()) + { + #region subscription_task_rql_syntax + var subscriptionName = store.Subscriptions + .Create(new SubscriptionCreationOptions() + { + Name = "ArchivedOrdersSubscription", + Query = "from Orders", + // Workers that will subscribe to this subscription task + // will receive only archived documents from the 'Orders' collection. + ArchivedDataProcessingBehavior = ArchivedDataProcessingBehavior.ArchivedOnly + + // You can set the behavior to 'ExcludeArchived', 'IncludeArchived, or 'ArchivedOnly' + }); + #endregion + } + } + + public async Task SmugglerExamples() + { + using (var store = new DocumentStore()) + { + #region export + var exportOperation = store.Smuggler.ExportAsync( + new DatabaseSmugglerExportOptions() + { + // Export only non-archived documents: + IncludeArchived = false + }, "DestinationFilePath"); + #endregion + } + + using (var store = new DocumentStore()) + { + #region import + var importOperation = store.Smuggler.ImportAsync( + new DatabaseSmugglerImportOptions() + { + // Include archived documents in the import: + IncludeArchived = true + }, "SourceFilePath"); + #endregion + } + } + } +} diff --git a/Documentation/6.2/Samples/csharp/Raven.Documentation.Samples/DataArchival/EnableArchiving.cs b/Documentation/6.2/Samples/csharp/Raven.Documentation.Samples/DataArchival/EnableArchiving.cs new file mode 100644 index 0000000000..a05f730c89 --- /dev/null +++ b/Documentation/6.2/Samples/csharp/Raven.Documentation.Samples/DataArchival/EnableArchiving.cs @@ -0,0 +1,83 @@ +using System.Threading.Tasks; +using Raven.Client.Documents; +using Raven.Client.Documents.Operations.DataArchival; + +namespace Raven.Documentation.Samples.DataArchival +{ + public class EnableArchiving + { + public async Task Examples() + { + using (var store = new DocumentStore()) + { + #region enable_archiving + // Define the archival configuration object + var configuration = new DataArchivalConfiguration + { + // Enable archiving + Disabled = false, + + // Optional: override the default archiving frequency + // Scan for documents scheduled for archiving every 180 seconds + ArchiveFrequencyInSec = 180, + + // Optional: limit the number of documents processed in each archival run + MaxItemsToProcess = 100 + }; + + // Define the archival operation, pass the configuration + var configureArchivalOp = new ConfigureDataArchivalOperation(configuration); + + // Execute the operation by passing it to Maintenance.Send + store.Maintenance.Send(configureArchivalOp); + #endregion + } + + using (var store = new DocumentStore()) + { + #region enable_archiving_async + // Define the archival configuration object + var configuration = new DataArchivalConfiguration + { + // Enable archiving + Disabled = false, + + // Optional: override the default archiving frequency + // Scan for documents scheduled for archiving every 180 seconds + ArchiveFrequencyInSec = 180, + + // Optional: limit the number of documents processed in each archival run + MaxItemsToProcess = 100 + }; + + // Define the archival operation, pass the configuration + var configureArchivalOp = new ConfigureDataArchivalOperation(configuration); + + // Execute the operation by passing it to Maintenance.SendAsync + await store.Maintenance.SendAsync(configureArchivalOp); + #endregion + } + } + + private interface IFoo + { + /* + #region syntax_1 + public ConfigureDataArchivalOperation(DataArchivalConfiguration configuration) + #endregion + */ + + /* + #region syntax_2 + public class DataArchivalConfiguration + { + public bool Disabled { get; set; } + public long? ArchiveFrequencyInSec { get; set; } + public long? MaxItemsToProcess { get; set; } + } + #endregion + */ + } + } +} + diff --git a/Documentation/6.2/Samples/csharp/Raven.Documentation.Samples/DataArchival/ScheduleArchiving.cs b/Documentation/6.2/Samples/csharp/Raven.Documentation.Samples/DataArchival/ScheduleArchiving.cs new file mode 100644 index 0000000000..1d49c0e361 --- /dev/null +++ b/Documentation/6.2/Samples/csharp/Raven.Documentation.Samples/DataArchival/ScheduleArchiving.cs @@ -0,0 +1,195 @@ +using System.Threading.Tasks; +using Raven.Client; +using Raven.Client.Documents; +using Raven.Client.Documents.Operations; +using Raven.Client.Documents.Queries; +using Raven.Client.Util; +using Raven.Documentation.Samples.Orders; + +namespace Raven.Documentation.Samples.DataArchival +{ + public class ScheduleArchiving + { + public async Task Examples() + { + // Schedule single document: + // ========================= + + using (var store = new DocumentStore()) + { + #region schedule_document + using (var session = store.OpenSession()) + { + // Load the document to schedule for archiving + var company = session.Load("companies/91-A"); + + // Access the document's metadata + var metadata = session.Advanced.GetMetadataFor(company); + + // Set the future archival date (in UTC) + var archiveDate = SystemTime.UtcNow.AddDays(1); + metadata["@archive-at"] = archiveDate; + + // Save the changes + session.SaveChanges(); + } + #endregion + + #region schedule_document_async + using (var asyncSession = store.OpenAsyncSession()) + { + // Load the document to schedule for archiving + var company = await asyncSession.LoadAsync("companies/91-A"); + + // Access the document's metadata + var metadata = asyncSession.Advanced.GetMetadataFor(company); + + // Set the future archival date (in UTC) + var archiveDate = SystemTime.UtcNow.AddDays(1); + metadata["@archive-at"] = archiveDate; + + // Save the changes + await asyncSession.SaveChangesAsync(); + } + #endregion + } + + // Schedule MULTIPLE documents: + // ============================ + + using (var store = new DocumentStore()) + { + #region schedule_documents + var archiveDate = SystemTime.UtcNow.AddDays(1); + string archiveDateString = archiveDate.ToString("o"); + + var oldDate = SystemTime.UtcNow.AddYears(-1); + string oldDateString = oldDate.ToString("o"); + + // Define the patch query string + // Request to archive all Orders older than one year + string patchByQuery = $@" + // The patch query: + // ================ + from Orders + where OrderedAt < '{oldDateString}' + update {{ + // The patch script - schedule for archival: + // ========================================= + archived.archiveAt(this, '{archiveDateString}') + }}"; + + // Define the patch operation, pass the patch query string + var patchByQueryOp = new PatchByQueryOperation(patchByQuery); + + // Execute the operation by passing it to Operations.Send + store.Operations.Send(patchByQueryOp); + #endregion + } + + using (var store = new DocumentStore()) + { + #region schedule_documents_async + var archiveDate = SystemTime.UtcNow.AddDays(1); + string archiveDateString = archiveDate.ToString("o"); + + var oldDate = SystemTime.UtcNow.AddYears(-1); + string oldDateString = oldDate.ToString("o"); + + // Define the patch query string + // Request to archive all Orders older than one year + string patchByQuery = $@" + from Orders + where OrderedAt < '{oldDateString}' + update {{ + archived.archiveAt(this, '{archiveDateString}') + }}"; + + // Define the patch operation, pass the patch query string + var patchByQueryOp = new PatchByQueryOperation(patchByQuery); + + // Execute the operation by passing it to Operations.SendAsync + await store.Operations.SendAsync(patchByQueryOp); + #endregion + } + + using (var store = new DocumentStore()) + { + #region schedule_documents_overload + var archiveDate = SystemTime.UtcNow.AddDays(1); + string archiveDateString = archiveDate.ToString("o"); + + var oldDate = SystemTime.UtcNow.AddYears(-1); + string oldDateString = oldDate.ToString("o"); + + // Define the patch string + // Request to archive all Orders older than one year + string patchByQuery = $@" + from Orders + where OrderedAt < $p0 + update {{ + archived.archiveAt(this, $p1) + }}"; + + // Define the patch operation, pass the patch query + var patchByQueryOp = new PatchByQueryOperation(new IndexQuery() + { + Query = patchByQuery, + QueryParameters = new Parameters() + { + { "p0", oldDateString }, + { "p1", archiveDateString } + } + }); + + // Execute the operation by passing it to Operations.Send + store.Operations.Send(patchByQueryOp); + #endregion + } + + using (var store = new DocumentStore()) + { + #region schedule_documents_overload_async + var archiveDate = SystemTime.UtcNow.AddDays(1); + string archiveDateString = archiveDate.ToString("o"); + + var oldDate = SystemTime.UtcNow.AddYears(-1); + string oldDateString = oldDate.ToString("o"); + + // Define the patch string + // Request to archive all Orders older than one year + string patchByQuery = $@" + from Orders + where OrderedAt < $p0 + update {{ + archived.archiveAt(this, $p1) + }}"; + + // Define the patch operation, pass the patch query + var patchByQueryOp = new PatchByQueryOperation(new IndexQuery() + { + Query = patchByQuery, + QueryParameters = new Parameters() + { + { "p0", oldDateString }, + { "p1", archiveDateString } + } + }); + + // Execute the operation by passing it to Operations.SendAsync + await store.Operations.SendAsync(patchByQueryOp); + #endregion + } + } + + private interface IFoo + { + /* + #region syntax + archived.archiveAt(doc, utcDateTimeString) + #endregion + */ + } + } +} + diff --git a/Documentation/6.2/Samples/csharp/Raven.Documentation.Samples/DataArchival/Unarchiving.cs b/Documentation/6.2/Samples/csharp/Raven.Documentation.Samples/DataArchival/Unarchiving.cs new file mode 100644 index 0000000000..33203abf0b --- /dev/null +++ b/Documentation/6.2/Samples/csharp/Raven.Documentation.Samples/DataArchival/Unarchiving.cs @@ -0,0 +1,149 @@ +using System.Threading.Tasks; +using Raven.Client.Documents; +using Raven.Client.Documents.Operations; +using Raven.Client.Documents.Queries; + +namespace Raven.Documentation.Samples.DataArchival +{ + public class Unarchiving + { + public async Task Examples() + { + using (var store = new DocumentStore()) + { + #region unarchiving + // Define the patch query string + string patchByQuery = @" + // The patch query: + // ================ + from Orders + update + { + // The patch script: + // ================= + archived.unarchive(this) + }"; + + // Define the patch operation, pass the patch string + var patchByQueryOp = new PatchByQueryOperation(patchByQuery); + + // Execute the operation by passing it to Operations.Send + store.Operations.Send(patchByQueryOp); + #endregion + } + + using (var store = new DocumentStore()) + { + #region unarchiving_async + // Define the patch query string + string patchByQuery = @" + from Orders + update + { + archived.unarchive(this) + }"; + + // Define the patch operation, pass the patch string + var patchByQueryOp = new PatchByQueryOperation(patchByQuery); + + // Execute the operation by passing it to Operations.Send + await store.Operations.SendAsync(patchByQueryOp); + #endregion + } + + using (var store = new DocumentStore()) + { + #region unarchiving_overload + // Define the patch query string + string patchByQuery = @" + from Orders + update + { + archived.unarchive(this) + }"; + + // Define the patch operation, pass the patch string + var patchByQueryOp = new PatchByQueryOperation(new IndexQuery() + { + Query = patchByQuery + }); + + // Execute the operation by passing it to Operations.Send + store.Operations.Send(patchByQueryOp); + #endregion + } + + using (var store = new DocumentStore()) + { + #region unarchiving_overload_async + // Define the patch query string + string patchByQuery = @" + from Orders + update + { + archived.unarchive(this) + }"; + + // Define the patch operation, pass the patch string + var patchByQueryOp = new PatchByQueryOperation(new IndexQuery() + { + Query = patchByQuery + }); + + // Execute the operation by passing it to Operations.Send + await store.Operations.SendAsync(patchByQueryOp); + #endregion + } + + using (var store = new DocumentStore()) + { + #region unarchive_using_auto_index + string patchByQuery = @" + // This filtering query creates an auto-index: + // =========================================== + from Orders + where ShipTo.Country == 'USA' + update + { + archived.unarchive(this) + }"; + + var patchByQueryOp = new PatchByQueryOperation(patchByQuery); + store.Operations.Send(patchByQueryOp); + #endregion + } + + using (var store = new DocumentStore()) + { + #region unarchive_using_filter_in_script + string patchByQuery = @" + // Perform a collection query: + // =========================== + from Orders as order + update + { + // Filter documents inside the script: + // =================================== + if (order.ShipTo.City == 'New York') + { + archived.unarchive(this) + } + }"; + + var patchByQueryOp = new PatchByQueryOperation(patchByQuery); + store.Operations.Send(patchByQueryOp); + #endregion + } + } + + private interface IFoo + { + /* + #region syntax + archived.unarchive(doc) + #endregion + */ + } + } +} + diff --git a/Documentation/7.0/Raven.Documentation.Pages/.docs.json b/Documentation/7.0/Raven.Documentation.Pages/.docs.json index 9e077b2dbe..c6eb52122c 100644 --- a/Documentation/7.0/Raven.Documentation.Pages/.docs.json +++ b/Documentation/7.0/Raven.Documentation.Pages/.docs.json @@ -19,6 +19,11 @@ "Name": "Document Extensions", "Mappings": [] }, + { + "Path": "/data-archival", + "Name": "Data Archival", + "Mappings": [] + }, { "Path": "/server", "Name": "Server", diff --git a/Documentation/7.0/Raven.Documentation.Pages/client-api/data-subscriptions/creation/api-overview.dotnet.markdown b/Documentation/7.0/Raven.Documentation.Pages/client-api/data-subscriptions/creation/api-overview.dotnet.markdown index 203c638f36..32bd94861a 100644 --- a/Documentation/7.0/Raven.Documentation.Pages/client-api/data-subscriptions/creation/api-overview.dotnet.markdown +++ b/Documentation/7.0/Raven.Documentation.Pages/client-api/data-subscriptions/creation/api-overview.dotnet.markdown @@ -68,7 +68,7 @@ Options for the **non-generic** version of the subscription creation options obj | **Disabled** | `bool` | `true` - task will be disabled.
`false` - task will be enabled. | | **MentorNode** | `string` | Allows to define a node in the cluster that will be responsible to handle the subscription. Useful when you prefer a specific server due to its stronger hardware, closer geographic proximity to clients, or other reasons. | | **PinToMentorNode** | `bool` | `true` - the selected responsible node will be pinned to handle the task.
`false` - Another node will execute the task if the responsible node is down. | -| **ArchivedDataProcessingBehavior** | `ArchivedDataProcessingBehavior?` | Define whether [archived documents](../../../server/extensions/archival#archiving-and-data-subscriptions) will be included in the subscription. | +| **ArchivedDataProcessingBehavior** | `ArchivedDataProcessingBehavior?` | Define whether [archived documents](../../../data-archival/archived-documents-and-other-features#archived-documents-and-subscriptions) will be included in the subscription. | {NOTE: } diff --git a/Documentation/7.0/Raven.Documentation.Pages/client-api/operations/server-wide/create-database.dotnet.markdown b/Documentation/7.0/Raven.Documentation.Pages/client-api/operations/server-wide/create-database.dotnet.markdown index 6bea4464e2..d64855fcec 100644 --- a/Documentation/7.0/Raven.Documentation.Pages/client-api/operations/server-wide/create-database.dotnet.markdown +++ b/Documentation/7.0/Raven.Documentation.Pages/client-api/operations/server-wide/create-database.dotnet.markdown @@ -109,7 +109,7 @@ The `DatabaseRecord` is a collection of database configurations: | **AutoIndexes** | `Dictionary` | Auto-index definitions for the database. | | **Client** | `ClientConfiguration` | [Client behavior](../../../studio/server/client-configuration) configuration. | | **ConflictSolverConfig** | `ConflictSolver` | Define the strategy used to resolve [Replication conflicts](../../../server/clustering/replication/replication-conflicts). | -| **DataArchival** | `DataArchivalConfiguration` | [Data Archival](../../../server/extensions/archival) configuration for the database. | +| **DataArchival** | `DataArchivalConfiguration` | [Data Archival](../../../data-archival/overview) configuration for the database. | | **DatabaseName** | `string` | The database name. | | **Disabled** | `bool` | Set the database initial state.
`true` - disable the database.
`false` - (default) the database will be enabled.

This can be modified later via [ToggleDatabasesStateOperation](../../../client-api/operations/server-wide/toggle-databases-state). | | **DocumentsCompression** | `DocumentsCompressionConfiguration` | Configuration settings for [Compressing documents](../../../server/storage/documents-compression). | diff --git a/Documentation/7.0/Raven.Documentation.Pages/client-api/operations/server-wide/create-database.java.markdown b/Documentation/7.0/Raven.Documentation.Pages/client-api/operations/server-wide/create-database.java.markdown index 130011c8fd..a3ad1346b4 100644 --- a/Documentation/7.0/Raven.Documentation.Pages/client-api/operations/server-wide/create-database.java.markdown +++ b/Documentation/7.0/Raven.Documentation.Pages/client-api/operations/server-wide/create-database.java.markdown @@ -76,7 +76,7 @@ | **autoIndexes** | `Map` | Auto-index definitions for the database. | | **client** | `ClientConfiguration` | [Client behavior](../../../studio/server/client-configuration) configuration. | | **conflictSolverConfig** | `ConflictSolver` | Define the strategy used to resolve [Replication conflicts](../../../server/clustering/replication/replication-conflicts). | -| **dataArchival** | `DataArchivalConfiguration` | [Data Archival](../../../server/extensions/archival) configuration for the database. | +| **dataArchival** | `DataArchivalConfiguration` | [Data Archival](../../../data-archival/overview) configuration for the database. | | **databaseName** | `String` | The database name. | | **disabled** | `boolean` (default: false) | Set the database initial state.
`true` - disable the database.
`false` - (default) the database will be enabled.

This can be modified later via [ToggleDatabasesStateOperation](../../../client-api/operations/server-wide/toggle-databases-state). | | **documentsCompression** | `DocumentsCompressionConfiguration` | Configuration settings for [Compressing documents](../../../server/storage/documents-compression). | diff --git a/Documentation/7.0/Raven.Documentation.Pages/client-api/operations/what-are-operations.dotnet.markdown b/Documentation/7.0/Raven.Documentation.Pages/client-api/operations/what-are-operations.dotnet.markdown index 77f58fef62..685d259f6d 100644 --- a/Documentation/7.0/Raven.Documentation.Pages/client-api/operations/what-are-operations.dotnet.markdown +++ b/Documentation/7.0/Raven.Documentation.Pages/client-api/operations/what-are-operations.dotnet.markdown @@ -273,6 +273,7 @@ * **Misc**:        ConfigureExpirationOperation        ConfigureRefreshOperation +        [ConfigureDataArchivalOperation](../../data-archival/enable-data-archiving#enable-archiving---from-the-client-api)        UpdateDocumentsCompressionConfigurationOperation        DatabaseHealthCheckOperation        GetOperationStateOperation diff --git a/Documentation/7.0/Raven.Documentation.Pages/client-api/operations/what-are-operations.js.markdown b/Documentation/7.0/Raven.Documentation.Pages/client-api/operations/what-are-operations.js.markdown index 7c623255e4..1081f46c6d 100644 --- a/Documentation/7.0/Raven.Documentation.Pages/client-api/operations/what-are-operations.js.markdown +++ b/Documentation/7.0/Raven.Documentation.Pages/client-api/operations/what-are-operations.js.markdown @@ -271,6 +271,7 @@ __Send syntax__: * __Misc__:        ConfigureExpirationOperation        ConfigureRefreshOperation +        ConfigureDataArchivalOperation        UpdateDocumentsCompressionConfigurationOperation        DatabaseHealthCheckOperation        GetOperationStateOperation diff --git a/Documentation/7.0/Raven.Documentation.Pages/client-api/operations/what-are-operations.php.markdown b/Documentation/7.0/Raven.Documentation.Pages/client-api/operations/what-are-operations.php.markdown index 502dcbe6fe..e5f737d1f9 100644 --- a/Documentation/7.0/Raven.Documentation.Pages/client-api/operations/what-are-operations.php.markdown +++ b/Documentation/7.0/Raven.Documentation.Pages/client-api/operations/what-are-operations.php.markdown @@ -256,6 +256,7 @@ * **Misc**:        ConfigureExpirationOperation        ConfigureRefreshOperation +        ConfigureDataArchivalOperation        UpdateDocumentsCompressionConfigurationOperation        DatabaseHealthCheckOperation        GetOperationStateOperation diff --git a/Documentation/7.0/Raven.Documentation.Pages/client-api/operations/what-are-operations.python.markdown b/Documentation/7.0/Raven.Documentation.Pages/client-api/operations/what-are-operations.python.markdown index 9fe3c6c944..ede914b0ca 100644 --- a/Documentation/7.0/Raven.Documentation.Pages/client-api/operations/what-are-operations.python.markdown +++ b/Documentation/7.0/Raven.Documentation.Pages/client-api/operations/what-are-operations.python.markdown @@ -257,6 +257,7 @@ * **Misc**:        ConfigureExpirationOperation        ConfigureRefreshOperation +        ConfigureDataArchivalOperation        UpdateDocumentsCompressionConfigurationOperation        DatabaseHealthCheckOperation        GetOperationStateOperation diff --git a/Documentation/7.0/Raven.Documentation.Pages/data-archival/.docs.json b/Documentation/7.0/Raven.Documentation.Pages/data-archival/.docs.json new file mode 100644 index 0000000000..3bc90f6a55 --- /dev/null +++ b/Documentation/7.0/Raven.Documentation.Pages/data-archival/.docs.json @@ -0,0 +1,32 @@ +[ + { + "Path": "overview.markdown", + "Name": "Overview", + "DiscussionId": "28bb68b0-3af7-46b3-ad4c-e34b7e81565d", + "Mappings": [] + }, + { + "Path": "enable-data-archiving.markdown", + "Name": "Enable Data Archiving", + "DiscussionId": "e5eeaaa7-bb98-4ae7-9cd1-75cb300738e7", + "Mappings": [] + }, + { + "Path": "schedule-document-archiving.markdown", + "Name": "Schedule Document Archiving", + "DiscussionId": "ec35fa9a-834a-48b0-96c3-d67e1648cbae", + "Mappings": [] + }, + { + "Path": "archived-documents-and-other-features.markdown", + "Name": "Archived Documents and Other Features", + "DiscussionId": "977c8625-ad1d-4dd9-8b29-eb92037957f8", + "Mappings": [] + }, + { + "Path": "unarchiving-documents.markdown", + "Name": "Unarchiving Documents", + "DiscussionId": "a01cdd59-257e-4a0e-8ec6-07d7d7d3d239", + "Mappings": [] + } +] diff --git a/Documentation/7.0/Raven.Documentation.Pages/server/configuration/indexing-configuration.markdown b/Documentation/7.0/Raven.Documentation.Pages/server/configuration/indexing-configuration.markdown index 864c4cfe91..f42fad893c 100644 --- a/Documentation/7.0/Raven.Documentation.Pages/server/configuration/indexing-configuration.markdown +++ b/Documentation/7.0/Raven.Documentation.Pages/server/configuration/indexing-configuration.markdown @@ -244,9 +244,12 @@ EXPERT ONLY: {PANEL: Indexing.Auto.ArchivedDataProcessingBehavior} -Set the default deployment mode for static indexes. +The default processing behavior for archived documents in auto-indexes. -- **Type**: `enum IndexDeploymentModeArchivedDataProcessingBehavior` (`ExcludeArchived`, `IncludeArchived`, `ArchivedOnly`) +- **Type**: `enum ArchivedDataProcessingBehavior`: + * `ExcludeArchived`: only non-archived documents are processed by the index. + * `IncludeArchived`: both archived and non-archived documents are processed by the index. + * `ArchivedOnly`: only archived documents are processed by the index. - **Default**: `ExcludeArchived` - **Scope**: Server-wide, or per database @@ -385,9 +388,16 @@ Allow to open an index without checking if current Database ID matched the one f {PANEL: Indexing.Static.ArchivedDataProcessingBehavior} -Set the default deployment mode for static indexes. +* Set the default processing behavior for archived documents in static indexes. +* This setting applies only to static indexes that use _Documents_ as their data source. + It does not apply to indexes based on _Time Series_ or _Counters_, which default to `IncludeArchived`. + +--- -- **Type**: `enum IndexDeploymentModeArchivedDataProcessingBehavior` (`ExcludeArchived`, `IncludeArchived`, `ArchivedOnly`) +- **Type**: `enum ArchivedDataProcessingBehavior`: + * `ExcludeArchived`: only non-archived documents are processed by the index. + * `IncludeArchived`: both archived and non-archived documents are processed by the index. + * `ArchivedOnly`: only archived documents are processed by the index. - **Default**: `ExcludeArchived` - **Scope**: Server-wide, or per database diff --git a/Documentation/7.0/Raven.Documentation.Pages/server/extensions/.docs.json b/Documentation/7.0/Raven.Documentation.Pages/server/extensions/.docs.json index 5c306d031c..8066b07990 100644 --- a/Documentation/7.0/Raven.Documentation.Pages/server/extensions/.docs.json +++ b/Documentation/7.0/Raven.Documentation.Pages/server/extensions/.docs.json @@ -15,11 +15,5 @@ "Name": "Document Refresh", "DiscussionId": "b976732a-3da7-48e5-a454-4f34b8f716f9", "Mappings": [] - }, - { - "Path": "archival.markdown", - "Name": "Data Archival", - "DiscussionId": "b976732a-3da7-48e5-a454-4f34b8f716f9", - "Mappings": [] } ] diff --git a/Documentation/7.0/Raven.Documentation.Pages/server/kb/javascript-engine.markdown b/Documentation/7.0/Raven.Documentation.Pages/server/kb/javascript-engine.markdown index 2fc151b3c6..c8025f8e65 100644 --- a/Documentation/7.0/Raven.Documentation.Pages/server/kb/javascript-engine.markdown +++ b/Documentation/7.0/Raven.Documentation.Pages/server/kb/javascript-engine.markdown @@ -58,13 +58,15 @@ RavenDB provides the following set of predefined functions: |-------------------------------------------------------------|--------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | **id(document)** | `string` | Returns the ID of the specified document[[ex]](../../client-api/operations/patching/set-based#updating-by-document-id). | | **load(documentId)** | `object` | Returns the document with the given ID.
Used in [patching](../../client-api/operations/patching/single-document#loading-documents-in-a-script) or [ETL scripts](../../server/ongoing-tasks/etl/basics#transform). | -| **load(documentId, collectionName)** | `object` | Returns the document with the given ID.
Used in [JavaScript indexes](../../indexes/javascript-indexes). | +| **load(documentId, collectionName)** | `object` | Returns the document with the given ID.
Used in [JavaScript indexes](../../indexes/javascript-indexes). | | **loadPath(document, pathString)** | `object` | Returns document(s) based on IDs found within the specified `pathString` in the given document.
The `pathString` can be in a simple _Foo.Bar_ form, in which case a single document is returned. A path like _Foo.Bars[].Buzz_ can return an array of documents. | | **getMetadata(document)** | `object` | Returns the metadata of the specified document, including properties like `ChangeVector`, `ID`, and `LastModified`. | | **lastModified(document)** | `number` | Returns the number of milliseconds elapsed since the last modification time (UTC) of the specified document. | | **include(documentId)** | `Task` | Used in RQL [queries](../../client-api/session/querying/what-is-rql) to include the document with the specified ID with the results. | | **put(documentId, document, [optional]changeVectorString)** | `Task` | Creates or updates a document with the specified ID.
To generate a new document ID, you can use the _"collectionPrefix/[Server-Side ID](../../server/kb/document-identifier-generation#strategy--2)"_ notation[[ex]](../../client-api/operations/patching/single-document#add-document).
This function can also clone an existing document.
Note: attachments & counters will not be included in the clone[[ex]](../../client-api/operations/patching/single-document#clone-document). Used in patching. | | **del(documentId)** | `void` | Deletes the document with the specified ID.
Used in [patching](../../client-api/operations/patching/set-based#updating-a-collection-name). | +| **archived.archiveAt(document, dateString)** | `void` | Schedules the specified document to be archived at the specified `dateString`.
Used in [patching](../../data-archival/schedule-document-archiving#schedule-multiple-documents-for-archiving---from-the-client-api). | +| **archived.unarchive(document)** | `void` | Unarchives the specified document.
Used in [patching](../../data-archival/unarchiving-documents#unarchive-all-documents-in-a-collection---from-the-client-api). | --- diff --git a/Documentation/7.0/Raven.Documentation.Pages/studio/database/settings/.docs.json b/Documentation/7.0/Raven.Documentation.Pages/studio/database/settings/.docs.json index 5c5d6e0894..14a9b454eb 100644 --- a/Documentation/7.0/Raven.Documentation.Pages/studio/database/settings/.docs.json +++ b/Documentation/7.0/Raven.Documentation.Pages/studio/database/settings/.docs.json @@ -49,7 +49,7 @@ }, { "Path": "document-archival.markdown", - "Name": "Document Archival", + "Name": "Data Archival", "DiscussionId": "c5633531-4b5d-4816-97d4-82d351f3f6a2", "Mappings": [] }, diff --git a/Documentation/7.1/Raven.Documentation.Pages/.docs.json b/Documentation/7.1/Raven.Documentation.Pages/.docs.json index 9e077b2dbe..c6eb52122c 100644 --- a/Documentation/7.1/Raven.Documentation.Pages/.docs.json +++ b/Documentation/7.1/Raven.Documentation.Pages/.docs.json @@ -19,6 +19,11 @@ "Name": "Document Extensions", "Mappings": [] }, + { + "Path": "/data-archival", + "Name": "Data Archival", + "Mappings": [] + }, { "Path": "/server", "Name": "Server", diff --git a/Documentation/7.1/Raven.Documentation.Pages/data-archival/.docs.json b/Documentation/7.1/Raven.Documentation.Pages/data-archival/.docs.json new file mode 100644 index 0000000000..3bc90f6a55 --- /dev/null +++ b/Documentation/7.1/Raven.Documentation.Pages/data-archival/.docs.json @@ -0,0 +1,32 @@ +[ + { + "Path": "overview.markdown", + "Name": "Overview", + "DiscussionId": "28bb68b0-3af7-46b3-ad4c-e34b7e81565d", + "Mappings": [] + }, + { + "Path": "enable-data-archiving.markdown", + "Name": "Enable Data Archiving", + "DiscussionId": "e5eeaaa7-bb98-4ae7-9cd1-75cb300738e7", + "Mappings": [] + }, + { + "Path": "schedule-document-archiving.markdown", + "Name": "Schedule Document Archiving", + "DiscussionId": "ec35fa9a-834a-48b0-96c3-d67e1648cbae", + "Mappings": [] + }, + { + "Path": "archived-documents-and-other-features.markdown", + "Name": "Archived Documents and Other Features", + "DiscussionId": "977c8625-ad1d-4dd9-8b29-eb92037957f8", + "Mappings": [] + }, + { + "Path": "unarchiving-documents.markdown", + "Name": "Unarchiving Documents", + "DiscussionId": "a01cdd59-257e-4a0e-8ec6-07d7d7d3d239", + "Mappings": [] + } +] diff --git a/Documentation/7.1/Raven.Documentation.Pages/server/extensions/.docs.json b/Documentation/7.1/Raven.Documentation.Pages/server/extensions/.docs.json index 5c306d031c..8066b07990 100644 --- a/Documentation/7.1/Raven.Documentation.Pages/server/extensions/.docs.json +++ b/Documentation/7.1/Raven.Documentation.Pages/server/extensions/.docs.json @@ -15,11 +15,5 @@ "Name": "Document Refresh", "DiscussionId": "b976732a-3da7-48e5-a454-4f34b8f716f9", "Mappings": [] - }, - { - "Path": "archival.markdown", - "Name": "Data Archival", - "DiscussionId": "b976732a-3da7-48e5-a454-4f34b8f716f9", - "Mappings": [] } ] diff --git a/Documentation/7.1/Raven.Documentation.Pages/studio/database/settings/.docs.json b/Documentation/7.1/Raven.Documentation.Pages/studio/database/settings/.docs.json index 5c5d6e0894..14a9b454eb 100644 --- a/Documentation/7.1/Raven.Documentation.Pages/studio/database/settings/.docs.json +++ b/Documentation/7.1/Raven.Documentation.Pages/studio/database/settings/.docs.json @@ -49,7 +49,7 @@ }, { "Path": "document-archival.markdown", - "Name": "Document Archival", + "Name": "Data Archival", "DiscussionId": "c5633531-4b5d-4816-97d4-82d351f3f6a2", "Mappings": [] }, diff --git a/Raven.Documentation.Parser/Data/Category.cs b/Raven.Documentation.Parser/Data/Category.cs index cd46ab01f1..965a323141 100644 --- a/Raven.Documentation.Parser/Data/Category.cs +++ b/Raven.Documentation.Parser/Data/Category.cs @@ -84,6 +84,10 @@ public enum Category [Description("Document Extensions")] DocumentExtensions, + [Prefix("data-archival")] + [Description("Data Archival")] + DataArchival, + [Prefix("ai-integration")] [Description("AI Integration")] AiIntegration