Class LiveIndexWriterConfig
- Direct Known Subclasses:
IndexWriterConfig
IndexWriter with few setters for settings that can be
changed on an IndexWriter instance "live".- Since:
- 4.0
-
Field Summary
FieldsModifier and TypeFieldDescriptionprotected booleanif an indexing thread should check for pending flushes on update in order to help out on a full flushprotected CodecCodecused to write new segments.protected IndexCommitIndexCommitthatIndexWriteris opened on.protected booleanTrue if calls toIndexWriter.close()should first do a commit.protected intCompatibility version to use for this index.protected IndexDeletionPolicyIndexDeletionPolicycontrolling when commit points are deleted.protected IndexWriterEventListenerThe IndexWriter event listener to record key events *protected org.apache.lucene.index.FlushPolicyFlushPolicyto control when segments are flushed.protected SortThe sort order to use to write merged segments.The field names involved in the index sortprotected InfoStreamInfoStreamfor debugging messages.protected Comparator<LeafReader> The comparator for sorting leaf readers.protected longAmount of time to wait for merges returned by MergePolicy.findFullFlushMerges(...)protected MergePolicyMergePolicyfor selecting merges.protected MergeSchedulerMergeSchedulerto use for running merges.protected IndexWriterConfig.OpenModeIndexWriterConfig.OpenModethatIndexWriteris opened with.protected Stringparent document fieldprotected intSets the hard upper bound on RAM usage for a single segment, after which the segment is forced to flush.protected booleanTrue if readers should be pooled.protected SimilaritySimilarityto use when encoding norms.protected Stringsoft deletes fieldprotected booleanTrue if segment flushes should use compound file format -
Method Summary
Modifier and TypeMethodDescriptionReturns the default analyzer to use for indexing documents.getCodec()Returns the currentCodec.booleanReturnstrueifIndexWriter.close()should first commit before closing.Returns theIndexCommitas specified inIndexWriterConfig.setIndexCommit(IndexCommit)or the default,nullwhich specifies to open the latest index commit point.intReturn the compatibility version to use for this index.Returns theIndexDeletionPolicyspecified inIndexWriterConfig.setIndexDeletionPolicy(IndexDeletionPolicy)or the defaultKeepOnlyLastCommitDeletionPolicy/Get the index-timeSortorder, applied to all (flushed and merged) segments.Returns the field names involved in the index sortReturns the IndexWriterEventListener callback that tracks the key IndexWriter operations.ReturnsInfoStreamused for debugging.Returns a comparator for sorting leaf readers.intReturns the number of buffered added documents that will trigger a flush if enabled.longExpert: return the amount of time to wait for merges returned by by MergePolicy.findFullFlushMerges(...).Returns the current merged segment warmer.Returns the current MergePolicy in use by this writer.Returns theMergeSchedulerthat was set byIndexWriterConfig.setMergeScheduler(MergeScheduler).Returns theIndexWriterConfig.OpenModeset byIndexWriterConfig.setOpenMode(OpenMode).Returns the parent document field name if configured.doubleReturns the value set bysetRAMBufferSizeMB(double)if enabled.intReturns the max amount of memory eachDocumentsWriterPerThreadcan consume until forcefully flushed.booleanReturnstrueifIndexWritershould pool readers even ifDirectoryReader.open(IndexWriter)has not been called.Expert: returns theSimilarityimplementation used by thisIndexWriter.Returns the soft deletes field ornullif soft-deletes are disabled.booleanReturnstrueiff theIndexWriterpacks newly written segments in a compound file.booleanExpert: Returns if indexing threads check for pending flushes on update in order to help our flushing indexing buffers to disksetCheckPendingFlushUpdate(boolean checkPendingFlushOnUpdate) Expert: sets if indexing threads check for pending flushes on update in order to help our flushing indexing buffers to disk.setMaxBufferedDocs(int maxBufferedDocs) Determines the minimal number of documents required before the buffered in-memory documents are flushed as a new Segment.setMergedSegmentWarmer(IndexWriter.IndexReaderWarmer mergeSegmentWarmer) Set the merged segment warmer.setMergePolicy(MergePolicy mergePolicy) Expert:MergePolicyis invoked whenever there are changes to the segments in the index.setRAMBufferSizeMB(double ramBufferSizeMB) Determines the amount of RAM that may be used for buffering added documents and deletions before they are flushed to the Directory.setUseCompoundFile(boolean useCompoundFile) Sets if theIndexWritershould pack newly written segments in a compound file.toString()
-
Field Details
-
delPolicy
IndexDeletionPolicycontrolling when commit points are deleted. -
commit
IndexCommitthatIndexWriteris opened on. -
openMode
IndexWriterConfig.OpenModethatIndexWriteris opened with. -
createdVersionMajor
protected int createdVersionMajorCompatibility version to use for this index. -
similarity
Similarityto use when encoding norms. -
mergeScheduler
MergeSchedulerto use for running merges. -
codec
Codecused to write new segments. -
infoStream
InfoStreamfor debugging messages. -
mergePolicy
MergePolicyfor selecting merges. -
readerPooling
protected volatile boolean readerPoolingTrue if readers should be pooled. -
flushPolicy
protected volatile org.apache.lucene.index.FlushPolicy flushPolicyFlushPolicyto control when segments are flushed. -
perThreadHardLimitMB
protected volatile int perThreadHardLimitMBSets the hard upper bound on RAM usage for a single segment, after which the segment is forced to flush. -
useCompoundFile
protected volatile boolean useCompoundFileTrue if segment flushes should use compound file format -
commitOnClose
protected boolean commitOnCloseTrue if calls toIndexWriter.close()should first do a commit. -
indexSort
The sort order to use to write merged segments. -
leafSorter
The comparator for sorting leaf readers. -
indexSortFields
The field names involved in the index sort -
parentField
parent document field -
checkPendingFlushOnUpdate
protected volatile boolean checkPendingFlushOnUpdateif an indexing thread should check for pending flushes on update in order to help out on a full flush -
softDeletesField
soft deletes field -
maxFullFlushMergeWaitMillis
protected volatile long maxFullFlushMergeWaitMillisAmount of time to wait for merges returned by MergePolicy.findFullFlushMerges(...) -
eventListener
The IndexWriter event listener to record key events *
-
-
Method Details
-
getAnalyzer
Returns the default analyzer to use for indexing documents. -
setRAMBufferSizeMB
Determines the amount of RAM that may be used for buffering added documents and deletions before they are flushed to the Directory. Generally for faster indexing performance it's best to flush by RAM usage instead of document count and use as large a RAM buffer as you can.When this is set, the writer will flush whenever buffered documents and deletions use this much RAM. Pass in
IndexWriterConfig.DISABLE_AUTO_FLUSHto prevent triggering a flush due to RAM usage. Note that if flushing by document count is also enabled, then the flush will be triggered by whichever comes first.The maximum RAM limit is inherently determined by the JVMs available memory. Yet, an
IndexWritersession can consume a significantly larger amount of memory than the given RAM limit since this limit is just an indicator when to flush memory resident documents to the Directory. Flushes are likely happen concurrently while other threads adding documents to the writer. For application stability the available memory in the JVM should be significantly larger than the RAM buffer used for indexing.NOTE: the account of RAM usage for pending deletions is only approximate. Specifically, if you delete by Query, Lucene currently has no way to measure the RAM usage of individual Queries so the accounting will under-estimate and you should compensate by either calling commit() or refresh() periodically yourself.
NOTE: It's not guaranteed that all memory resident documents are flushed once this limit is exceeded. Depending on the configured
FlushPolicyonly a subset of the buffered documents are flushed and therefore only parts of the RAM buffer is released.The default value is
IndexWriterConfig.DEFAULT_RAM_BUFFER_SIZE_MB.Takes effect immediately, but only the next time a document is added, updated or deleted.
- Throws:
IllegalArgumentException- if ramBufferSize is enabled but non-positive, or it disables ramBufferSize when maxBufferedDocs is already disabled- See Also:
-
getRAMBufferSizeMB
public double getRAMBufferSizeMB()Returns the value set bysetRAMBufferSizeMB(double)if enabled. -
setMaxBufferedDocs
Determines the minimal number of documents required before the buffered in-memory documents are flushed as a new Segment. Large values generally give faster indexing.When this is set, the writer will flush every maxBufferedDocs added documents. Pass in
IndexWriterConfig.DISABLE_AUTO_FLUSHto prevent triggering a flush due to number of buffered documents. Note that if flushing by RAM usage is also enabled, then the flush will be triggered by whichever comes first.Disabled by default (writer flushes by RAM usage).
Takes effect immediately, but only the next time a document is added, updated or deleted.
- Throws:
IllegalArgumentException- if maxBufferedDocs is enabled but smaller than 2, or it disables maxBufferedDocs when ramBufferSize is already disabled- See Also:
-
getMaxBufferedDocs
public int getMaxBufferedDocs()Returns the number of buffered added documents that will trigger a flush if enabled.- See Also:
-
setMergePolicy
Expert:MergePolicyis invoked whenever there are changes to the segments in the index. Its role is to select which merges to do, if any, and return aMergePolicy.MergeSpecificationdescribing the merges. It also selects merges to do for forceMerge.Takes effect on subsequent merge selections. Any merges in flight or any merges already registered by the previous
MergePolicyare not affected. -
setMergedSegmentWarmer
public LiveIndexWriterConfig setMergedSegmentWarmer(IndexWriter.IndexReaderWarmer mergeSegmentWarmer) Set the merged segment warmer. SeeIndexWriter.IndexReaderWarmer.Takes effect on the next merge.
-
getMergedSegmentWarmer
Returns the current merged segment warmer. SeeIndexWriter.IndexReaderWarmer. -
getOpenMode
Returns theIndexWriterConfig.OpenModeset byIndexWriterConfig.setOpenMode(OpenMode). -
getIndexCreatedVersionMajor
public int getIndexCreatedVersionMajor()Return the compatibility version to use for this index.- See Also:
-
getIndexDeletionPolicy
Returns theIndexDeletionPolicyspecified inIndexWriterConfig.setIndexDeletionPolicy(IndexDeletionPolicy)or the defaultKeepOnlyLastCommitDeletionPolicy/ -
getIndexCommit
Returns theIndexCommitas specified inIndexWriterConfig.setIndexCommit(IndexCommit)or the default,nullwhich specifies to open the latest index commit point. -
getSimilarity
Expert: returns theSimilarityimplementation used by thisIndexWriter. -
getMergeScheduler
Returns theMergeSchedulerthat was set byIndexWriterConfig.setMergeScheduler(MergeScheduler). -
getCodec
Returns the currentCodec. -
getMergePolicy
Returns the current MergePolicy in use by this writer.- See Also:
-
getReaderPooling
public boolean getReaderPooling()ReturnstrueifIndexWritershould pool readers even ifDirectoryReader.open(IndexWriter)has not been called. -
getRAMPerThreadHardLimitMB
public int getRAMPerThreadHardLimitMB()Returns the max amount of memory eachDocumentsWriterPerThreadcan consume until forcefully flushed.- See Also:
-
getInfoStream
ReturnsInfoStreamused for debugging.- See Also:
-
setUseCompoundFile
Sets if theIndexWritershould pack newly written segments in a compound file. Default istrue.Use
falsefor batch indexing with very large ram buffer settings.Note: To control compound file usage during segment merges see
MergePolicy.setNoCFSRatio(double)andMergePolicy.setMaxCFSSegmentSizeMB(double). This setting only applies to newly created segments. -
getUseCompoundFile
public boolean getUseCompoundFile() -
getCommitOnClose
public boolean getCommitOnClose()ReturnstrueifIndexWriter.close()should first commit before closing. -
getIndexSort
Get the index-timeSortorder, applied to all (flushed and merged) segments. -
getIndexSortFields
Returns the field names involved in the index sort -
getLeafSorter
Returns a comparator for sorting leaf readers. If notnull, this comparator is used to sort leaf readers withinDirectoryReaderopened from theIndexWriterof this configuration.- Returns:
- a comparator for sorting leaf readers
-
isCheckPendingFlushOnUpdate
public boolean isCheckPendingFlushOnUpdate()Expert: Returns if indexing threads check for pending flushes on update in order to help our flushing indexing buffers to disk- WARNING: This API is experimental and might change in incompatible ways in the next release.
-
setCheckPendingFlushUpdate
Expert: sets if indexing threads check for pending flushes on update in order to help our flushing indexing buffers to disk. As a consequence, threads callingDirectoryReader.openIfChanged(DirectoryReader, IndexWriter)orIndexWriter.flush()will be the only thread writing segments to disk unless flushes are falling behind. If indexing is stalled due to too many pending flushes indexing threads will help our writing pending segment flushes to disk.- WARNING: This API is experimental and might change in incompatible ways in the next release.
-
getSoftDeletesField
Returns the soft deletes field ornullif soft-deletes are disabled. SeeIndexWriterConfig.setSoftDeletesField(String)for details. -
getMaxFullFlushMergeWaitMillis
public long getMaxFullFlushMergeWaitMillis()Expert: return the amount of time to wait for merges returned by by MergePolicy.findFullFlushMerges(...). If this time is reached, we proceed with the commit based on segments merged up to that point. The merges are not cancelled, and may still run to completion independent of the commit. -
getIndexWriterEventListener
Returns the IndexWriterEventListener callback that tracks the key IndexWriter operations. -
getParentField
Returns the parent document field name if configured. -
toString
-