Skip to content
Snippets Groups Projects
Commit d519905b authored by Philippe Canal's avatar Philippe Canal
Browse files

Skip compression of the top TKeys in the TMemFile by default.

To save space, the compression can be re-enabled by TBuffferMerger::SetCompressTemporaryKeys.

The TBaskets are still compressed and thus the end result is unchanged.
parent 06f6f863
No related branches found
No related tags found
No related merge requests found
......@@ -110,6 +110,30 @@ public:
return fMerger.GetNotrees();
}
/** Indicates that the temporary keys (corresponding to the object held by the directories
* of the TMemFile) should be compressed or not. Those object are stored in the TMemFile
* (and thus possibly compressed) when a thread push its data forward (by calling
* TBufferMergerFile::Write) and the queue is being processed by another.
* Once the TMemFile is picked (by any thread to be merged), *after* taking the
* TBufferMerger::fMergeMutex, those object are read back (and thus possibly uncompressed)
* and then used by merging.
* In order word, the compression of those objects/keys is only usefull to reduce the size
* in memory (of the TMemFile) and does not affect (at all) the compression factor of the end
* result.
*/
void SetCompressTemporaryKeys(Bool_t request_compression = true)
{
fCompressTemporaryKeys = request_compression;
}
/** Returns whether to compressed the TKey in the TMemFile for the object held by
* the TDirectories. See TBufferMerger::SetCompressTemporaryKeys for more details.
*/
Bool_t GetCompressTemporaryKeys() const
{
return fCompressTemporaryKeys;
}
friend class TBufferMergerFile;
private:
......@@ -130,6 +154,7 @@ private:
void Push(TBufferFile *buffer);
bool TryMerge(TBufferMergerFile *memfile);
bool fCompressTemporaryKeys{false}; //< Enable compression of the TKeys in the TMemFile (save memory at the expense of time, end result is unchanged)
size_t fAutoSave{0}; //< AutoSave only every fAutoSave bytes
std::atomic<size_t> fBuffered{0}; //< Number of bytes currently buffered
TFileMerger fMerger{false, false}; //< TFileMerger used to merge all buffers
......
......@@ -131,7 +131,7 @@ void TBufferMerger::MergeImpl()
queue.pop();
}
fMerger.PartialMerge(TFileMerger::kAll | TFileMerger::kIncremental | TFileMerger::kDelayWrite);
fMerger.PartialMerge(TFileMerger::kAll | TFileMerger::kIncremental | TFileMerger::kDelayWrite | TFileMerger::kKeepCompression);
fMerger.Reset();
}
......
......@@ -42,7 +42,11 @@ Int_t TBufferMergerFile::Write(const char *name, Int_t opt, Int_t bufsize)
return 0;
}
auto oldCompLevel = GetCompressionLevel();
if (!fMerger.GetCompressTemporaryKeys())
SetCompressionLevel(0);
Int_t nbytes = TMemFile::Write(name, opt, bufsize);
SetCompressionLevel(oldCompLevel);
if (nbytes) {
TBufferFile *buffer = new TBufferFile(TBuffer::kWrite, GetSize());
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment