View Issue Details

IDProjectCategoryView StatusLast Update
0003514SymmetricDSImprovementpublic2018-08-09 15:03
ReporterelongAssigned Toelong 
Prioritynormal 
Status closedResolutionfixed 
Product Version3.9.0 
Target Version3.9.7Fixed in Version3.9.7 
Summary0003514: Improve purge performance of sym_data and sym_data_event
DescriptionThe delete statements used on sym_data and sym_data_event use joins to verify the rows are part of a batch in OK status, which can be slow on some systems. Use a first pass on the tables that looks at all rows before the smallest data_id (for sym_data) or batch_id (for sym_data_event) associated with outstanding (not OK) batches. Once the safe range for purging is determined, it can delete without any joins. Afterwards, the normal purge routines run, but on a health system (one without any or very few outstanding batches), there shouldn't any data left to purge.

This feature is disabled by default for now. Enable with job.purge.first.pass=true parameter.

Relationships

related to 0001445 closed improve performance of purge queries 
related to 0003670 closedelong Improve purge performance of sym_data and sym_data_event 

Activities

There are no notes attached to this issue.

Related Changesets

SymmetricDS: 3.9 6dc7ee2d

2018-04-09 15:24:22

admin

Details Diff
0003514: Improve purge performance of sym_data and sym_data_event
0003514
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/common/ParameterConstants.java Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/service/impl/PurgeService.java Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/service/impl/PurgeServiceSqlMap.java Diff File
mod - symmetric-core/src/main/resources/symmetric-default.properties Diff File

SymmetricDS: 3.9 2ad90c14

2018-08-09 09:05:11

admin

Details Diff
0003514: add outgoing batch first pass, honor retention time on sym_data
0003514
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/service/impl/PurgeService.java Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/service/impl/PurgeServiceSqlMap.java Diff File

Issue History

Date Modified Username Field Change
2018-04-09 15:22 elong New Issue
2018-04-09 15:22 elong Status new => assigned
2018-04-09 15:22 elong Assigned To => elong
2018-04-09 15:23 elong Relationship added related to 0001445
2018-04-09 15:25 elong Status assigned => resolved
2018-04-09 15:25 elong Resolution open => fixed
2018-04-09 15:25 elong Fixed in Version => 3.9.7
2018-04-09 15:25 elong Description Updated View Revisions
2018-04-09 16:00 admin Changeset attached => SymmetricDS 3.9 6dc7ee2d
2018-05-08 16:09 mmichalek Status resolved => closed
2018-08-09 10:00 admin Changeset attached => SymmetricDS 3.9 2ad90c14
2018-08-09 15:03 elong Relationship added related to 0003670