View Issue Details

IDProjectCategoryView StatusLast Update
0005231SymmetricDS ProImprovementpublic2022-09-30 21:19
Reporterelong Assigned Toelong  
Prioritynormal 
Status closedResolutionfixed 
Product Version3.13.0 
Target Version3.13.4Fixed in Version3.13.4 
Summary0005231: Support snapshot file improvements
DescriptionMake some improvements to support snapshot, based on trouble we've seen on support:

- Get table definitions for tables that are loaded into, supporting variables and transforms
- Sort NOT OK batches by batch_id or create_time, OK batches sorted descending
- Add batch summary files, grouping by node_id and channel_id
- Add log summary file, based on errors and warnings from log
- Limit number of log files to current log plus 3 more
- Filter parameters that should be secret
- Use double quote escaping in CSV files so they open easily in spreadsheet tools
- Limit number of files in directory listings
- Separate directory listing of software from staging listing
- Staging directory listing for just current node, not multi-homed nodes
- Add parameters for limits on number of files, batches, and statistics
- Add purging of old snapshot files, with parameter for retention period
- Get staging directory location from staging manager, in case user has relocated it
- Only record node channel and host stats if they are non-zero to reduce size of tables
- Use of try-with-resources
- Logging of each step
Tagsstatistics

Relationships

related to 0005489 closedelong SymmetricDS Backport support snapshot improvements 

Activities

There are no notes attached to this issue.

Related Changesets

SymmetricDS: 3.13 b9dff9a3

2022-02-25 20:27:08

admin

Details Diff
0005231: Support snapshot file improvements Affected Issues
0005231
mod - symmetric-client/src/main/java/org/jumpmind/symmetric/util/SnapshotUtil.java Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/common/ParameterConstants.java Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/extract/ColumnsAccordingToTriggerHistory.java Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/service/ITriggerRouterService.java Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/service/impl/TriggerRouterService.java Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/service/impl/TriggerRouterServiceSqlMap.java Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/statistic/ChannelStats.java Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/statistic/HostStats.java Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/statistic/StatisticManager.java Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/util/LogSummaryAppenderUtils.java Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/util/SymmetricUtils.java Diff File
mod - symmetric-core/src/main/resources/symmetric-default.properties Diff File
mod - symmetric-io/src/main/java/org/jumpmind/symmetric/io/stage/IStagingManager.java Diff File
mod - symmetric-io/src/main/java/org/jumpmind/symmetric/io/stage/StagingManager.java Diff File
mod - symmetric-util/src/main/java/org/jumpmind/util/Log4j2Helper.java Diff File

Issue History

Date Modified Username Field Change
2022-02-25 20:25 elong New Issue
2022-02-25 20:25 elong Status new => assigned
2022-02-25 20:25 elong Assigned To => elong
2022-02-25 20:25 elong Tag Attached: statistics
2022-02-25 20:39 elong Status assigned => resolved
2022-02-25 20:39 elong Resolution open => fixed
2022-02-25 20:39 elong Fixed in Version => 3.13.4
2022-02-25 21:00 admin Changeset attached => SymmetricDS 3.13 b9dff9a3
2022-04-11 19:29 admin Status resolved => closed
2022-09-30 21:19 elong Relationship added related to 0005489