View Issue Details

IDProjectCategoryView StatusLast Update
0005825SymmetricDSBugpublic2023-05-08 19:00
Reporteremiller Assigned Toemiller  
Prioritynormal 
Status closedResolutionfixed 
Product Version3.13.0 
Target Version3.13.12Fixed in Version3.13.12 
Summary0005825: Data can become misaligned with column names during extraction after a column gets renamed
DescriptionIf a change is captured for a table and then one of its columns gets renamed and its triggers get rebuilt before extraction occurs, the data can become misaligned with the column names. This can cause extraction to fail or data to be put in the wrong columns. Also, if the final column in a table gets renamed, then its data would not be included in the extracted batch.

This occurs because the data extractor gets the table definition with the new column name from the database and searches using the old column name that it got from the old row in sym_trigger_hist. As a result, the extractor fails to find the column, so it includes neither the old column nor the new column in its list of columns. The correct behavior would be to include the old column in this list so that the batch contains the old column name and correctly lines it up with the captured value.

The following is an example of an extraction failure that occurred as a result of this issue, where an empty string value got treated as an integer.

Failed to extract batch [batch ID] StackTraceKey.init [RuntimeException:3890067872] java.lang.RuntimeException: Could not convert a value of for column [column name] of mapped type TINYINT jdbc type tinyint (-6)
    at org.jumpmind.db.platform.AbstractDatabasePlatform.getObjectValues(AbstractDatabasePlatform.java:404)
    at org.jumpmind.db.platform.AbstractDatabasePlatform.getObjectValues(AbstractDatabasePlatform.java:385)
    at org.jumpmind.symmetric.io.data.reader.ExtractDataReader.enhanceWithLobsFromSourceIfNeeded(ExtractDataReader.java:183)
    at org.jumpmind.symmetric.io.data.reader.ExtractDataReader.nextData(ExtractDataReader.java:142)
    at org.jumpmind.symmetric.io.data.DataProcessor.forEachDataInTable(DataProcessor.java:199)
    at org.jumpmind.symmetric.io.data.DataProcessor.forEachTableInBatch(DataProcessor.java:178)
    at org.jumpmind.symmetric.io.data.DataProcessor.process(DataProcessor.java:124)
    at org.jumpmind.symmetric.service.impl.DataExtractorService.extractOutgoingBatch(DataExtractorService.java:1091)
    at org.jumpmind.symmetric.service.impl.DataExtractorService.extractBatch(DataExtractorService.java:967)
    at org.jumpmind.symmetric.service.impl.DataExtractorService.lambda$extract$1(DataExtractorService.java:723)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.lang.NumberFormatException: Zero length BigInteger
    at java.base/java.math.BigInteger.<init>(BigInteger.java:484)
    at java.base/java.math.BigInteger.<init>(BigInteger.java:672)
    at org.jumpmind.db.platform.AbstractDatabasePlatform.parseInteger(AbstractDatabasePlatform.java:518)
    at org.jumpmind.db.platform.AbstractDatabasePlatform.getObjectValue(AbstractDatabasePlatform.java:444)
    at org.jumpmind.db.platform.AbstractDatabasePlatform.getObjectValues(AbstractDatabasePlatform.java:397)
    ... 13 more
Steps To Reproduce1) Configure a table for replication
2) Stop the routing job
3) Insert a row into the table
4) Rename one of the table's columns
5) Rebuild the triggers for the table
6) Run the routing job
TagsNo tags attached.

Relationships

related to 0005824 closedemiller Data can become misaligned with column names during extraction after a column gets renamed 
related to 0005826 closedemiller Data can become misaligned with column names during extraction after a column gets renamed 

Activities

There are no notes attached to this issue.

Related Changesets

SymmetricDS: 3.13 5a05c39c

2023-05-08 18:59:28

evan-miller-jumpmind

Details Diff
0005825: Fixed data becoming misaligned with column names during extraction after a column gets renamed Affected Issues
0005825
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/db/AbstractTriggerTemplate.java Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/extract/ColumnsAccordingToTriggerHistory.java Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/extract/SelectFromSymDataSource.java Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/extract/SelectFromTableSource.java Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/io/DbCompare.java Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/route/AuditTableDataRouter.java Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/service/impl/DataService.java Diff File
mod - symmetric-db/src/main/java/org/jumpmind/db/model/Table.java Diff File
mod - symmetric-db/src/main/java/org/jumpmind/db/platform/AbstractDatabasePlatform.java Diff File
mod - symmetric-io/src/main/java/org/jumpmind/symmetric/io/data/writer/DefaultDatabaseWriter.java Diff File
mod - symmetric-io/src/main/java/org/jumpmind/symmetric/io/data/writer/StructureDataWriter.java Diff File

Issue History

Date Modified Username Field Change
2023-05-05 20:22 emiller New Issue
2023-05-05 20:22 emiller Status new => assigned
2023-05-05 20:22 emiller Assigned To => emiller
2023-05-05 20:22 emiller Issue generated from: 0005824
2023-05-05 20:22 emiller Relationship added related to 0005824
2023-05-05 20:23 emiller Issue cloned: 0005826
2023-05-05 20:23 emiller Relationship added related to 0005826
2023-05-08 12:13 emiller Target Version => 3.13.12
2023-05-08 19:00 Changeset attached => SymmetricDS 3.13 5a05c39c
2023-05-08 19:00 emiller Status assigned => resolved
2023-05-08 19:00 emiller Resolution open => fixed
2023-05-08 19:00 emiller Fixed in Version => 3.13.12
2023-05-12 19:43 admin Status resolved => closed