View Issue Details

IDProjectCategoryView StatusLast Update
0003972SymmetricDSBugpublic2020-05-26 18:48
Reporterkraynopp Assigned Toelong  
Priorityhigh 
Status closedResolutionfixed 
Product Version3.10.2 
Target Version3.12.0Fixed in Version3.12.0 
Summary0003972: Tables become out of sync due incorrect conflict detection (and/or invalid conflict resolution)
DescriptionTables become out of sync if different nodes update different fields of the same row simultaneously.
Steps To ReproduceI have two nodes - master and slave. In case of conflict master node has priority.
1. Make two installations of SymmetricDS on the same PC - master and slave.
2. copy files from attachement into conf and engines direcories (master - in master installation, slave - in slave installation).
3. Copy create_sample.xml and insert_sample.sql into bin directory of master installation.
4. In bin directory of master installation execute:
  ./dbimport --format XML --alter-case create_sample.xml
  ./symadmin create-sym-tables
  ./dbimport insert_sample.sql
5. start both SymmetricDS instances.
6. execute
  ./symadmin reload-node 001
  from master bin directory.
7. wait. On both nodes table testtable must appear:
ID | DATA1 | DATA2
1 | data | data
8. now execute statement
update testtable set data1='master';
on master node and
update testtable set data2='slave';
on slave node.
After syncronization execute select * from testtable on both nodes. You will see on master node:
ID | DATA1 | DATA2
1 | master | data
and on slave node:
ID | DATA1 | DATA2
1 | master | slave

So the table testtable is out of sync now. In log file you will see conflict detection (however there is no conflict because of different fields is updated).
Additional InformationI have made some little investigation and discovered that this bug is introduced in version 3.9.5 of SymmetricDS. In versions 3.9.4 and earlier changes are merged successfully and statement select * from testtable returns the same result on both nodes:
ID | DATA1 | DATA2
1 | master | slave
Tagsconflict manager

Relationships

related to 0004302 closedelong Conflict resolution NEWER_TIME based on capture time of row 
related to 0004303 closedelong Change default of dataloader.apply.changes.only to true to support better conflict resolution 

Activities

kraynopp

2019-05-20 12:20

reporter  

attachment.tar.gz (4,212 bytes)

elong

2019-05-20 17:20

developer   ~0001496

Confirmed, it's not working properly. In 3.9.5, issue 0003462 changed the dataloader.apply.changes.only parameter default to false. I wouldn't expect that to affect conflict management, but maybe it was hiding the problem by allowing column-level updates to work?

I logged the data loader SQL, and I'm not sure I understand what I'm seeing. I expected the use_changed_data to only put columns in the where clause that changed, but it's putting all columns in the where clause for detection. Even worse, when it applies the winning row, it only updates the changed columns! The losing row appears to try some kind of second operation.

-- WINNER
[client-1-dataloader-14] Preparing dml: update "CONFTEST2"."PUBLIC"."DATA1" set "DATA1" = ?, "DATA2" = ? where "DATA1" = ? and "DATA2" = ? and "ID" = ?
[client-1-dataloader-14] Submitting data ['master1', 'slave', 'master', 'slave', 1] with types [VARCHAR, VARCHAR, VARCHAR, VARCHAR, INTEGER]
[client-1-dataloader-14] Preparing dml: update "CONFTEST2"."PUBLIC"."DATA1" set "DATA1" = ? where "ID" = ?
[client-1-dataloader-14] Submitting data ['master1', 1] with types [VARCHAR, INTEGER]

-- LOSER
[server-dataloader-7] Preparing dml: update "CONFTEST"."PUBLIC"."DATA1" set "DATA1" = ?, "DATA2" = ? where "DATA1" = ? and "DATA2" = ? and "ID" = ?
[server-dataloader-7] Submitting data ['master', 'slave1', 'master', 'slave', 1] with types [VARCHAR, VARCHAR, VARCHAR, VARCHAR, INTEGER]
[server-dataloader-7] Submitting data ['master1', 'slave1', 'master', 'slave1', 1] with types [VARCHAR, VARCHAR, VARCHAR, VARCHAR, INTEGER]

I switched to use_old_data, which does a little better, but still doesn't work. The losing row appears to get detected and ignored right away. The winning row still only applies changes, but it does capture a ping back.

-- WINNER
[client-1-dataloader-34] Preparing dml: update "CONFTEST2"."PUBLIC"."DATA1" set "DATA1" = ?, "DATA2" = ? where "ID" = ? and "DATA1" = ? and "DATA2" = ?
[client-1-dataloader-34] Submitting data ['master1', 'slave', 1, 'master', 'slave'] with types [VARCHAR, VARCHAR, INTEGER, VARCHAR, VARCHAR]
[client-1-dataloader-34] Preparing dml: update "CONFTEST2"."PUBLIC"."DATA1" set "DATA1" = ? where "ID" = ?
[client-1-dataloader-34] Submitting data ['master1', 1] with types [VARCHAR, INTEGER]

-- LOSER
[server-dataloader-32] Preparing dml: update "CONFTEST"."PUBLIC"."DATA1" set "DATA1" = ?, "DATA2" = ? where "ID" = ? and "DATA1" = ? and "DATA2" = ?
[server-dataloader-32] Submitting data ['master', 'slave1', 1, 'master', 'slave'] with types [VARCHAR, VARCHAR, INTEGER, VARCHAR, VARCHAR]

-- PING BACK OF WINNER
[server-dataloader-33] Preparing dml: update "CONFTEST"."PUBLIC"."DATA1" set "DATA1" = ?, "DATA2" = ? where "ID" = ? and "DATA1" = ? and "DATA2" = ?
[server-dataloader-33] Submitting data ['master1', 'slave1', 1, 'master', 'slave1'] with types [VARCHAR, VARCHAR, INTEGER, VARCHAR, VARCHAR]

kraynopp

2019-05-21 08:45

reporter   ~0001499

Yes, syncronization works without issues with dataloader.apply.changes.only=true.

Issue History

Date Modified Username Field Change
2019-05-20 12:20 kraynopp New Issue
2019-05-20 12:20 kraynopp File Added: attachment.tar.gz
2019-05-20 17:20 elong Assigned To => elong
2019-05-20 17:20 elong Status new => confirmed
2019-05-20 17:20 elong Note Added: 0001496
2019-05-20 23:50 elong Tag Attached: conflict manager
2019-05-21 08:45 kraynopp Note Added: 0001499
2020-03-12 12:57 elong Relationship added related to 0004302
2020-03-12 12:57 elong Relationship added related to 0004303
2020-03-12 12:59 elong Status confirmed => resolved
2020-03-12 12:59 elong Resolution open => fixed
2020-03-12 12:59 elong Fixed in Version => 3.12.0
2020-03-12 12:59 elong Target Version => 3.12.0
2020-03-12 12:59 elong Steps to Reproduce Updated View Revisions
2020-03-12 12:59 elong Additional Information Updated View Revisions
2020-05-26 18:48 elong Status resolved => closed