View Issue Details

IDProjectCategoryView StatusLast Update
0002964SymmetricDSBugpublic2017-05-01 14:03
Reporterivan.konevAssigned Tomaxwellpettit 
PrioritynormalSeveritymajorReproducibilityalways
Status closedResolutionfixed 
Product Version3.8.13 
Target Version3.8.22Fixed in Version3.8.22 
Summary0002964: MEDIUMTEXT/LONGTEXT columns being converted to hex representation on mysql data on extract at source node
Descriptionit does what it says. Probably(by the logs) type recognized as binary and extracted as binary. But on destination node, it's still TEXT and inserting hex representation of source text.
Steps To Reproduce1. Get two synchronized nodes, like from example.
2. Create 2 tables with MEDUIMTEXT/LONGTEXT fields.
3. Link it with trigger+trigger_router.
4. Insert something on source node.

It's all ok when changing only SOURCE NODE data type to text/varchar, extracted correctly and inserted to destination MEDIUMTEXT/LONGTEXT field correctly.
TagsNo tags attached.

Activities

ivan.konev

2017-01-18 04:46

reporter   ~0000953

Update - TEXT affected also. But with corrupted data.

ivan.konev

2017-01-18 04:53

reporter   ~0000954

Update - corrupted data appearing when using reload-node/reload-table only.
It looks like
[B@312cff16

ivan.konev

2017-01-18 04:54

reporter   ~0000955

Reproduce - do reload, every row will be corrupted and truncated. Do update of any record(with correct type at source node) - will be OK. Do with mentioned data types on source node - correct hex representation will be "replicated"

maxwellpettit

2017-03-24 13:31

developer   ~0000983

ivan.konev,

I have tried to reproduce this error following your steps and your notes, but I was not able to recreate this error. If this is still an issue you are noticing, could you post your versions of mysql and symmetricds as well as an example of your tables on each node?

Thanks,
Max

ivan.konev

2017-03-24 13:35

reporter   ~0000984

Sure, mysql 5.7 with additional show_compatibility_56=1 , encodings
character-set-server=utf8mb4
collation-server=utf8mb4_unicode_ci

SymmetricDS 3.8.13

Table template

CREATE TABLE `xxx_log` (
    `dt` DATETIME(3) NOT NULL DEFAULT '',
    `xxx` CHAR(128) NOT NULL COLLATE 'utf8_bin',
    `cam` CHAR(128) NOT NULL COLLATE 'utf8_bin',
    `text` VARCHAR(21400) NULL DEFAULT NULL COLLATE 'utf8_bin',
    PRIMARY KEY (`dt`, `xxx`, `cam`),
    INDEX `idx_dt` (`dt`)
)
COLLATE='utf8_bin'
ENGINE=InnoDB
ROW_FORMAT=COMPRESSED
;

Text was originally TEXT/MEDIUMTEXT type.

ivan.konev

2017-03-24 13:35

reporter   ~0000985

Tables on source and destination are equal. No transforms also.

ivan.konev

2017-03-24 13:37

reporter   ~0000986

Engine JDBC connection string
db.url=jdbc:mysql://127.0.0.1/db?useUnicode=true&characterEncoding=utf8

But changing/removing all paramaters does not gives any change. java/jdbc utf8 equal to full utf8mb4 mysql mentioned above.

maxwellpettit

2017-04-07 14:51

developer   ~0000996

I have tried copying all of your settings as you described and I was not able to reproduce any similar error. Are there any other setting that might be influencing this error?

ivan.konev

2017-04-07 15:05

reporter   ~0000997

Well, i will try to post you logs after reproducing this. Sure there MUST be UTF data in TEXT fields, not only 1 byte latin characters.

ivan.konev

2017-04-13 19:50

reporter   ~0001005

100% reproduced.

Definitions as above.

I've created table for testing - test.
CREATE TABLE `test` (
    `id` INT(10) UNSIGNED NOT NULL AUTO_INCREMENT,
    `dt` TIMESTAMP NULL DEFAULT NULL,
    `text` TEXT NULL COLLATE 'utf8mb4_bin',
    `char` VARCHAR(200) NULL DEFAULT NULL COLLATE 'utf8mb4_bin',
    PRIMARY KEY (`id`)
)
COLLATE='utf8mb4_bin'
ENGINE=InnoDB
;


linked it with trigger and trigger_router.


1. Inserting some UTF data - OK(pic), replicated correctly(pic)
2. Making reload node or table
./symadmin reload-table test -p ../engines/home-000.properties -n 001
FAIL, TEXT type column - garbled, varchar - OK (pic)

ivan.konev

2017-04-13 19:51

reporter  

ivan.konev

2017-04-13 19:51

reporter  

ivan.konev

2017-04-13 19:51

reporter  

ivan.konev

2017-04-13 19:51

reporter  

ivan.konev

2017-04-13 19:57

reporter   ~0001006

Retrying with debug logs.

Logs attached. Probably - TEXT recognized as binary.

ivan.konev

2017-04-13 19:58

reporter  

symmetric.log (100,254 bytes)

Related Changesets

SymmetricDS: 3.8 0189008e

2017-04-26 13:20:08

maxwellpettit

Details Diff
0002964: MEDIUMTEXT/LONGTEXT columns being converted to hex
representation on mysql data on extract at source node

0002964
mod - symmetric-jdbc/src/main/java/org/jumpmind/db/platform/mysql/MySqlDdlReader.java Diff File

Issue History

Date Modified Username Field Change
2017-01-18 04:36 ivan.konev New Issue
2017-01-18 04:46 ivan.konev Note Added: 0000953
2017-01-18 04:53 ivan.konev Note Added: 0000954
2017-01-18 04:54 ivan.konev Note Added: 0000955
2017-03-22 14:44 chenson Assigned To => maxwellpettit
2017-03-22 14:44 chenson Status new => assigned
2017-03-24 13:31 maxwellpettit Note Added: 0000983
2017-03-24 13:31 maxwellpettit Status assigned => feedback
2017-03-24 13:35 ivan.konev Note Added: 0000984
2017-03-24 13:35 ivan.konev Status feedback => assigned
2017-03-24 13:35 ivan.konev Note Added: 0000985
2017-03-24 13:37 ivan.konev Note Added: 0000986
2017-04-07 14:51 maxwellpettit Note Added: 0000996
2017-04-07 14:51 maxwellpettit Status assigned => feedback
2017-04-07 15:05 ivan.konev Note Added: 0000997
2017-04-07 15:05 ivan.konev Status feedback => assigned
2017-04-13 19:50 ivan.konev Note Added: 0001005
2017-04-13 19:51 ivan.konev File Added: Screenshot 2017-04-14 02.44.55.png
2017-04-13 19:51 ivan.konev File Added: Screenshot 2017-04-14 02.45.40.png
2017-04-13 19:51 ivan.konev File Added: Screenshot 2017-04-14 02.46.40.png
2017-04-13 19:51 ivan.konev File Added: Screenshot 2017-04-14 02.49.37.png
2017-04-13 19:57 ivan.konev Note Added: 0001006
2017-04-13 19:58 ivan.konev File Added: symmetric.log
2017-04-26 14:00 maxwellpettit Changeset attached => SymmetricDS 3.8 0189008e
2017-04-27 13:08 maxwellpettit Fixed in Version => 3.8.22
2017-04-27 13:08 maxwellpettit Target Version => 3.8.22
2017-04-27 13:09 maxwellpettit Status assigned => resolved
2017-04-27 13:09 maxwellpettit Resolution open => fixed
2017-05-01 14:03 chenson Status resolved => closed