View Issue Details

IDProjectCategoryView StatusLast Update
0003806SymmetricDSImprovementpublic2018-11-28 11:00
ReporterelongAssigned Toelong 
Prioritynormal 
Status resolvedResolutionfixed 
Product Version3.10.0 
Target Version3.10.0Fixed in Version3.10.0 
Summary0003806: Initial load large character (varchar 4000) columns and stay under SQL limit
DescriptionThe initial load uses the same column templates used by the triggers that wrap SQL functions around columns to format and convert them into a string that is in CSV format. The SQL result set has a limit for how much character data can be returned -- for example, 4000 characters on Oracle. If a column is defined up to the SQL limit, such as varchar(4000) on Oracle, and the field is filled but needs characters escaped for CSV, like a quote or backslash, then the escaping pushes the string past the limit, making the initial load fail. Instead of formatting the columns in the database, select the columns directly and format the data in the program. An added side effect should be the SQL running faster. Use initial.load.use.column.templates.enabled=false parameter in case we need to go back to old behavior. Limit the column types to char, integer, and LOB for now, and other column types can be added in future.

Activities

There are no notes attached to this issue.

Related Changesets

SymmetricDS: 3.10 7cb365cd

2018-11-27 10:50:07

admin

Details Diff
0003806: Initial load large character (varchar 4000) columns and stay
under SQL limit

0003806
mod - symmetric-client/src/main/java/org/jumpmind/symmetric/db/db2/Db2As400TriggerTemplate.java Diff File
mod - symmetric-client/src/main/java/org/jumpmind/symmetric/db/firebird/FirebirdTriggerTemplate.java Diff File
mod - symmetric-client/src/main/java/org/jumpmind/symmetric/db/interbase/InterbaseTriggerTemplate.java Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/common/ParameterConstants.java Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/db/AbstractSymmetricDialect.java Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/db/AbstractTriggerTemplate.java Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/db/ISymmetricDialect.java Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/service/impl/DataExtractorService.java Diff File
mod - symmetric-core/src/main/resources/symmetric-default.properties Diff File
mod - symmetric-db/src/main/java/org/jumpmind/db/platform/AbstractDatabasePlatform.java Diff File
mod - symmetric-db/src/main/java/org/jumpmind/db/platform/IDatabasePlatform.java Diff File

SymmetricDS: 3.10 758a001d

2018-11-27 13:21:35

admin

Details Diff
0003806: Initial load large character (varchar 4000) columns and stay
under SQL limit

0003806
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/service/impl/DataExtractorService.java Diff File

SymmetricDS: 3.10 46a01ddb

2018-11-28 10:45:39

admin

Details Diff
0003806: Initial load large character (varchar 4000) columns and stay
under SQL limit

0003806
mod - symmetric-db/src/main/java/org/jumpmind/db/model/ColumnTypes.java Diff File

Issue History

Date Modified Username Field Change
2018-11-27 10:37 elong New Issue
2018-11-27 10:37 elong Status new => assigned
2018-11-27 10:37 elong Assigned To => elong
2018-11-27 10:52 elong Status assigned => resolved
2018-11-27 10:52 elong Resolution open => fixed
2018-11-27 10:52 elong Fixed in Version => 3.10.0
2018-11-28 11:00 admin Changeset attached => SymmetricDS 3.10 46a01ddb
2018-11-28 11:00 admin Changeset attached => SymmetricDS 3.10 758a001d
2018-11-28 11:00 admin Changeset attached => SymmetricDS 3.10 7cb365cd