View Issue Details

Related ChangesetsJump to NotesJump to History
IDProjectCategoryView StatusDate SubmittedLast Update
0002363SymmetricDSBugpublic2015-08-05 12:312015-10-02 14:23
Reportermsc Assigned Tochenson  
Prioritynormal 
Status closedResolutionfixed 
Product Version3.7.16 
Target Version3.7.22Fixed in Version3.7.22 
Summary0002363: CHAR(x) Column Padding with whitespaces won't work if character conversion is needed to
Descriptioni have a postgresql (UTF-8) <-> Interbase_XE7 (8859-1) Setup.

my Interbase has a table like:

CREATE TABLE KUNDEN (
        KDN_NR INTEGER NOT NULL,
        ANREDE CHAR(5) CHARACTER SET ISO8859_1,
        KNAME CHAR(25) CHARACTER SET ISO8859_1,
        FAM_STAND CHAR(20) CHARACTER SET ISO8859_1,
PRIMARY KEY (KDN_NR));

if the table receives data, the org.jumpmind.db.platform.interbase.InterbaseDdlBuilder has enabled the padding of non-blank-Char-Columns

this.databaseInfo.setNonBlankCharColumnSpacePadded(true);

therefore the received String "Bü" for column "KNAME" was padded to "Bü ...".
But afterwards it will put the (padded) value into the update query and now the 16-Bit-UTF-Java -padded-String Bytes-Length will be compared to the target-DB-Column-Size:

But this will throw a DataTruncation Exception because:

Target: Interbase-Column with ISO8859_1-Encoding -> 25 Chars => 25 Bytes
Source: Padded-String with "Umlauts" 25 Chars -> 26 Bytes !!! -> DataTruncation.

Please allow to disable the NonBlankCharColumnSpacePadded setting for Interbase (System-Property would be fine)!
Or you need to pad only up-to allowed byte-size?!?
Perhaps the interclient.jar (JDBC-Driver for interbase) has a bug here?
the interbase.interclient.PreparedStatement#setString function looks like (only relevant part)

  this.inputs_[var1 - 1] = var2;
  if(var2.getBytes().length > this.inputCharLengths_[var1 - 1]) {
    throw new DataTruncation(var1, true, false, var2.length(), this.inputCharLengths_[var1 - 1]);
  }

var1 => prepared-statement-index
var2 => padded String.
inputCharLengths_[var1 - 1] => 25 !!!
var2.getBytes().length => 26 !!!


SymmetricDS Log:
[apo2_interbase] - DefaultDatabaseWriter - Failed to process a update event in batch 1551.
Failed sql was: update "KUNDEN" set "KNAME" = ?, "FAM_STAND" = ?, "ANREDE" = ? where "KDN_NR" = ?
Failed sql parameters: [Bü , ledig , Herr , 900817783]
Failed sql parameters types: [1, 1, 1, 4]
Failed pk data was: "900817783"
Failed row data was: "900817783","Bu","Bü","1998-12-07 00:00:00","ledig","Herr"
Failed old data was: "900817783","Bu","Bo","1998-12-07 00:00:00",,

[apo2_interbase] - DataLoaderService - Failed to load batch apo2db-1551 because: Data truncation
org.jumpmind.db.sql.SqlException: Data truncation
        at org.jumpmind.db.sql.AbstractSqlTemplate.translate(AbstractSqlTemplate.java:288)
        at org.jumpmind.db.sql.AbstractSqlTemplate.translate(AbstractSqlTemplate.java:279)
        at org.jumpmind.db.sql.JdbcSqlTransaction.addRow(JdbcSqlTransaction.java:410)
        at org.jumpmind.symmetric.io.data.writer.DefaultDatabaseWriter.execute(DefaultDatabaseWriter.java:786)
        at org.jumpmind.symmetric.io.data.writer.DefaultDatabaseWriter.update(DefaultDatabaseWriter.java:456)
        at org.jumpmind.symmetric.io.data.writer.AbstractDatabaseWriter.write(AbstractDatabaseWriter.java:152)
        at org.jumpmind.symmetric.io.data.writer.NestedDataWriter.write(NestedDataWriter.java:64)
        at org.jumpmind.symmetric.model.ProcessInfoDataWriter.write(ProcessInfoDataWriter.java:65)
        at org.jumpmind.symmetric.io.data.writer.NestedDataWriter.write(NestedDataWriter.java:64)
        at org.jumpmind.symmetric.io.data.writer.TransformWriter.write(TransformWriter.java:204)
        at org.jumpmind.symmetric.io.data.DataProcessor.forEachDataInTable(DataProcessor.java:200)
        at org.jumpmind.symmetric.io.data.DataProcessor.forEachTableInBatch(DataProcessor.java:170)
        at org.jumpmind.symmetric.io.data.DataProcessor.process(DataProcessor.java:116)
        at org.jumpmind.symmetric.service.impl.DataLoaderService$LoadIntoDatabaseOnArrivalListener.end(DataLoaderService.java:820)
        at org.jumpmind.symmetric.io.data.writer.StagingDataWriter.notifyEndBatch(StagingDataWriter.java:75)
        at org.jumpmind.symmetric.io.data.writer.AbstractProtocolDataWriter.end(AbstractProtocolDataWriter.java:220)
        at org.jumpmind.symmetric.io.data.DataProcessor.process(DataProcessor.java:130)
        at org.jumpmind.symmetric.service.impl.DataLoaderService.loadDataFromTransport(DataLoaderService.java:430)
        at org.jumpmind.symmetric.service.impl.DataLoaderService.loadDataFromPush(DataLoaderService.java:344)
        at org.jumpmind.symmetric.web.PushUriHandler.push(PushUriHandler.java:78)
        at org.jumpmind.symmetric.web.PushUriHandler.handle(PushUriHandler.java:67)
        at org.jumpmind.symmetric.web.SymmetricServlet.service(SymmetricServlet.java:103)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
        at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:565)
        at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:479)
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119)
        at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
        at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:227)
        at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1031)
        at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:406)
        at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:186)
        at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:965)
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:111)
        at org.eclipse.jetty.server.Server.handle(Server.java:348)
        at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:452)
        at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:894)
        at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:948)
        at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:851)
        at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
        at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:77)
        at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:606)
        at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:46)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:603)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:538)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.sql.DataTruncation: Data truncation
        at interbase.interclient.PreparedStatement.setString(Unknown Source)
        at interbase.interclient.PreparedStatement.setObject(Unknown Source)
        at interbase.interclient.PreparedStatement.setObject(Unknown Source)
        at org.apache.commons.dbcp.DelegatingPreparedStatement.setObject(DelegatingPreparedStatement.java:166)
        at org.apache.commons.dbcp.DelegatingPreparedStatement.setObject(DelegatingPreparedStatement.java:166)
        at org.springframework.jdbc.core.StatementCreatorUtils.setValue(StatementCreatorUtils.java:407)
        at org.springframework.jdbc.core.StatementCreatorUtils.setParameterValueInternal(StatementCreatorUtils.java:235)
        at org.springframework.jdbc.core.StatementCreatorUtils.setParameterValue(StatementCreatorUtils.java:166)
        at org.jumpmind.db.sql.JdbcSqlTemplate.setValues(JdbcSqlTemplate.java:942)
        at org.jumpmind.db.sql.JdbcSqlTransaction.addRow(JdbcSqlTransaction.java:394)
        ... 43 more
TagsNo tags attached.

Activities

msc

2015-08-10 12:23

reporter   ~0000709

Last edited: 2015-08-25 09:20

View 2 revisions

 this patch is a workaround
https://github.com/NiasSt90/symmetric-ds/commit/1282982c6e92ed7dc934e4e7fa8be59c6826fdfc

in this case the Interbase server would do the padding for you (i don't understand why a client should do the padding by self).

Related Changesets

SymmetricDS: 3.7 1282982c

2015-08-10 03:16:15

msc

Details Diff 		0002363: CHAR(x) Column Padding with whitespaces won't work if character conversion is needed to Affected Issues
0002363
mod - symmetric-db/src/main/java/org/jumpmind/db/platform/interbase/InterbaseDdlBuilder.java Diff File

Issue History

Date Modified Username Field Change
2015-08-05 12:31 msc New Issue
2015-08-10 12:23 msc Note Added: 0000709
2015-08-25 09:20 msc Note Edited: 0000709 View Revisions
2015-09-11 20:13 chenson Target Version => 3.7.22
2015-09-24 15:34 chenson Fixed in Version => 3.7.22
2015-09-24 18:46 chenson Status new => resolved
2015-09-24 18:46 chenson Resolution open => fixed
2015-09-24 18:46 chenson Assigned To => chenson
2015-09-24 19:00 msc Changeset attached => SymmetricDS 3.7 1282982c
2015-10-02 14:23 chenson Status resolved => closed