View Issue Details

IDProjectCategoryView StatusLast Update
0002296SymmetricDSBugpublic2015-10-22 08:27
ReporterxBlackCatAssigned Tochenson 
PrioritynormalSeverityblockReproducibilityalways
Status closedResolutionfixed 
Product Version3.7.16 
Target Version3.7.25Fixed in Version3.7.25 
Summary0002296: MySQL bulk loader produced invalid data file if the first column in table is binary.
DescriptionIf table contains binary columns at the begining, coma before non-binary column is missing

If table described like
create table (
 node_id binary(8),
 node_parent_id binary(8),
 status_id int
)

Data file for bulk loading generated like this:

"<binary>","<binary>""<int>",
"<binary>","<binary>""<int>",
TagsNo tags attached.

Activities

xBlackCat

2015-05-15 03:37

reporter  

symmetricds-Fix_#2296_and_#2297.patch (3,567 bytes)
Index: symmetric-client/src/main/java/org/jumpmind/symmetric/io/MySqlBulkDatabaseWriter.java
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
--- symmetric-client/src/main/java/org/jumpmind/symmetric/io/MySqlBulkDatabaseWriter.java	(revision 9349)
+++ symmetric-client/src/main/java/org/jumpmind/symmetric/io/MySqlBulkDatabaseWriter.java	(revision )
@@ -128,10 +128,13 @@
                         writer.setForceQualifier(true);
                         writer.setNullString("\\N");
                         Column[] columns = targetTable.getColumns();
+                        boolean binaryFirst = false;
                         for (int i = 0; i < columns.length; i++) {
                             if (columns[i].isOfBinaryType() && parsedData[i] != null) {
                                 if (i > 0) {
                                         out.write(',');
+                                } else {
+                                    binaryFirst = true;
                                 }
                                 out.write('"');
                                 if (batch.getBinaryEncoding().equals(BinaryEncoding.HEX)) {
@@ -141,6 +144,10 @@
                                 }
                                 out.write('"');
                             } else {
+                                if (binaryFirst) {
+                                    out.write(',');
+                                    binaryFirst = false;
+                                }
                                 writer.write(parsedData[i], true);
                                 writer.flush();
                             }
@@ -173,7 +180,7 @@
             flush();
         }
     }
-    
+
     protected void flush() {
         if (loadedRows > 0) {
                 this.stagedInputFile.close();
@@ -190,7 +197,7 @@
 	            		(isReplace ? "REPLACE " : "IGNORE ") + "INTO TABLE " +
 	            		this.getTargetTable().getQualifiedTableName(quote, catalogSeparator, schemaSeparator) +
                                 " FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '\"' ESCAPED BY '\\\\' LINES TERMINATED BY '\\n' STARTING BY ''" +
-                                " (" + Table.getCommaDeliminatedColumns(table.getColumns()) + ")";
+                                " (" + getCommaDeliminatedColumns(table.getColumns()) + ")";
 	            Statement stmt = c.createStatement();
 	
 	            //TODO:  clean this up, deal with errors, etc.?
@@ -228,6 +235,24 @@
     	//      but we don't want to depend on symmetric core.
         this.stagedInputFile = stagingManager.create(0, "bulkloaddir",
                 table.getName() + this.getBatch().getBatchId() + ".csv");
+    }
+
+    public String getCommaDeliminatedColumns(Column[] cols) {
+        DatabaseInfo dbInfo = platform.getDatabaseInfo();
+        String quote = dbInfo.getDelimiterToken();
+        StringBuilder columns = new StringBuilder();
+        if (cols != null && cols.length > 0) {
+            for (Column column : cols) {
+                columns.append(quote);
+                columns.append(column.getName());
+                columns.append(quote);
+                columns.append(",");
+            }
+            columns.replace(columns.length() - 1, columns.length(), "");
+            return columns.toString();
+        } else {
+            return " ";
+        }
     }
 
 }

xBlackCat

2015-05-15 03:38

reporter   ~0000689

Attached patch with fixes of 0002296 and 0002297 bugs

Related Changesets

SymmetricDS: 3.7 c58879b8

2015-10-21 21:06:39

chenson

Details Diff
0002296: MySQL bulk loader produced invalid data file if the first
column in table is binary.
0002297: MySQL bulk loader should deliminate column names in load
command

0002296
mod - symmetric-client/src/main/java/org/jumpmind/symmetric/io/MySqlBulkDatabaseWriter.java Diff File

Issue History

Date Modified Username Field Change
2015-05-14 09:56 xBlackCat New Issue
2015-05-15 03:37 xBlackCat File Added: symmetricds-Fix_#2296_and_#2297.patch
2015-05-15 03:38 xBlackCat Note Added: 0000689
2015-10-13 11:38 admin Assigned To => chenson
2015-10-13 11:38 admin Status new => assigned
2015-10-13 11:38 admin Fixed in Version => 3.7.25
2015-10-13 11:38 admin Target Version => 3.7.25
2015-10-21 22:00 chenson Changeset attached => SymmetricDS 3.7 c58879b8
2015-10-22 08:26 chenson Status assigned => resolved
2015-10-22 08:26 chenson Resolution open => fixed
2015-10-22 08:27 chenson Status resolved => closed