View Issue Details
ID | Project | Category | View Status | Date Submitted | Last Update |
---|---|---|---|---|---|
0002296 | SymmetricDS | Bug | public | 2015-05-14 13:56 | 2015-10-22 12:27 |
Reporter | xBlackCat | Assigned To | chenson | ||
Priority | normal | ||||
Status | closed | Resolution | fixed | ||
Product Version | 3.7.16 | ||||
Target Version | 3.7.25 | Fixed in Version | 3.7.25 | ||
Summary | 0002296: MySQL bulk loader produced invalid data file if the first column in table is binary. | ||||
Description | If table contains binary columns at the begining, coma before non-binary column is missing If table described like create table ( node_id binary(8), node_parent_id binary(8), status_id int ) Data file for bulk loading generated like this: "<binary>","<binary>""<int>", "<binary>","<binary>""<int>", | ||||
Tags | No tags attached. | ||||
|
symmetricds-Fix_#2296_and_#2297.patch (3,567 bytes)
Index: symmetric-client/src/main/java/org/jumpmind/symmetric/io/MySqlBulkDatabaseWriter.java IDEA additional info: Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP <+>UTF-8 =================================================================== --- symmetric-client/src/main/java/org/jumpmind/symmetric/io/MySqlBulkDatabaseWriter.java (revision 9349) +++ symmetric-client/src/main/java/org/jumpmind/symmetric/io/MySqlBulkDatabaseWriter.java (revision ) @@ -128,10 +128,13 @@ writer.setForceQualifier(true); writer.setNullString("\\N"); Column[] columns = targetTable.getColumns(); + boolean binaryFirst = false; for (int i = 0; i < columns.length; i++) { if (columns[i].isOfBinaryType() && parsedData[i] != null) { if (i > 0) { out.write(','); + } else { + binaryFirst = true; } out.write('"'); if (batch.getBinaryEncoding().equals(BinaryEncoding.HEX)) { @@ -141,6 +144,10 @@ } out.write('"'); } else { + if (binaryFirst) { + out.write(','); + binaryFirst = false; + } writer.write(parsedData[i], true); writer.flush(); } @@ -173,7 +180,7 @@ flush(); } } - + protected void flush() { if (loadedRows > 0) { this.stagedInputFile.close(); @@ -190,7 +197,7 @@ (isReplace ? "REPLACE " : "IGNORE ") + "INTO TABLE " + this.getTargetTable().getQualifiedTableName(quote, catalogSeparator, schemaSeparator) + " FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '\"' ESCAPED BY '\\\\' LINES TERMINATED BY '\\n' STARTING BY ''" + - " (" + Table.getCommaDeliminatedColumns(table.getColumns()) + ")"; + " (" + getCommaDeliminatedColumns(table.getColumns()) + ")"; Statement stmt = c.createStatement(); //TODO: clean this up, deal with errors, etc.? @@ -228,6 +235,24 @@ // but we don't want to depend on symmetric core. this.stagedInputFile = stagingManager.create(0, "bulkloaddir", table.getName() + this.getBatch().getBatchId() + ".csv"); + } + + public String getCommaDeliminatedColumns(Column[] cols) { + DatabaseInfo dbInfo = platform.getDatabaseInfo(); + String quote = dbInfo.getDelimiterToken(); + StringBuilder columns = new StringBuilder(); + if (cols != null && cols.length > 0) { + for (Column column : cols) { + columns.append(quote); + columns.append(column.getName()); + columns.append(quote); + columns.append(","); + } + columns.replace(columns.length() - 1, columns.length(), ""); + return columns.toString(); + } else { + return " "; + } } } |
|
Attached patch with fixes of 0002296 and 0002297 bugs |
SymmetricDS: 3.7 c58879b8 2015-10-21 21:06:39 Details Diff |
0002296: MySQL bulk loader produced invalid data file if the first column in table is binary. 0002297: MySQL bulk loader should deliminate column names in load command |
Affected Issues 0002296 |
|
mod - symmetric-client/src/main/java/org/jumpmind/symmetric/io/MySqlBulkDatabaseWriter.java | Diff File |
Date Modified | Username | Field | Change |
---|---|---|---|
2015-05-14 13:56 | xBlackCat | New Issue | |
2015-05-15 07:37 | xBlackCat | File Added: symmetricds-Fix_#2296_and_#2297.patch | |
2015-05-15 07:38 | xBlackCat | Note Added: 0000689 | |
2015-10-13 15:38 | admin | Assigned To | => chenson |
2015-10-13 15:38 | admin | Status | new => assigned |
2015-10-13 15:38 | admin | Fixed in Version | => 3.7.25 |
2015-10-13 15:38 | admin | Target Version | => 3.7.25 |
2015-10-22 02:00 | chenson | Changeset attached | => SymmetricDS 3.7 c58879b8 |
2015-10-22 12:26 | chenson | Status | assigned => resolved |
2015-10-22 12:26 | chenson | Resolution | open => fixed |
2015-10-22 12:27 | chenson | Status | resolved => closed |