View Issue Details

IDProjectCategoryView StatusLast Update
0004787SymmetricDSBugpublic2021-01-26 20:09
Reporterelong Assigned To 
Prioritynormal 
Status newResolutionopen 
Product Version3.9.0 
Target Version3.13.0 
Summary0004787: Mixed up columns and error if load transform column order doesn't match table definition
DescriptionFor a load transform, the order of columns specified must match the table definition, otherwise the data is put in the wrong column and can produce an error. This does not affect extract transforms.

A user claimed to experience this after using 3.8.32 successfully and upgrading to 3.12.4. Looking at commit history, not much has changed since 3.8.32 for core transform classes. It looks like load transforms have always used the transform column order to generate the transformed data.

In TransformWriter, the start() is called with a Table object, which has the correct ordering of columns that will match the SQL. However, in write(), the following lines get the order wrong:

// This table will have columns ordered incorrectly, because it is based on the transform column order
Table transformedTable = transformedData.buildTargetTable();
// This CSV data is also based on transform column order
CsvData csvData = transformedData.buildTargetCsvData();
Steps To Reproduce

CREATE TABLE my_summary (
    id BIGINT NOT NULL,
    mintimestamp TIMESTAMP,
    maxtimestamp TIMESTAMP,
    numevents INTEGER,
    flag BOOLEAN,
    duration REAL,
    interface INTEGER,
    PRIMARY KEY (id)
);

Create LOAD transform that is IMPLIED. Create column transforms for update on mintimestamp, maxtimestamp, duration, and numevents in that order. Insert a row into my_summary. Update the row and change the numevents and duration fields. The values for those fields will bind to wrong position in the SQL.
Tagstransformation

Activities

There are no notes attached to this issue.

Issue History

Date Modified Username Field Change
2021-01-26 20:09 elong New Issue
2021-01-26 20:09 elong Status new => assigned
2021-01-26 20:09 elong Assigned To => elong
2021-01-26 20:09 elong Tag Attached: transformation
2021-01-26 20:09 elong Assigned To elong =>
2021-01-26 20:09 elong Status assigned => new
2021-01-26 20:09 elong Target Version => 3.13.0