View Issue Details

Jump to NotesJump to History
IDProjectCategoryView StatusDate SubmittedLast Update
0005357SymmetricDSBugpublic2022-07-11 16:072022-11-21 18:50
Reporterpmarzullo Assigned Topmarzullo  
Prioritynormal 
Status closedResolutionunable to reproduce 
Product Version3.12.5 
Summary0005357: Load filter failing with missing column variables
DescriptionHi team,

I have a batch, 1248701, which contains 3 rows, all update from PGMF mast_unit_fin to the VMA mp0010 table.

The first row is in the target table and updates w/out any errors.
The second one is not on the target. The update does not find any rows, SDS generates an insert and processes w/out any errors.
The third row is in the table but when it runs the load filter (VMA rfnd1_reference mp0010), it fails because the column variables do not exist!

This is a trace from the load filter. As you can see, the two variables are void! Because of this, the filter fails.

2021-03-25 17:30:22 [INFO ] [java ] UNI_INVNBR: N1QL7C, SRC_TBL_NM: mast_unit_fin, data.getDataEventType(): UPDATE
2021-03-25 17:30:22 [INFO ] [java ] UNI_INVNBR: N1QRL7, SRC_TBL_NM: mast_unit_fin, data.getDataEventType(): UPDATE
2021-03-25 17:30:22 [INFO ] [java ] UNI_INVNBR: void, SRC_TBL_NM: void, data.getDataEventType(): UPDATE

I have a snapshot from the source server but it is 11 Mb so it you want it, please send me ftp info to upload it.

Here is a quick copy/paste for the error itself:

2021-03-25 16:21:29,477 ERROR [VMA_SYBASE-VMA_SYB] [BshDatabaseWriterFilter] [vma_sybase-vma_syb-dataloader-1842] Error executing beanshell script for load filter VMA rfnd1_reference mp0010 on table mp0010. The error was: Sourced file: inline evaluation of: ``// VMA rfnd1_reference mp0010 import org.jumpmind.symmetric.io.data.DataEventTyp . . . '' : Void assignment.
2021-03-25 16:21:29,477 INFO [VMA_SYBASE-VMA_SYB] [DefaultDatabaseWriter] [vma_sybase-vma_syb-dataloader-1842] Failed to process update event in batch MF_PG-1248701 on channel 'MF_PG'.
Failed pk data was: "FA","7850","93130"
Failed row data was: "FA","7850","93130",,"N1JJ29","0.00","0.00","2020-11-20",,"0","","000","2021-03-25 15:52:18.864","FA","7850","93130","mast_unit_fin","UPDATE"
Failed old data was: "FA","7850","93130",,"N1JJ29","0.00","0.00",,,,"","000",,,,,,
StackTraceKey.init [SymmetricException:191915980] org.jumpmind.symmetric.SymmetricException: Error executing beanshell script for load filter VMA rfnd1_reference mp0010 on table mp0010. The error was: Sourced file: inline evaluation of: ``// VMA rfnd1_reference mp0010 import org.jumpmind.symmetric.io.data.DataEventTyp . . . '' : Void assignment.
    at org.jumpmind.symmetric.load.BshDatabaseWriterFilter.processError(BshDatabaseWriterFilter.java:210)
    at org.jumpmind.symmetric.load.BshDatabaseWriterFilter.processLoadFilters(BshDatabaseWriterFilter.java:98)
    at org.jumpmind.symmetric.load.DynamicDatabaseWriterFilter.processLoadFilters(DynamicDatabaseWriterFilter.java:151)
    at org.jumpmind.symmetric.load.DynamicDatabaseWriterFilter.beforeWrite(DynamicDatabaseWriterFilter.java:89)
    at org.jumpmind.symmetric.io.data.writer.AbstractDatabaseWriter.filterBefore(AbstractDatabaseWriter.java:332)
    at org.jumpmind.symmetric.io.data.writer.AbstractDatabaseWriter.write(AbstractDatabaseWriter.java:172)
    at org.jumpmind.symmetric.io.data.writer.NestedDataWriter.write(NestedDataWriter.java:64)
    at org.jumpmind.symmetric.model.ProcessInfoDataWriter.write(ProcessInfoDataWriter.java:84)
    at org.jumpmind.symmetric.io.data.writer.TransformWriter.write(TransformWriter.java:194)
    at org.jumpmind.symmetric.io.data.DataProcessor.forEachDataInTable(DataProcessor.java:212)
    at org.jumpmind.symmetric.io.data.DataProcessor.forEachTableInBatch(DataProcessor.java:178)
    at org.jumpmind.symmetric.io.data.DataProcessor.process(DataProcessor.java:124)
    at org.jumpmind.symmetric.service.impl.DataLoaderService$LoadIntoDatabaseOnArrivalListener$2.call(DataLoaderService.java:1091)
    at org.jumpmind.symmetric.service.impl.DataLoaderService$LoadIntoDatabaseOnArrivalListener$2.call(DataLoaderService.java:1067)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
Caused by: Sourced file: inline evaluation of: ``// VMA rfnd1_reference mp0010 import org.jumpmind.symmetric.io.data.DataEventTyp . . . '' : Void assignment. : at Line: 11 : in file: inline evaluation of: ``// VMA rfnd1_reference mp0010 import org.jumpmind.symmetric.io.data.DataEventTyp . . . '' : srcTblNm = SRC_TBL_NM
    at bsh.BSHAssignment.eval(BSHAssignment.java:74)
    at bsh.BSHBlock.evalBlock(BSHBlock.java:125)
    at bsh.BSHBlock.eval(BSHBlock.java:75)
    at bsh.BSHBlock.eval(BSHBlock.java:41)
    at bsh.BSHIfStatement.eval(BSHIfStatement.java:45)
    at bsh.Interpreter.eval(Interpreter.java:659)
    at bsh.Interpreter.eval(Interpreter.java:750)
    at bsh.Interpreter.eval(Interpreter.java:739)
    at org.jumpmind.symmetric.load.BshDatabaseWriterFilter.processLoadFilters(BshDatabaseWriterFilter.java:83)
    ... 16 more
Additional InformationCan you export the CSV from the incoming batches screen for batch 1248701 on the target side and attach?

Here it is. I had to recreate the error so the batch # is different. I also ran a test. Currently, the second row in the batch does not exists on the target server and the batch fails. But if I add the row on the target server and reprocess the same batch (3 updates), the batch processes successfully. Hopefully, this is helpful.

BTW, when I ignored the batch in error, the batch file was deleted on the target server but the file on the source server was recreated with this name MF_POSTGRES-MF_PG/outgoing/VMA_SYB/0001253044.create and this content:
[svc_symmetricds@vsplsymmds01d /opt/symmetricds/tmp]$ cat MF_POSTGRES-MF_PG/outgoing/VMA_SYB/0001253044.create
nodeid,MF_PG
binary,BASE64
channel,MF_PG
batch,1253044
ignore
commit,1253044

It was never changed to "done". I do not think it is correct, is it?

Also, the "outgoing batches" menu is hanging all the time when you try to make a change to the "Status" box. I have to kill the browser and re-login.
Tagsdialect: postgresql, dialect: sybase

Activities

pmarzullo

2022-07-11 16:07

developer  

0001253044.done (1,041 bytes)    
nodeid,MF_PG
binary,BASE64
channel,MF_PG
batch,1253044
catalog,
schema,
table,mast_unit_fin_mp0010
keys,corp_cd,cli_nbr,uni_nbr
columns,ignore_row_test,uni_invnbr,uni_inicst,uni_bokval,uni_deldat,uni_soldat,uni_odorea,uni_leactr,uni_leatrn,chg_dtm,corp_cd_old,cli_nbr_old,uni_nbr_old,src_tbl_nm,ops,corp_cd,cli_nbr,uni_nbr
old,,"N1QL7C","0.00","0.00",,,,"","000",,,,,,,"FA","2424","60937"
update,,"N1QL7C","0.00","0.00","2021-3-30",,"0","","000","2021-03-30 10:07:31.247","FA","2424","60937","mast_unit_fin","UPDATE","FA","2424","60937","FA","2424","60937"
old,,"N1QRL7","0.00","0.00",,,,"","000",,,,,,,"FA","9829","00008"
update,,"N1QRL7","0.00","0.00","2021-3-20",,"0","","000","2021-03-30 10:07:31.249","FA","9829","00008","mast_unit_fin","UPDATE","FA","9829","00008","FA","9829","00008"
old,,"N1JJ29","0.00","0.00",,,,"","000",,,,,,,"FA","7850","93130"
update,,"N1JJ29","0.00","0.00","2020-11-20",,"0","","000","2021-03-30 10:07:31.250","FA","7850","93130","mast_unit_fin","UPDATE","FA","7850","93130","FA","7850","93130"
commit,1253044
0001253044.done (1,041 bytes)   

pmarzullo

2022-11-21 18:50

developer   ~0002220

I have been unable to reproduce this issue.

I attempted:
1. Update 1 row.
2. Update a second row that does not exist at the target.
3. Update a third row.

The transform added a src_tbl_nm from variable source_table_name.
The load filter on the third row that was updated had the src_tbl_nm value filled out.

Issue History

Date Modified Username Field Change
2022-07-11 16:07 pmarzullo New Issue
2022-07-11 16:07 pmarzullo Status new => assigned
2022-07-11 16:07 pmarzullo Assigned To => pmarzullo
2022-07-11 16:07 pmarzullo Tag Attached: dialect: postgresql
2022-07-11 16:07 pmarzullo Tag Attached: dialect: sybase
2022-07-11 16:07 pmarzullo File Added: 0001253044.done
2022-07-25 16:52 elong Target Version 3.14.0 => 3.14.1
2022-08-24 18:56 elong Target Version 3.14.1 => 3.14.2
2022-10-03 14:38 elong Target Version 3.14.2 =>
2022-11-21 18:50 pmarzullo Status assigned => resolved
2022-11-21 18:50 pmarzullo Resolution open => unable to reproduce
2022-11-21 18:50 pmarzullo Note Added: 0002220
2022-11-21 18:50 pmarzullo Status resolved => closed