View Issue Details

IDProjectCategoryView StatusLast Update
0003339SymmetricDSBugpublic2018-12-05 15:30
Reporterwoehrl01Assigned To 
Prioritynormal 
Status feedbackResolutionreopened 
Product Version3.8.33 
Target Version3.10.0Fixed in Version 
Summary0003339: Retrying batch gets ignored/OKed if stream.to.file.enabled = false
DescriptionIf a batch fails at it gets retried, but on the client stream.to.file.enabled = false is set, the batch gets ignored as the protocoldatawriter doesn't have the original content. So the batch gets silently OKed even so an error occurred the first time.
Steps To ReproduceHave client with stream.to.file.enabled = false

send an SQL which fails.

The first time it fails, the second time the batch gets OKed.

Activities

chenson

2017-12-14 10:35

administrator   ~0001085

Can you test this in the 3.9 branch? I think it might be fixed there. We are hoping to release 3.9 tomorrow ...

woehrl01

2017-12-14 11:33

reporter   ~0001086

I haven't tried on the 3.9 but looked at the code there to possible port something, and haven't found any noticeable difference.

I can have a look for this in 3.9 tomorrow.

The following (pretty hackish) fixes the problem for me:

--- a/symmetric-core/src/main/java/org/jumpmind/symmetric/service/impl/DataLoaderService.java Fri Nov 24 11:37:19 2017 +0100
+++ b/symmetric-core/src/main/java/org/jumpmind/symmetric/service/impl/DataLoaderService.java Thu Dec 14 16:32:32 2017 +0100
@@ -1070,8 +1070,16 @@
                     dataService.deleteCapturedConfigChannelData();
                     outgoingBatchService.markAllConfigAsSentForNode(batch.getSourceNodeId());
                 }
+ boolean isRetry = (boolean)context.get("batchIsRetryFromPureTransport");
+ context.remove("batchIsRetryFromPureTransport");
                 IncomingBatch incomingBatch = new IncomingBatch(batch);
                 this.batchesProcessed.add(incomingBatch);
+ if(isRetry) {
+ incomingBatch.setStatus(Status.RS);
+ incomingBatchService.updateIncomingBatch(incomingBatch);
+ return false;
+ }
+
                 if (incomingBatchService.acquireIncomingBatch(incomingBatch)) {
                     this.currentBatch = incomingBatch;
                     context.put("currentBatch", this.currentBatch);
--- a/symmetric-io/src/main/java/org/jumpmind/symmetric/io/data/reader/ProtocolDataReader.java Fri Nov 24 11:37:19 2017 +0100
+++ b/symmetric-io/src/main/java/org/jumpmind/symmetric/io/data/reader/ProtocolDataReader.java Thu Dec 14 16:32:32 2017 +0100
@@ -208,6 +208,7 @@
                     Batch batch = new Batch(batchType, Long.parseLong(tokens[1]), channelId,
                             binaryEncoding, sourceNodeId, targetNodeId, false, dataCreateTime);
                     statistics.put(batch, new DataReaderStatistics());
+ context.put("batchIsRetryFromPureTransport", tokens[0].equals(CsvConstants.RETRY));
                     tokens = null;
                     return batch;

woehrl01

2017-12-14 11:37

reporter   ~0001087

this bug sounds similar to 3333 but that is only on the extract side.

woehrl01

2018-01-05 05:52

reporter   ~0001096

I just verified that this problem still exists in 3.9.2.

It still can be fixed via this workaround, the code just moved from DataLoaderService.java to ManageIncomingBatchListener.java

hanes

2018-12-05 15:19

developer   ~0001364

This seems to be working correctly in 3.9.14. Note that you can't choose the combination stream.to.file.enabled = false and initial.load.use.extract.job.enabled=true. SymmetricDS won't start up in that case. I've tried setting both values to false and batches that go into error are not ignored based on my testing.

woehrl01

2018-12-05 15:30

reporter   ~0001365

Thanks Hanes. Have you also tested having different settings for Corp and store? So that you have stream to file and extract job disabled for stores, while having it enabled for Corp?

Issue History

Date Modified Username Field Change
2017-12-14 09:48 woehrl01 New Issue
2017-12-14 10:35 chenson Note Added: 0001085
2017-12-14 11:33 woehrl01 Note Added: 0001086
2017-12-14 11:37 woehrl01 Note Added: 0001087
2018-01-05 05:52 woehrl01 Note Added: 0001096
2018-01-05 07:52 chenson Target Version => 3.9.3
2018-01-05 07:53 chenson Assigned To => maxwellpettit
2018-01-05 07:53 chenson Status new => assigned
2018-01-22 08:38 chenson Target Version 3.9.3 => 3.9.4
2018-02-16 16:27 mmichalek Target Version 3.9.4 => 3.9.5
2018-03-12 08:25 chenson Target Version 3.9.5 => 3.9.6
2018-03-22 14:54 admin Target Version 3.9.6 => 3.9.7
2018-05-03 10:52 admin Assigned To maxwellpettit => josh-a-hicks
2018-05-03 10:52 admin Target Version 3.9.7 => 3.9.8
2018-06-05 14:26 josh-a-hicks Target Version 3.9.8 => 3.10.0
2018-06-05 14:29 josh-a-hicks Target Version 3.10.0 => 3.9.9
2018-06-21 14:53 josh-a-hicks Target Version 3.9.9 => 3.10.0
2018-12-05 15:01 hanes Assigned To josh-a-hicks => hanes
2018-12-05 15:19 hanes Note Added: 0001364
2018-12-05 15:19 hanes Assigned To hanes =>
2018-12-05 15:19 hanes Status assigned => closed
2018-12-05 15:19 hanes Resolution open => unable to reproduce
2018-12-05 15:30 woehrl01 Status closed => feedback
2018-12-05 15:30 woehrl01 Resolution unable to reproduce => reopened
2018-12-05 15:30 woehrl01 Note Added: 0001365