View Issue Details

IDProjectCategoryView StatusLast Update
0002959SymmetricDSBugpublic2017-01-19 14:56
ReportergatokinoAssigned Tochenson 
PriorityhighSeveritymajorReproducibilityalways
Status closedResolutionfixed 
Product Version3.8.11 
Target VersionFixed in Version3.8.14 
Summary0002959: BatchStagingManager deleting incoming files from staging that are not yet processed
DescriptionWe have a situation during the inital load with a lot of slow loading incoming batches. For that reason in the temporary directory we have aprox 250 files for incoming batches.

BatchStaginManager is called and is deleting files that are present in temp directory for incoming batches but not yet present in the sym_incoming_batch sql table.

Because of that we have later the following error:

***** ERROR [org.jumpmind.symmetric.service.impl.DataLoaderService] (client-pull-default-4) Failed while parsing batch: java.lang.IllegalStateException: There is no content to read. Memory buffer was empty and D:\********\incoming\00000\0000008322.ready was not found.
    at org.jumpmind.symmetric.io.stage.StagedResource.getReader(StagedResource.java:184)
    at org.jumpmind.symmetric.io.data.reader.ProtocolDataReader.open(ProtocolDataReader.java:121)
    at org.jumpmind.symmetric.io.data.DataProcessor.process(DataProcessor.java:86)
    at org.jumpmind.symmetric.service.impl.DataLoaderService$LoadIntoDatabaseOnArrivalListener$1.call(DataLoaderService.java:938)
    at org.jumpmind.symmetric.service.impl.DataLoaderService$LoadIntoDatabaseOnArrivalListener$1.call(DataLoaderService.java:913)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)


As all other files are deleted the rest of the incoming batches, not yet processed, are redownloaded and retried ...
Additional InformationWorkaround was to set a high time to live for staging files and to use that value in the if that is deciding to erase:

line 59 -> (recordIncomingBatchesEnabled && !incomingBatches.contains(batchId) && resourceIsOld)

Also we have added a the line 44 -> (!resource.isInUse()) condition in a hope that this will fix the problem.
TagsNo tags attached.

Activities

chenson

2017-01-12 09:45

administrator   ~0000949

This is definitely a bug. We'll fix it in 3.8.14. Thank you for reporting it!

chenson

2017-01-12 14:07

administrator   ~0000950

I checked in a fix that I am testing. The approach is to not purge if the largest existing incoming batch for the specific node is smaller than the batch being evaluated for purging.

Related Changesets

SymmetricDS: 3.8 250a7da6

2017-01-12 14:04:21

chenson

Details Diff
0002959: BatchStagingManager deleting incoming files from staging that
are not yet processed

0002959
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/io/stage/BatchStagingManager.java Diff File

SymmetricDS: 3.8 92f32684

2017-01-12 14:06:04

chenson

Details Diff
0002959: BatchStagingManager deleting incoming files from staging that
are not yet processed

0002959
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/io/stage/BatchStagingManager.java Diff File

Issue History

Date Modified Username Field Change
2017-01-12 04:08 gatokino New Issue
2017-01-12 09:44 chenson Assigned To => chenson
2017-01-12 09:44 chenson Status new => assigned
2017-01-12 09:45 chenson Note Added: 0000949
2017-01-12 14:04 chenson Status assigned => resolved
2017-01-12 14:04 chenson Fixed in Version => 3.8.14
2017-01-12 14:04 chenson Resolution open => fixed
2017-01-12 14:07 chenson Note Added: 0000950
2017-01-12 15:00 chenson Changeset attached => SymmetricDS 3.8 92f32684
2017-01-12 15:00 chenson Changeset attached => SymmetricDS 3.8 250a7da6
2017-01-19 14:56 chenson Status resolved => closed