View Issue Details

IDProjectCategoryView StatusLast Update
0002309SymmetricDSBugpublic2019-04-23 17:16
Reporterondrejhala Assigned Toelong  
Priorityurgent 
Status closedResolutionfixed 
Product Version3.7.19 
Target Version3.9.0Fixed in Version3.9.0 
Summary0002309: Error during initial load - send from stage
DescriptionDuring the batch transfering phase of an initial load process the java.io.IOException: Stream closed is thrown.

2015-06-11 16:57:24,325 INFO [SymmetricDS] [DataProcessor] [qtp1548776324-15] Batch '331', for node 'node-ihil-pilsen', for process 'send from stage' has been processing for 3781 seconds. The following stats have been gathered: {STATEMENTCOUNT=6128100, BYTECOUNT=12891848902, LINENUMBER=6128100}
Steps To ReproduceTransfer initial load batch with size bigger than 12GB. The transfer throws the exception after 12GB are transferred.

See attached log files.
Additional Information2015-06-11 16:57:26,684 ERROR [SymmetricDS] [NodeCommunicationService] [symmetricds-extract-1] Failed to execute EXTRACT for node node-ihil-pilsen
org.jumpmind.exception.IoException: java.io.IOException: Stream closed
    at org.jumpmind.symmetric.io.data.writer.StagingDataWriter.print(StagingDataWriter.java:117)
    at org.jumpmind.symmetric.io.data.writer.AbstractProtocolDataWriter.println(AbstractProtocolDataWriter.java:252)
    at org.jumpmind.symmetric.io.data.writer.AbstractProtocolDataWriter.write(AbstractProtocolDataWriter.java:161)
    at org.jumpmind.symmetric.service.impl.DataExtractorService$MultiBatchStagingWriter.write(DataExtractorService.java:1252)
    at org.jumpmind.symmetric.io.data.writer.NestedDataWriter.write(NestedDataWriter.java:64)
    at org.jumpmind.symmetric.model.ProcessInfoDataWriter.write(ProcessInfoDataWriter.java:65)
    at org.jumpmind.symmetric.io.data.writer.NestedDataWriter.write(NestedDataWriter.java:64)
    at org.jumpmind.symmetric.io.data.writer.TransformWriter.write(TransformWriter.java:204)
    at org.jumpmind.symmetric.io.data.DataProcessor.forEachDataInTable(DataProcessor.java:200)
    at org.jumpmind.symmetric.io.data.DataProcessor.forEachTableInBatch(DataProcessor.java:170)
    at org.jumpmind.symmetric.io.data.DataProcessor.process(DataProcessor.java:116)
    at org.jumpmind.symmetric.service.impl.DataExtractorService.extractOutgoingBatch(DataExtractorService.java:733)
    at org.jumpmind.symmetric.service.impl.DataExtractorService.execute(DataExtractorService.java:1096)
    at org.jumpmind.symmetric.service.impl.NodeCommunicationService$2.run(NodeCommunicationService.java:317)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: Stream closed
    at java.io.BufferedWriter.ensureOpen(BufferedWriter.java:116)
    at java.io.BufferedWriter.write(BufferedWriter.java:221)
    at java.io.Writer.write(Writer.java:157)
    at java.io.Writer.append(Writer.java:269)
    at org.jumpmind.symmetric.io.data.writer.StagingDataWriter.print(StagingDataWriter.java:114)
    ... 16 more
ndle(AsyncHttpConnection.java:77)
    at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:606)
    at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:46)
    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:603)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:538)
    at java.lang.Thread.run(Thread.java:745)
TagsNo tags attached.

Activities

ondrejhala

2015-06-12 07:19

reporter  

TransferToStage.zip (495,770 bytes)

elong

2019-04-23 17:16

developer   ~0001419

Fixed in 3.9 and newer, which uses multiple batches for a table to break up the transfer and loading.

Issue History

Date Modified Username Field Change
2015-06-12 07:19 ondrejhala New Issue
2015-06-12 07:19 ondrejhala File Added: TransferToStage.zip
2019-04-23 17:16 elong Assigned To => elong
2019-04-23 17:16 elong Status new => resolved
2019-04-23 17:16 elong Resolution open => fixed
2019-04-23 17:16 elong Fixed in Version => 3.9.0
2019-04-23 17:16 elong Note Added: 0001419
2019-04-23 17:16 elong Status resolved => closed