View Issue Details

IDProjectCategoryView StatusLast Update
0003748SymmetricDSBugpublic2018-11-28 11:00
ReporterdadancopAssigned Toelong 
Priorityurgent 
Status resolvedResolutionfixed 
Product Version3.9.14 
Target Version3.10.0Fixed in Version3.10.0 
Summary0003748: Batch was not complete
Description[From elong: re-opening issue to fix in 3.10. A batch can be corrupted on either sending or receiving side, which gets an error as "batch was not complete". We can clear the file from staging when we get that error, which should let it automatically recover.]

Before this, my symmetric run well on windows 2012
But after i restart server i get error like this :

2018-10-06 22:36:57,544 ERROR [corp-000] [ManageIncomingBatchListener] [corp-000-dataloader-19123] Failed to load batch global-8401 StackTraceKey [ProtocolException:3090605812]
2018-10-06 22:36:57,560 ERROR [corp-000] [DataLoaderService] [qtp1877453512-29] Failed to process batch StackTraceKey [ProtocolException:3090605812]
2018-10-06 22:36:57,732 ERROR [global] [AcknowledgeService] [global-push-default-7] The outgoing batch 000-8401 failed: The batch global-8401 was not complete
2018-10-06 22:37:00,341 INFO [corp-000] [IncomingBatchService] [corp-000-dataloader-19126] Retrying batch global-8401
2018-10-06 22:37:00,559 INFO [global] [DataExtractorService] [global-push-default-8] Reached the total byte threshold for initial load after 36 of 60 batches were sent for node '000' (sent 524792539 bytes, the max is 524288000). The remaining batches will be send on a subsequent sync.
2018-10-06 22:37:00,559 INFO [global] [PushService] [global-push-default-8] Push data sent to corp:000:000
2018-10-06 22:37:05,154 INFO [corp-000] [IncomingBatchService] [corp-000-dataloader-19127] Retrying batch idempierewh-3832
2018-10-06 22:37:05,248 ERROR [corp-000] [ManageIncomingBatchListener] [corp-000-dataloader-19127] Failed to load batch idempierewh-3832 StackTraceKey [ProtocolException:250523011]
2018-10-06 22:37:05,514 INFO [idempierewh] [PushService] [idempierewh-push-default-10] Push data sent to corp:000:000
2018-10-06 22:37:05,514 ERROR [corp-000] [DataLoaderService] [qtp1877453512-22] Failed to process batch StackTraceKey [ProtocolException:250523011]
2018-10-06 22:37:05,529 ERROR [idempierewh] [AcknowledgeService] [idempierewh-push-default-10] The outgoing batch 000-3832 failed: The batch idempierewh-3832 was not compl

What does cause it ?
Steps To ReproduceUsing demo, stop the store's pull job. Change an item at corp. View the batch CSV from Outgoing Batches. Find the batch csv file in tmp/corp-000/outgoing/123.csv and edit it. Modify the file to corrupt it. Run the store's pull job manually.

Corruption scenarios:
1. Choose a line to remove until the end of the file
2. Start removing in the middle of a line, like an insert, update, or delete

Activities

elong

2018-10-06 12:31

developer   ~0001258

Is this Oracle? If the row size exceeds 4K, you will get that error. You can set sym_channel.contains_big_lobs = 1 to work around it.

dadancop

2018-10-06 12:39

reporter   ~0001259

HI, ..this is postgres.

dadancop

2018-10-06 12:48

reporter   ~0001260

i hava changed the contain_big_lobs to 1, but the error still exists.

elong

2018-10-06 15:58

developer   ~0001261

Try stopping SymmetricDS, delete all the files from its "tmp" directory, then start again. (I'm wondering if there is a corrupted file in staging area.)

dadancop

2018-10-06 23:19

reporter   ~0001262

Thank you, elong. After delete the tmp directory. It Solved.

elong

2018-10-12 14:25

developer   ~0001272

Committed fix to 3.10

Related Changesets

SymmetricDS: 3.10 9ea369d7

2018-10-12 14:25:00

admin

Details Diff
0003748: Batch was not complete
0003748
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/common/ErrorConstants.java Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/io/stage/SimpleStagingDataWriter.java Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/service/impl/AcknowledgeService.java Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/service/impl/DataLoaderService.java Diff File

Issue History

Date Modified Username Field Change
2018-10-06 12:20 dadancop New Issue
2018-10-06 12:31 elong Note Added: 0001258
2018-10-06 12:39 dadancop Note Added: 0001259
2018-10-06 12:48 dadancop Note Added: 0001260
2018-10-06 15:58 elong Note Added: 0001261
2018-10-06 23:19 dadancop Note Added: 0001262
2018-10-10 07:31 hanes Status new => closed
2018-10-10 07:31 hanes Resolution open => no change required
2018-10-12 14:23 elong Assigned To => elong
2018-10-12 14:23 elong Status closed => assigned
2018-10-12 14:23 elong Resolution no change required => open
2018-10-12 14:23 elong Fixed in Version => 3.10.0
2018-10-12 14:23 elong Target Version => 3.10.0
2018-10-12 14:23 elong Summary Error after restart server => Batch was not complete
2018-10-12 14:23 elong Description Updated View Revisions
2018-10-12 14:23 elong Steps to Reproduce Updated View Revisions
2018-10-12 14:25 elong Status assigned => resolved
2018-10-12 14:25 elong Resolution open => fixed
2018-10-12 14:25 elong Note Added: 0001272
2018-11-28 11:00 admin Changeset attached => SymmetricDS 3.10 9ea369d7