View Issue Details
|ID||Project||Category||View Status||Date Submitted||Last Update|
|0004471||SymmetricDS||Bug||public||2020-07-09 15:34||2020-07-09 16:03|
|Target Version||3.10.13||Fixed in Version||3.10.13|
|Summary||0004471: DataExtractorService fails to set StagedResource for extracted file to DONE|
|Description||The DataExtractorService is unable to set the StagedResource for extracted file to the DONE state.|
The reason for this can be reproduced using the steps below.
This is a problem when the number of files that need to be extracted for a table is large, and the synchronization catches up with the extract of the table and the synchronization can not find the extracted file (because the state is CREATE instead of DONE).
The synchronization attempts to interrupt the extraction, then resets the batches for the table back to RQ status, and sets the sym_table_reload_request status back to NE.
The problem occurs when the SymmetricDS node is running in clustered mode, and the initial load extract is running on another instance of the cluster than the synchronization, and the synchronization can not interrupt the initial load.
The batches are reset to RQ, and the sym_table_reload_request is reset to NE, but the initial load extract continues on because it is running on the other node. The initial load extract then marks the rest of the batches for the current table to status NE, and marks the sym_table_reload_request status to OK when it finishes extracting.
Then you are in a state where the beginning batches for the current table are in the RQ status, the rest of the batches for the table are in the NE status, and the reload request status is OK. The batches are stuck in the RQ status.
|Steps To Reproduce||1. Start an initial load with the initial load extract running in the background.|
2. Run the StagingManager job to clean the staging purge.
3. While the StagingManager clean is running, the initial load extract is unable to retrieve the StagedResource because of synchronized method clean() and synchronized block in find() in the StagingManager class.
4. When the StagingManager clean is complete, the file that was written to remains in the CREATE state, and the initial load extractor goes onto the next batch extract.
|Tags||initial/partial load, staging|
The change to the StagedResource was to undo some changes that I interpreted as network file share locking.
That is not the case, so the old code is put back.
The real change for this is in StagingManager find() method.
||Code has been merged into 3.11 and 3.12 as well.|
SymmetricDS: 3.10 57882a01
Philip MarzulloDetails Diff
|0004471: DataExtractorService fails to set StagedResource for extracted
file to DONE
|mod - symmetric-io/src/main/java/org/jumpmind/symmetric/io/stage/StagedResource.java||Diff File|
|mod - symmetric-io/src/main/java/org/jumpmind/symmetric/io/stage/StagingManager.java||Diff File|
|2020-07-09 15:34||pmarzullo||New Issue|
|2020-07-09 15:34||pmarzullo||Status||new => assigned|
|2020-07-09 15:34||pmarzullo||Assigned To||=> pmarzullo|
|2020-07-09 15:34||pmarzullo||Tag Attached: initial/partial load|
|2020-07-09 15:34||pmarzullo||Tag Attached: staging|
|2020-07-09 15:34||pmarzullo||Relationship added||related to 0003250|
|2020-07-09 16:00||Changeset attached||=> SymmetricDS 3.10 57882a01|
|2020-07-09 16:02||pmarzullo||Note Added: 0001729|
|2020-07-09 16:03||pmarzullo||Status||assigned => resolved|
|2020-07-09 16:03||pmarzullo||Resolution||open => fixed|
|2020-07-09 16:03||pmarzullo||Fixed in Version||=> 3.10.13|
|2020-07-09 16:03||pmarzullo||Note Added: 0001730|