View Issue Details

IDProjectCategoryView StatusLast Update
0004400SymmetricDSBugpublic2020-06-24 16:50
ReporterpmarzulloAssigned Topmarzullo 
Prioritynormal 
Status closedResolutionfixed 
Product Version3.11.5 
Target Version3.11.10Fixed in Version3.11.10 
Summary0004400: Initial load breaks when changing parameters during initial load request processing with clustering
DescriptionWhen an initial load is requested by changing the sym_node_security.initial_load_enabled=1
and sym_node_security.initial_load_time=null, the initial load job sees that and creates
a sym_table_reload_request with values of ALL/ALL.

Then the initial load job sees that request and calls the dataloadservice.insertreloadevents
method.

During that method, if the parameter datareload.batch.insert.transactional is set to false,
the work is done piece by piece and committed to the database piece by piece.

If a parameter is changed, the jobs are restarted, and the initial load job sends a thread
interrupt to the data loader service to interrupt the insert of the reload events. When
that occurs, whatever requests that have been committed so far remain, and the table_reload_request
remains unprocessed. This can happen also if the instance is restarted.

When the initial load job runs again, it starts over from the beginning with the same load
identifier because the load identifier is associated with the table_reload_request.

Symptoms that are seen are:
Too many extract requests and outgoing batches for the initial load.

Outgoing batches extract requests are attempted again, and the outgoing batch is marked as
'RQ' and the extract begins. The extract service extracts the data, renames the extract
file to ".done", and something is immediately deleting the extract file. So the initial load
never again makes any progress.
Additional InformationLog files attached.

symmetric.log.interrupted contains the interrupted dataloader. Look for "Insert reload events was interrupted".

symmetric.log.2.second_processing contains the retry of the same table reload request. Look for "Queueing up an initial load to node 906".
Tagsinitial/partial load

Activities

There are no notes attached to this issue.

Related Changesets

SymmetricDS: 3.11 5e78e703

2020-06-09 11:10:49

Philip Marzullo

Details Diff
0004400: Initial load breaks when changing parameters during initial
load request processing with clustering

0004400
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/service/IInitialLoadService.java Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/service/impl/DataService.java Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/service/impl/InitialLoadService.java Diff File

Issue History

Date Modified Username Field Change
2020-05-14 17:25 pmarzullo New Issue
2020-05-14 17:25 pmarzullo Status new => assigned
2020-05-14 17:25 pmarzullo Assigned To => pmarzullo
2020-05-14 17:25 pmarzullo Tag Attached: initial/partial load
2020-05-14 17:28 pmarzullo Additional Information Updated View Revisions
2020-06-09 11:19 pmarzullo Status assigned => resolved
2020-06-09 11:19 pmarzullo Resolution open => fixed
2020-06-09 11:19 pmarzullo Fixed in Version => 3.11.10
2020-06-09 12:00 Changeset attached => SymmetricDS 3.11 5e78e703
2020-06-24 16:50 admin Status resolved => closed
2020-06-24 16:50 admin Target Version => 3.11.10