View Issue Details

IDProjectCategoryView StatusLast Update
0001575SymmetricDSNew Featurepublic2015-09-11 20:14
Reporterelong Assigned Toelong  
Prioritynormal 
Status closedResolutionfixed 
Product Version3.7.21 
Target Version3.7.21Fixed in Version3.7.21 
Summary0001575: Offline data synchronization
DescriptionAllow SymmetricDS to run without using the network. Batches would write to files in an outgoing folder. The user would pick up the outgoing files and copy them to the incoming folder on the destination node. A job would pick up the incoming files and load them.
TagsNo tags attached.

Relationships

related to 0000213 closedelong Allow the SymmetricDS API to be called from the command line 

Activities

pjhaynes

2014-02-18 21:28

reporter   ~0000486

Last edited: 2014-02-18 21:30

View 2 revisions

My project needs to be able to push changes from a master database to several read-only replicas. In many situations, we have no network at all and move files around on memory sticks. We operate in remote locations where networks are impossible. We need to provided updates once a day to stakeholders who have a read-only copy of our application/database for reporting purposes.

Ideally, we would want the following features:
1. The target system will prevent import of data if a previous update file has not been applied (ie, changes 1-100 were applied, changes 101-200 were missed because the file was never applied, then an attempt is made to apply the file containing changes 201-300)
2. A user can can trigger the output of an update file (or trigger the system to package up for transport files that were written as changes were made), and the system will include all database changes since the previous update file was generated
3. A user can specify the range of changes to include, to allow inclusion of database changes a given slave may have missed (ie, a screen allowing a user to specify the generation of an update file containing changes 101-300 to catch the target system up in the example above for feature 1)
4. We should be able to specify the maximum file size

LittleFish

2015-02-19 17:45

reporter   ~0000671

I would like to add my support to this issue.
I am dealing with several developing countries where the situation described is the rule rather than the exception, and having this possibility would make it possible for us to adopt SymmetricDS in our applications.
Extension: The satellite DBs would be able to update the DB (or parts of it), and this should be replicated back to the master.
As I understand it, the normal workflow of SymmetricDS already produces a file of changes as an intermediate step in its processing, before connecting to the DB to update, and the DB being updated first receives the changes as a file.
So, implementing this request should require "only" to separate the producing and reading of the channges files from the actual network communications, and allow the system to produce and read these files without ever establishing a connection.

Related Changesets

SymmetricDS: 3.7 fcabdf8f

2015-08-13 10:11:02

admin

Details Diff
0001575: Offline data synchronization Affected Issues
0001575
mod - symmetric-core/src/main/resources/symmetric-default.properties Diff File
add - symmetric-core/src/main/java/org/jumpmind/symmetric/transport/file/FileTransportManager.java Diff File
add - symmetric-core/src/main/java/org/jumpmind/symmetric/transport/file/FileOutgoingTransport.java Diff File
add - symmetric-core/src/main/java/org/jumpmind/symmetric/transport/file/FileIncomingTransport.java Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/transport/TransportManagerFactory.java Diff File
add - symmetric-core/src/main/java/org/jumpmind/symmetric/service/impl/OfflinePushService.java Diff File
add - symmetric-core/src/main/java/org/jumpmind/symmetric/service/impl/OfflinePullService.java Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/service/impl/NodeCommunicationService.java Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/service/impl/DataLoaderService.java Diff File
add - symmetric-core/src/main/java/org/jumpmind/symmetric/service/IOfflinePushService.java Diff File
add - symmetric-core/src/main/java/org/jumpmind/symmetric/service/IOfflinePullService.java Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/service/IDataLoaderService.java Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/service/ClusterConstants.java Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/model/ProcessInfoKey.java Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/model/NodeCommunication.java Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/common/ParameterConstants.java Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/common/Constants.java Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/ISymmetricEngine.java Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/AbstractSymmetricEngine.java Diff File
add - symmetric-client/src/main/java/org/jumpmind/symmetric/job/OfflinePushJob.java Diff File
add - symmetric-client/src/main/java/org/jumpmind/symmetric/job/OfflinePullJob.java Diff File
mod - symmetric-client/src/main/java/org/jumpmind/symmetric/job/JobManager.java Diff File

Issue History

Date Modified Username Field Change
2014-02-13 16:12 elong New Issue
2014-02-13 16:12 elong Relationship added related to 0000213
2014-02-18 21:28 pjhaynes Note Added: 0000486
2014-02-18 21:30 pjhaynes Note Edited: 0000486 View Revisions
2015-02-19 17:45 LittleFish Note Added: 0000671
2015-08-13 15:00 admin Changeset attached => SymmetricDS 3.7 fcabdf8f
2015-08-13 19:42 elong Assigned To => elong
2015-08-13 19:42 elong Status new => resolved
2015-08-13 19:42 elong Resolution open => fixed
2015-08-13 19:42 elong Product Version => 3.7.21
2015-08-13 19:42 elong Fixed in Version => 3.7.21
2015-08-13 19:42 elong Target Version => 3.7.21
2015-09-11 20:14 chenson Status resolved => closed