Skip to content

ADVANCED_SYNC_OPTIMIZATION.md

Latest commit

 

History

History
84 lines (58 loc) · 4.58 KB

ADVANCED_SYNC_OPTIMIZATION.md

File metadata and controls

84 lines (58 loc) · 4.58 KB
Raw

Advanced Sync Optimization

This document explains the optimization implemented for the Google Tasks CLI advanced synchronization feature.

Overview

The advanced sync process has been optimized to improve performance and reduce unnecessary data transfer. The optimization follows a two-phase approach:

  1. Full Sync for Empty Databases: When the local database is empty, all remote tasks are pulled and stored directly.
  2. Incremental Sync for Existing Databases: When the local database contains data, only tasks from a specified time range are pulled.

Implementation Details

1. Full Sync for Empty Databases

When the local database is detected as empty, the system performs a full sync:

  • All remote tasks are pulled from Google Tasks
  • Tasks are stored directly to the main local tables
  • No temporary tables or complex comparison logic is used

This approach is efficient for initial setup or when starting fresh with a Google Tasks account.

2. Incremental Sync for Existing Databases

When the local database contains data, the system performs an incremental sync:

  1. Check the configured pull data range (default: 90 days)
  2. Pull only tasks modified within that time range using combined date filters:
    • completedMin: Tasks completed since the minimum date
    • dueMin: Tasks due since the minimum date
    • updatedMin: Tasks updated since the minimum date
  3. Follow the existing processing workflow with these filtered tasks

3. API Call Optimization

To reduce the number of API calls, the system now uses combined filters in a single API call per task list:

  • Instead of 3 separate calls per task list (completed, due, updated), we now make 1 call per task list
  • Each call includes all three filters, returning tasks that match any of the criteria
  • Results are deduplicated by task ID to avoid processing the same task multiple times

4. Operation Skipping Optimization

To reduce unnecessary processing time:

  • Push operations are skipped entirely when there are no tasks to push
  • Pull operations are skipped entirely when there are no tasks to pull
  • This prevents time-consuming operations when no changes are needed

5. Configuration

The sync behavior can be configured using the sync.pull_range_days setting in the configuration file:

sync:
  pull_range_days: 90  # Default to 3 months

Users can adjust this value to balance between sync performance and data completeness.

Benefits

  1. Reduced API Calls: Using combined filters reduces API calls by ~66%
  2. Faster Sync Times: Processing fewer tasks results in faster synchronization
  3. Lower Bandwidth Usage: Transferring only necessary data reduces bandwidth consumption
  4. Improved Performance: Less data to process means better overall performance
  5. Better API Quota Usage: Fewer API calls help stay within Google Tasks API quotas
  6. Reduced Idle Processing: Skipping operations when no changes are needed saves CPU time

Technical Implementation

The optimization is implemented in the AdvancedSyncManager class with two main methods:

  • _perform_full_sync(): Handles full sync for empty databases
  • _perform_incremental_sync(): Handles incremental sync for existing databases

Date filtering is implemented in the GoogleTasksClient.list_tasks_with_combined_filters() method.

Operation skipping is implemented in:

  • _execute_push_operations()
  • _execute_pull_operations()

Future Improvements

  1. Smart Range Adjustment: Dynamically adjust the pull range based on user behavior
  2. Task Importance Filtering: Prioritize important tasks even if they're outside the date range
  3. Cache Optimization: Store metadata about task lists to further optimize API calls
  4. Parallel Processing: Process multiple task lists in parallel to reduce overall sync time