Uploaded image for project: 'Grouper'
  1. Grouper
  2. GRP-2229

sql table provisioner: incremental ability needed

    XMLWordPrintable

Details

    • Improvement
    • Resolution: Unresolved
    • Minor
    • None
    • 2.4.0
    • provisioning
    • None
    • tier docker

    Description

      full sync may not be enough - is incremental hard to do?  Below shows 13k records - total of about 4MB.  I need to sync 800k rows - about 140MB.  Trying an initial run of the 800k - over 40 min.  Can there be periodic status written to the loader log as I have no idea where i am in the process even with DEBUG log.

      fullSync: true, key: EXT_IDENTITY, finalLog: true, state: deletes, databaseFrom: shib, tableFrom: EXT_IDENTITY, databaseTo: grouper, tableTo: uf_EXT_IDENTITY, totalCountFrom: 12972, fromGroupingUniqueValues: 12972, toGroupingUniqueValues: 12972, groupingsToDelete: 0, numberOfBatches: 2, currentBatch: 1, rowsSelectedFrom: 12972, rowsSelectedTo: 12972, sqlBatchExecute: 1 of 2, rowsNeedUpdate: 12972, took: 0:01:54.240

      Attachments

        Activity

          People

            chris.hyzer@at.internet2.edu Chris Hyzer (upenn.edu)
            michael.gettes.4@at.internet2.edu Michael Gettes (ufl.edu)
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated: