Uploaded image for project: 'Grouper'
  1. Grouper
  2. GRP-3708

Incremental loader: prevent extra full syncs



    • Improvement
    • Resolution: Unresolved
    • Minor
    • None
    • None
    • grouperLoader
    • None


      The incremental loader (https://spaces.at.internet2.edu/display/Grouper/Grouper+loader+real+time+updates) will trigger a full sync when a transaction arrives for a group that did not previously exist. We would like an option that could prevent the full syncs from being triggered. If possible, we would like the incremental loader to create the group(s) with the transaction subject as a member. If only full sync can create groups, we would like some way to detect that memberships are pending the next sync (perhaps something in the grouper_incremental table row?).

      Our HR data loader creates/updates a multitude of ref groups. To avoid creating bunches of groups that are unlikely to have members (eg, faculty in the plumbing shop department), we don't create some classes of ref groups until a member arrives. If the incremental loader sees a transaction for a group that doesn't yet exist, the new member is the only member. The 'true up' of a full sync is not required. Also, we don't want extra full syncs during the day. 

      This was discussed briefly in the grouper slack channel on Nov 16, 2021.





            shilen.patel@at.internet2.edu Shilen Patel (duke.edu)
            gail.lift@at.internet2.edu Gail Lift
            Michael Gettes
            0 Vote for this issue
            1 Start watching this issue