Uploaded image for project: 'Grouper'
  1. Grouper
  2. GRP-3708

Incremental loader: prevent extra full syncs

    XMLWordPrintable

Details

    • Improvement
    • Resolution: Unresolved
    • Minor
    • None
    • None
    • grouperLoader
    • None

    Description

      The incremental loader (https://spaces.at.internet2.edu/display/Grouper/Grouper+loader+real+time+updates) will trigger a full sync when a transaction arrives for a group that did not previously exist. We would like an option that could prevent the full syncs from being triggered. If possible, we would like the incremental loader to create the group(s) with the transaction subject as a member. If only full sync can create groups, we would like some way to detect that memberships are pending the next sync (perhaps something in the grouper_incremental table row?).

      Our HR data loader creates/updates a multitude of ref groups. To avoid creating bunches of groups that are unlikely to have members (eg, faculty in the plumbing shop department), we don't create some classes of ref groups until a member arrives. If the incremental loader sees a transaction for a group that doesn't yet exist, the new member is the only member. The 'true up' of a full sync is not required. Also, we don't want extra full syncs during the day. 

      This was discussed briefly in the grouper slack channel on Nov 16, 2021.

       

      Attachments

        Activity

          People

            shilen.patel@at.internet2.edu Shilen Patel (duke.edu)
            gail.lift@at.internet2.edu Gail Lift
            Michael Gettes
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated: