Uploaded image for project: 'Grouper'
  1. Grouper
  2. GRP-1125

loader looks up all subjects when it doesnt need to

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 2.2.1
    • Fix Version/s: None
    • Component/s: grouperLoader
    • Labels:
      None

      Description

      From: grouper-core-request On Behalf Of Shilen Patel
      Sent: Wednesday, February 18, 2015 7:46 AM
      To: grouper-core
      Subject: [grouper-core] GRP-1051 - grouper loader performance bulk subject queries

      Hi Chris,

      Regarding this feature you added in 2.2.1. It looks like in 2.2.0 and in 2.2.1 when this feature is turned off, the loader will only lookup subjects that are changing memberships. So if I have a single loader group with 50K members and only 5 members are changing, it’ll do approximately 5 subject lookups. With the feature in 2.2.1, it seems like it wants to resolve all 50K members regardless of whether there are any changes.

      Not sure if this behavior was expected. The issue came up after NYU upgraded from 2.2.0 to 2.2.1. With the default settings, some of their loader jobs went from taking ~75 seconds to about ~75 minutes. After they turned off the feature, their performance was back to the way it was. So they should be good for now but just reporting what they were seeing.

      Thanks!

      • Shilen

        Smart Checklist

          Attachments

            Activity

              People

              Assignee:
              chris.hyzer@at.internet2.edu Chris Hyzer (upenn.edu)
              Reporter:
              chris.hyzer@at.internet2.edu Chris Hyzer (upenn.edu)
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Dates

                Created:
                Updated: