Uploaded image for project: 'Grouper'
  1. Grouper
  2. GRP-503

ldappcng performance improvement - reduce the number of ldap queries

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.6.0, 1.6.1
    • Fix Version/s: 2.0.0
    • Component/s: provisioning
    • Labels:
      None

      Description

      I added exactly 100 groups to Grouper and added myself as a member to each group. Then I ran bulkSync and looked at my LDAP logs. There were 101 queries for me in this form...

      [15/Oct/2010:11:14:12 -0400] conn=4137 op=63 msgId=64 - SRCH base="ou=people,dc=duke,dc=edu" scope=1 filter="(duDukeID=0374183)" attrs="objectClass objectClass"

      And 1 query for me in this form...

      [15/Oct/2010:11:14:13 -0400] conn=4137 op=68 msgId=69 - SRCH base="duldapkey=ff0a7b5a-1dd1-11b2-91ba-9442cd793b76,ou=people,dc=duke,dc=edu" scope=0 filter="(objectClass=*)" attrs="objectClass objectClass ou description cn member"

      And there were 2 queries for each group (200 total group queries) like this...

      [15/Oct/2010:11:13:50 -0400] conn=4138 op=1 msgId=2 - SRCH base="cn=group0,ou=duke,ou=grouper,ou=groups,dc=duke,dc=edu" scope=0 filter="(objectClass=*)" attrs="objectClass objectClass ou description cn member"

      Then if I run bulkSync again when nothing changed in Grouper, ldappcng made the same number of searches for me, but this time only 1 query per group.

      Is it possible to reduce the number of queries made by ldappcng? Perhaps it can cache DNs to reduce the subject queries? This issue becomes very easily noticeable when you have a large number of memberships with a relatively few members. For instance, if you have 1 million memberships for 100,000 unique members, ldappcng would be making 1 million or more subject queries than needed during a bulkSync.

        Smart Checklist

          Attachments

            Activity

              People

              Assignee:
              tom.zeller@at.internet2.edu Tom Zeller
              Reporter:
              shilen.patel@at.internet2.edu Shilen Patel (duke.edu)
              Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: