composites slow

r

Justin Robinson Yesterday at 8:10 AM
What are the recommendations for building groups based on specific compliance? We have one particular department in our organization that has made thousands of groups which are a composite to one or two specific groups that are very large. Making changes to those groups with all the composite dependencies times out. Thoughts?
17 replies

Jeffrey Crawford 13 hours ago
When you say times out, I take it you mean the UI? When we have to deal with a very large group, we tend to either use the WS or the GSH which can just sit there and work.
However I think we made a suggestion a while back that Grouper UI could default to only show direct memberships if the indirect membership is "high". Or perhaps just be a setting.

Justin Robinson 13 hours ago
Hi @Jeffrey Crawford (UCLA) - it does timeout in the UI, though I have tried using a GSH script to insert using the new Daemon job structure. Seems like it takes long enough that the Daemon job gets terminated.

Justin Robinson 13 hours ago
I suspect the usage may not be an efficient way for this particular department to manage compliance needs, but trying to find the appropriate alternatives for them. These are basically large groups like - all students or all employees that have taken FERPA training.

Justin Robinson 13 hours ago
that group is then tied together with n* groups via a composite intersection to enforce the compliance.

Justin Robinson 13 hours ago
we’re wanting to adjust the way they do things to a more efficient way

Jeffrey Crawford 13 hours ago
For technologically efficient, someone from the development team may need to weigh in. However generally speaking the TIER deployment guide does recommend utilizing core groups to drive access/compliance etc. to other groups.
I don't think I've ever see GSH time out. I've had a script run for nearly 13 hours one time. I did execute it under "screen" just in case my connection died.

Jeffrey Crawford 13 hours ago
That said I did have GSH run out of memory one time in the past though, but I think that was a bug that got fixed.

Chris Hyzer 13 hours ago
are they manual groups you are intersecting? would a rule help that just removes users who arent in the eligible population?
https://spaces.dev.at.internet2.edu/display/Grouper/Grouper+rules+use+case+-+Composite-ng+intersection

Chris Hyzer 13 hours ago
What database are you using?

Justin Robinson 13 hours ago
we use postgres

Chris Hyzer 13 hours ago
when you say "changes" you mean editing the composite right?

Chris Hyzer 13 hours ago
not just adding members

Justin Robinson 13 hours ago
adding members to the central group would then flow to all the composites

Justin Robinson 13 hours ago
i’ve noticed that if we have a loader manage the larger groups it handles it differently than one off additions

Justin Robinson 13 hours ago
the rule looks like a good option here

Chris Hyzer 13 hours ago
i mean is adding members slow, or just changing the composite?

Chris Hyzer 9 hours ago
shilen and i discussed and we have ideas for making composites perform better. hopefully we can improve this before too long...