Handle case issue between subject source and loader source

From: <Hyzer>, Chris <mchyzer@isc.upenn.edu>
Date: Wednesday, June 15, 2016 at 11:23 AM
To: grouper-announce <grouper-core@internet2.edu>
Subject: [grouper-core] Fw: Troubles with the Grouper Loader

Shilen, is this a bug/enhancement? i.e. when the loader compares to records it doesnt consider case, but the subject source does... do we need another catch there? i.e. if it doesnt find it, then resolve the subject, and see if that one is there? so it doesnt affect performance but doesnt remove all add all?

thanks,
chris

From: Sean Mason <sean.mason@uwaterloo.ca>
Sent: Thursday, June 9, 2016 10:16 AM
To: Hyzer, Chris; grouper-users@internet2.edu
Subject: RE: Troubles with the Grouper Loader

Case! Case is the answer.
Oracle is returning in all caps, grouper_members in lower.

select lower(uid) is the solution.

Thank you so much. Now to go regrow some hair, and send a note on the issues I’m having with the PSP

Thanks again,
Sean.

From: Hyzer, Chris mchyzer@isc.upenn.edu
Sent: Thursday, June 09, 2016 10:10 AM
To: Sean Mason <sean.mason@uwaterloo.ca>; grouper-users@internet2.edu
Subject: RE: Troubles with the Grouper Loader

Might also be a case problem… is the case coming back from oracle the same as the case where the subject is resolved? i.e. look at the results of the query vs the data in the grouper_members table…

Thanks,
Chris

From: grouper-users-request@internet2.edu grouper-users-request@internet2.edu On Behalf Of Sean Mason
Sent: Thursday, June 09, 2016 9:33 AM
To: grouper-users@internet2.edu
Subject: [grouper-users] Troubles with the Grouper Loader

Hi There,

I’ve been experimenting with Grouper for some time with some success. Recently a small production use case to fill a need that arose suddenly that I’m looking to fill with Grouper, and I’m having a very difficult time. Time is now tight, and I will have to move to another strategy to make deadlines, but I would like to know where I’ve gone wrong. I will break the two issues I’m having into two email to hopefully keep any offered help focused.

I’m running Grouper 2.3.0, on RHEL 7, Oracle JDK 1.8.0_91. The repository is Postgres 9.2.14.

I have a single SQL_SIMPLE load job.
Parameters:
grouperLoaderDbName: CSPR
grouperLoaderQuartzCron: 0 0 * * * ?
grouperLoaderQuery: select uid as SUBJECT_ID from sysadm.firsttimeundergrad_vw
grouperLoaderScheduleType: CRON
grouperLoaderType: SQL_SIMPLE

When I run loaderRunOneJob(group) from gsh, I see:
loader ran successfully, inserted 8307 memberships, deleted 8307 memberships, total membership count: 8307, unresolvable subjects: 0
If I run the loaderRunOneJob(group) from gsh once again, I see:
loader ran successfully, inserted 8307 memberships, deleted 8307 memberships, total membership count: 8307, unresolvable subjects: 0
I know the contents of the view have not changed between runs.

If I set loader.failsafe.use = true, loader.failsafe.maxPercentRemove = 30, the load fails as it’s trying to remove the entire population before re-adding.

Surely this isn’t the expected behavior. I’m at a loss where else to turn. Is there another configuration step that is required? Is something amiss?

Please advise,
Sean.