From: Shilen Patel firstname.lastname@example.org
Sent: Friday, February 16, 2018 2:06 PM
To: Hyzer, Chris <email@example.com>
Subject: Re: [grouper-core] RE: [grouper-users] StaleObjectStateException thrown by loader jobs after patch
Yeah I can’t imagine anybody would want to remove those builtins.
From: <firstname.lastname@example.org> on behalf of "Hyzer, Chris" <email@example.com>
Date: Friday, February 16, 2018 at 1:37 PM
Cc: "firstname.lastname@example.org" <email@example.com>
Subject: [grouper-core] RE: [grouper-users] StaleObjectStateException thrown by loader jobs after patch
Im kinda thinking that the builtins should be hard coded or configured somewhere else.
i.e. if someone overrides, they cant easily upgrade. Know what I mean? I think its ok for Grouper just to mandate we aren’t change logging certain things…
From: Shilen Patel firstname.lastname@example.org
Sent: Friday, February 16, 2018 1:33 PM
To: Scott Koranda <email@example.com>; Hyzer, Chris <firstname.lastname@example.org>
Cc: grouper-users <email@example.com>
Subject: Re: [grouper-users] StaleObjectStateException thrown by loader jobs after patch
Just to make sure, you do have the latest grouper.base.properties with this config:
grouper.attribute.namesOfAttributeDefNamesToIgnoreAuditsChangeLogPit.elConfig = $$grouper.attribute.rootStem$$:attestation:attestationCalculatedDaysLeft,$$grouper.attribute.rootStem$$:attestation:attestationLastEmailedDate,$$grouper.attribute.rootStem$$:loaderMetadata:grouperLoaderMetadataLastFullMillisSince1970,$$grouper.attribute.rootStem$$:loaderMetadata:grouperLoaderMetadataLastIncrementalMillisSince1970,$$grouper.attribute.rootStem$$:loaderMetadata:grouperLoaderMetadataLastSummary
(making sure it includes all the loaderMetadata attributes)
And you haven’t overwritten this config in grouper.properties. Just making sure.
On 2/16/18, 1:16 PM, "firstname.lastname@example.org on behalf of Scott Koranda" <email@example.com on behalf of firstname.lastname@example.org> wrote:
> How many groups are loaded? Millions?
This morning I see this:
SQL> SELECT COUNT FROM GROUPER_GROUPS;
SQL> SELECT COUNT FROM GROUPER_CHANGE_LOG_ENTRY_TEMP;
SQL> SELECT COUNT FROM GROUPER_CHANGE_LOG_ENTRY_TEMP WHERE STRING04 NOT LIKE 'etc:attribute:loaderMetadata:%';
SQL> SELECT COUNT FROM GROUPER_CHANGE_LOG_ENTRY;
There were no entries in either the change log or the change log temp
I do not understand how so many entries were put into
GROUPER_CHANGE_LOG_ENTRY_TEMP by the new functionality.
I also drastically underestimated that amount of time it will take for
this deployment to process 27 million rows in the change log. It is
clear it will not be able to do it in a reasonable time (less than a few
weeks) so I am going to TRUNCATE the tables now instead.
> > Maybe I am not, however, understanding how those 27 million rows in
> > GROUPER_CHANGE_LOG_TEMP come about. Are they only created when the
> > "real" loader jobs run?
> Hmmmm. I deleted two of the new attributes, and change the incorrect value in a list of groups with two groups. I ran the loader. I got two change log entries per loaded group. One for etc:attribute:loaderMetadata:grouperLoaderMetadataLastSummary and one for etc:attribute:loaderMetadata:grouperLoaderMetadataLastFullMillisSince1970
> I believe you already have assignments on loaded groups for grouperLoaderMetadataLoaded and grouperLoaderMetadataGroupId.
> Not saying this is better, but before the patch, you could create those attributes, and run a GSH script to assign them to loaded groups, maybe a sleeping a second after a dozen assignments. Up to you.
I will consider it as an option. Thanks.
Another issue with these patches: every time a loader job runs it
generates thousands of log entries at the WARN level with a message like
2018-02-16 13:04:55,826: [DefaultQuartzScheduler_Worker-9] WARN
- Non-fatal error removing metadata on group:
This is filling up the Grouper logs.
Is this expected?