Description
Our ChangelogTemp-to-Changelog process stopped processing due to errors like this:{}
groovy:000> loaderRunOneJob("CHANGE_LOG_changeLogTempToChangeLog");
|
ERROR java.lang.RuntimeException:
|
java.lang.RuntimeException: Active PITGroup with sourceId=4b3b35de1931406c83a644539b985428 not found,
|
Problem in HibernateSession: HibernateSession (67c5a4cc): new, notReadonly, READ_WRITE_NEW, notActiveTransaction, session (f299239),
|
jobName: CHANGE_LOG_changeLogTempToChangeLog
|
at edu.internet2.middleware.grouper.app.loader.GrouperLoader.runOnceByJobName (GrouperLoader.java:1803)
|
at edu.internet2.middleware.grouper.app.loader.GrouperLoader.runOnceByJobName (GrouperLoader.java:1747)
|
at edu.internet2.middleware.grouper.app.gsh.loaderRunOneJob.invoke (loaderRunOneJob.java:95)
|
at edu.internet2.middleware.grouper.app.gsh.loaderRunOneJob$invoke.call (Unknown Source)
|
at groovysh_evaluate.loaderRunOneJob (groovysh_evaluate:4)
|
The group id, indeed, is not found in the Groups nor the PITGroups tables.
This 2017 email thread 2017 email thread (Scott Koranda and Shilen) referred to two corrective utilities – processMissingActivePITMembers() and syncAllPITTables() – neither of which found any problems in our environment.
After focusing on deleting individual entries from CLET:
select * from grouper_change_log_entry_temp where string02='4b3b35de1931406c83a644539b985428'
|
This got the CLT2CL job moving forward, but it eventually got stuck on other missing groups.
Therefore, we used the following to find and delete all the entries referring to missing groups:
CREATE TEMPORARY TABLE valid_group_ids AS SELECT id from grouper_pit_groups UNION select id from grouper_groups;
|
|
|
create index on valid_group_ids (id);
|
select count(*) from valid_group_ids;
|
|
|
drop table if exists invalid_change_log_entry_temps;
|
|
|
create temporary table invalid_change_log_entry_temps AS
|
SELECT grouper_change_log_entry_temp.*
|
FROM grouper_change_log_entry_temp
|
LEFT JOIN valid_group_ids ON (valid_group_ids.id=string02)
|
WHERE change_log_type_id='729b73f3709d4a89a8b99fc2d774482d'
|
AND valid_group_ids.id IS NULL;
|
|
|
select count(*) from invalid_change_log_entry_temps;
|
|
|
select * from invalid_change_log_entry_temps
|
ORDER BY created_on asc;
|
|
|
DELETE FROM grouper_change_log_entry_temp
|
WHERE id IN (SELECT id FROM invalid_change_log_entry_temps);
|
NOTE: The temporary tables aren't mathematically necessary, but they helped with performance and could have helped us if we needed to delete the entries in chunks to shorten lock durations.
It seems like these entries should not stop all CLT processing.
Attachments
Issue Links
- is related to
-
GRP-1568 active PIT objects not found should be handled gracefully
-
- Resolved
-