Details
-
Bug
-
Resolution: Fixed
-
Minor
-
4.5.5
-
None
Description
After our upgrade from 2.6.19 to 4.5.5, we're seeing unique-constraint errors in our SQL Provisioning Full Syncs. We have date columns managed by the database schema to know when rows are created (when_created TIMESTAMP DEFAULT current_timestamp), and, therefore, know that the duplicate rows being created are duplicates of rows that have been in the database table for multiple weeks; this is not some kind of race condition.
In order to try to fix this problem, we removed all Provisioning data for this provisioner, so we might have made the problem worse. However, in theory, all that data shouldn't prevent a FullSync from reading the Target and Grouper and updating the Target with the necessary changes.
To get things running for the immediate term, we added a trigger to our grouper-provisioned table to discard duplicates. Therefore, we're not expecting to see these errors any more, but we could disable the trigger and test when a fix is available.
| grouper-ui;provisioning.log;2023-09-14T04:10:05,112: v2 [Thread-220] ERROR e.int.middleware.grouper.app.provisioning.targetDao.GrouperProvisionerTargetDaoAdapter - [] - Provisioner 'iam_unifieddb_group_memberships' (vwsvdonn) Error inserting memberships, e.g. Mship(matchingAttrs: LinkedHashSet(1): [0]: [id, val: MultiKey[gt:ref:affiliations:guest@gt, 992D5FF8E4BA8A6F96E4F06C8B1A786A], compareVal: MultiKey[gt:ref:affiliations:guest@gt, 992D5FF8E4BA8A6F96E4F06C8B1A786A], currentValue: true], attr[group_name]: "gt:ref:affiliations:guest@gt", attr[subject_id]: "992D5FF8E4BA8A6F96E4F06C8B1A786A", attr[subject_source_id]: "gted-people", ins group_name "gt:ref:affiliations:guest@gt", ins subject_id "992D5FF8E4BA8A6F96E4F06C8B1A786A", ins subject_source_id "gted-people", recalcObject: true, create: true, selectProcessed: true) |
| (vwsvdonn): java.lang.RuntimeException: sql: insert into grouper.memberships_from_grouper(group_name,subject_id,subject_source_id) values (?, ?, ?), |
| (vwsvdonn): at edu.internet2.middleware.grouperClient.jdbc.GcDbAccess.callbackResultSet(GcDbAccess.java:2361) |
| (vwsvdonn): at edu.internet2.middleware.grouperClient.jdbc.GcDbAccess.executeBatchSql(GcDbAccess.java:2407) |
| (vwsvdonn): at edu.internet2.middleware.grouper.app.sqlProvisioning.SqlProvisionerCommands.insertObjects(SqlProvisionerCommands.java:508) |
| (vwsvdonn): at edu.internet2.middleware.grouper.app.sqlProvisioning.SqlProvisioningDao.insertMemberships(SqlProvisioningDao.java:1102) |
| (vwsvdonn): at edu.internet2.middleware.grouper.app.provisioning.targetDao.GrouperProvisionerTargetDaoAdapter.insertMembershipsHelper(GrouperProvisionerTargetDaoAdapter.java:4012) |
| (vwsvdonn): at edu.internet2.middleware.grouper.app.provisioning.targetDao.GrouperProvisionerTargetDaoAdapter$31.callLogic(GrouperProvisionerTargetDaoAdapter.java:4072) |
| (vwsvdonn): at edu.internet2.middleware.grouper.app.provisioning.targetDao.GrouperProvisionerTargetDaoAdapter$31.callLogic(GrouperProvisionerTargetDaoAdapter.java:4066) |
| (vwsvdonn): at edu.internet2.middleware.grouper.util.GrouperCallable$1.callback(GrouperCallable.java:203) |
| (vwsvdonn): at edu.internet2.middleware.grouper.GrouperSession.callbackGrouperSession(GrouperSession.java:1000) |
| (vwsvdonn): at edu.internet2.middleware.grouper.util.GrouperCallable.callLogicWithSessionIfExists(GrouperCallable.java:200) |
| (vwsvdonn): at edu.internet2.middleware.grouper.util.GrouperCallable.call(GrouperCallable.java:166) |
| (vwsvdonn): at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) |
| (vwsvdonn): at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) |
| (vwsvdonn): at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) |
| (vwsvdonn): at java.base/java.lang.Thread.run(Thread.java:833) |
| (vwsvdonn): Caused by: java.sql.BatchUpdateException: Batch entry 0 insert into grouper.memberships_from_grouper(group_name,subject_id,subject_source_id) values ('gt:ref:affiliations:guest@gt', '992D5FF8E4BA8A6F96E4F06C8B1A786A', 'gted-people') was aborted: ERROR: duplicate key value violates unique constraint "memberships_from_grouper_idx1" |
| (vwsvdonn): Detail: Key (group_name, subject_source_id, subject_id)=(gt:ref:affiliations:guest@gt, gted-people, 992D5FF8E4BA8A6F96E4F06C8B1A786A) already exists. Call getNextException to see other errors in the batch. |
| (vwsvdonn): at org.postgresql.jdbc.BatchResultHandler.handleCompletion(BatchResultHandler.java:186) |
| (vwsvdonn): at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:571) |
| (vwsvdonn): at org.postgresql.jdbc.PgStatement.internalExecuteBatch(PgStatement.java:893) |
| (vwsvdonn): at org.postgresql.jdbc.PgStatement.executeBatch(PgStatement.java:916) |
| (vwsvdonn): at org.postgresql.jdbc.PgPreparedStatement.executeBatch(PgPreparedStatement.java:1684) |
| (vwsvdonn): at com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeBatch(NewProxyPreparedStatement.java:2544) |
| (vwsvdonn): at edu.internet2.middleware.grouperClient.jdbc.GcDbAccess.callbackResultSet(GcDbAccess.java:2344) |
| (vwsvdonn): ... 14 more |
| (vwsvdonn): Caused by: org.postgresql.util.PSQLException: ERROR: duplicate key value violates unique constraint "memberships_from_grouper_idx1" |
| (vwsvdonn): Detail: Key (group_name, subject_source_id, subject_id)=(gt:ref:affiliations:guest@gt, gted-people, 992D5FF8E4BA8A6F96E4F06C8B1A786A) already exists. |
| (vwsvdonn): at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2676) |
| (vwsvdonn): at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2366) |
| (vwsvdonn): at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2099) |
| (vwsvdonn): at org.postgresql.core.v3.QueryExecutorImpl.flushIfDeadlockRisk(QueryExecutorImpl.java:1456) |
| (vwsvdonn): at org.postgresql.core.v3.QueryExecutorImpl.sendQuery(QueryExecutorImpl.java:1481) |
| (vwsvdonn): at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:546) |
| (vwsvdonn): ... 19 more |
Attachments
Issue Links
- is related to
-
-
- Resolved
-