Uploaded image for project: 'COmanage'
  1. COmanage
  2. CO-2819

CoPetitions index view causes large database load and slow query

    XMLWordPrintable

Details

    Description

      The production CILogon deployment for LSST has approximately 19,555 rows in the cm_co_petitions table:

      MySQL [lsst_registry]> select count(*) from cm_co_petitions;
      +----------+
      | count(*) |
      +----------+
      |    19555 |
      +----------+
      1 row in set (0.438 sec)
      

      Using the UI and browsing to People > CO Petitions -> View All results in the following URL:

      registry/co_petitions/index/co:2/sort:CoPetition.created/direction:desc

      Loading that URL causes the following SQL query to be sent to the database:

      SELECT `CoPetition`.`id`, `CoPetition`.`co_enrollment_flow_id`, `CoPetition`.`co_id`, `CoPetition`.`cou_id`, `CoPetition`.`enrollee_org_identity_id`, `CoPetition`.`archived_org_identity_id`, `CoPetition`.`enrollee_co_person_id`, `CoPetition`.`enrollee_co_person_role_id`, `CoPetition`.`petitioner_co_person_id`, `CoPetition`.`sponsor_co_person_id`, `CoPetition`.`approver_co_person_id`, `CoPetition`.`co_invite_id`, `CoPetition`.`authenticated_identifier`, `CoPetition`.`petitioner_token`, `CoPetition`.`enrollee_token`, `CoPetition`.`return_url`, `CoPetition`.`approver_comment`, `CoPetition`.`status`, `CoPetition`.`created`, `CoPetition`.`modified`, `CoPetition`.`co_petition_id`, `CoPetition`.`revision`, `CoPetition`.`deleted`, `CoPetition`.`actor_identifier`, `CoPetition`.`vetting_request_id`, `CoPetition`.`reference_identifier`, `ApproverCoPerson`.`id`, `ApproverCoPerson`.`co_id`, `ApproverCoPerson`.`status`, `ApproverCoPerson`.`date_of_birth`, `ApproverCoPerson`.`timezone`, `ApproverCoPerson`.`created`, `ApproverCoPerson`.`modified`, `ApproverCoPerson`.`co_person_id`, `ApproverCoPerson`.`revision`, `ApproverCoPerson`.`deleted`, `ApproverCoPerson`.`actor_identifier`, `CoEnrollmentFlow`.`id`, `CoEnrollmentFlow`.`name`, `CoEnrollmentFlow`.`co_id`, `CoEnrollmentFlow`.`authz_level`, `CoEnrollmentFlow`.`authz_cou_id`, `CoEnrollmentFlow`.`authz_co_group_id`, `CoEnrollmentFlow`.`my_identity_shortcut`, `CoEnrollmentFlow`.`match_policy`, `CoEnrollmentFlow`.`enable_person_find`, `CoEnrollmentFlow`.`approval_required`, `CoEnrollmentFlow`.`approver_co_group_id`, `CoEnrollmentFlow`.`verify_email`, `CoEnrollmentFlow`.`email_verification_mode`, `CoEnrollmentFlow`.`invitation_validity`, `CoEnrollmentFlow`.`require_authn`, `CoEnrollmentFlow`.`notification_co_group_id`, `CoEnrollmentFlow`.`status`, `CoEnrollmentFlow`.`notify_from`, `CoEnrollmentFlow`.`verification_subject`, `CoEnrollmentFlow`.`verification_body`, `CoEnrollmentFlow`.`verification_template_id`, `CoEnrollmentFlow`.`notify_on_approval`, `CoEnrollmentFlow`.`approval_subject`, `CoEnrollmentFlow`.`approval_body`, `CoEnrollmentFlow`.`approval_template_id`, `CoEnrollmentFlow`.`denial_template_id`, `CoEnrollmentFlow`.`notify_on_finalize`, `CoEnrollmentFlow`.`finalization_template_id`, `CoEnrollmentFlow`.`introduction_text`, `CoEnrollmentFlow`.`conclusion_text`, `CoEnrollmentFlow`.`t_and_c_mode`, `CoEnrollmentFlow`.`redirect_on_submit`, `CoEnrollmentFlow`.`redirect_on_confirm`, `CoEnrollmentFlow`.`redirect_on_finalize`, `CoEnrollmentFlow`.`return_url_allowlist`, `CoEnrollmentFlow`.`ignore_authoritative`, `CoEnrollmentFlow`.`duplicate_mode`, `CoEnrollmentFlow`.`co_theme_id`, `CoEnrollmentFlow`.`theme_stacking`, `CoEnrollmentFlow`.`establish_authenticators`, `CoEnrollmentFlow`.`establish_cluster_accounts`, `CoEnrollmentFlow`.`created`, `CoEnrollmentFlow`.`modified`, `CoEnrollmentFlow`.`co_enrollment_flow_id`, `CoEnrollmentFlow`.`revision`, `CoEnrollmentFlow`.`deleted`, `CoEnrollmentFlow`.`actor_identifier`, `CoEnrollmentFlow`.`sor_label`, `CoEnrollmentFlow`.`match_server_id`, `CoEnrollmentFlow`.`regenerate_expired_verification`, `CoEnrollmentFlow`.`request_vetting`, `CoEnrollmentFlow`.`approver_template_id`, `CoEnrollmentFlow`.`introduction_text_pa`, `Cou`.`id`, `Cou`.`co_id`, `Cou`.`name`, `Cou`.`description`, `Cou`.`parent_id`, `Cou`.`lft`, `Cou`.`rght`, `Cou`.`created`, `Cou`.`modified`, `Cou`.`cou_id`, `Cou`.`revision`, `Cou`.`deleted`, `Cou`.`actor_identifier`, `EnrolleeCoPerson`.`id`, `EnrolleeCoPerson`.`co_id`, `EnrolleeCoPerson`.`status`, `EnrolleeCoPerson`.`date_of_birth`, `EnrolleeCoPerson`.`timezone`, `EnrolleeCoPerson`.`created`, `EnrolleeCoPerson`.`modified`, `EnrolleeCoPerson`.`co_person_id`, `EnrolleeCoPerson`.`revision`, `EnrolleeCoPerson`.`deleted`, `EnrolleeCoPerson`.`actor_identifier`, `PetitionerCoPerson`.`id`, `PetitionerCoPerson`.`co_id`, `PetitionerCoPerson`.`status`, `PetitionerCoPerson`.`date_of_birth`, `PetitionerCoPerson`.`timezone`, `PetitionerCoPerson`.`created`, `PetitionerCoPerson`.`modified`, `PetitionerCoPerson`.`co_person_id`, `PetitionerCoPerson`.`revision`, `PetitionerCoPerson`.`deleted`, `PetitionerCoPerson`.`actor_identifier`, `SponsorCoPerson`.`id`, `SponsorCoPerson`.`co_id`, `SponsorCoPerson`.`status`, `SponsorCoPerson`.`date_of_birth`, `SponsorCoPerson`.`timezone`, `SponsorCoPerson`.`created`, `SponsorCoPerson`.`modified`, `SponsorCoPerson`.`co_person_id`, `SponsorCoPerson`.`revision`, `SponsorCoPerson`.`deleted`, `SponsorCoPerson`.`actor_identifier`, `ApproverPrimaryName`.`id`, `ApproverPrimaryName`.`honorific`, `ApproverPrimaryName`.`given`, `ApproverPrimaryName`.`middle`, `ApproverPrimaryName`.`family`, `ApproverPrimaryName`.`suffix`, `ApproverPrimaryName`.`type`, `ApproverPrimaryName`.`language`, `ApproverPrimaryName`.`co_person_id`, `ApproverPrimaryName`.`org_identity_id`, `ApproverPrimaryName`.`primary_name`, `ApproverPrimaryName`.`source_name_id`, `ApproverPrimaryName`.`created`, `ApproverPrimaryName`.`modified`, `ApproverPrimaryName`.`name_id`, `ApproverPrimaryName`.`revision`, `ApproverPrimaryName`.`deleted`, `ApproverPrimaryName`.`actor_identifier`, `EnrolleePrimaryName`.`id`, `EnrolleePrimaryName`.`honorific`, `EnrolleePrimaryName`.`given`, `EnrolleePrimaryName`.`middle`, `EnrolleePrimaryName`.`family`, `EnrolleePrimaryName`.`suffix`, `EnrolleePrimaryName`.`type`, `EnrolleePrimaryName`.`language`, `EnrolleePrimaryName`.`co_person_id`, `EnrolleePrimaryName`.`org_identity_id`, `EnrolleePrimaryName`.`primary_name`, `EnrolleePrimaryName`.`source_name_id`, `EnrolleePrimaryName`.`created`, `EnrolleePrimaryName`.`modified`, `EnrolleePrimaryName`.`name_id`, `EnrolleePrimaryName`.`revision`, `EnrolleePrimaryName`.`deleted`, `EnrolleePrimaryName`.`actor_identifier`, `PetitionerPrimaryName`.`id`, `PetitionerPrimaryName`.`honorific`, `PetitionerPrimaryName`.`given`, `PetitionerPrimaryName`.`middle`, `PetitionerPrimaryName`.`family`, `PetitionerPrimaryName`.`suffix`, `PetitionerPrimaryName`.`type`, `PetitionerPrimaryName`.`language`, `PetitionerPrimaryName`.`co_person_id`, `PetitionerPrimaryName`.`org_identity_id`, `PetitionerPrimaryName`.`primary_name`, `PetitionerPrimaryName`.`source_name_id`, `PetitionerPrimaryName`.`created`, `PetitionerPrimaryName`.`modified`, `PetitionerPrimaryName`.`name_id`, `PetitionerPrimaryName`.`revision`, `PetitionerPrimaryName`.`deleted`, `PetitionerPrimaryName`.`actor_identifier`, `SponsorPrimaryName`.`id`, `SponsorPrimaryName`.`honorific`, `SponsorPrimaryName`.`given`, `SponsorPrimaryName`.`middle`, `SponsorPrimaryName`.`family`, `SponsorPrimaryName`.`suffix`, `SponsorPrimaryName`.`type`, `SponsorPrimaryName`.`language`, `SponsorPrimaryName`.`co_person_id`, `SponsorPrimaryName`.`org_identity_id`, `SponsorPrimaryName`.`primary_name`, `SponsorPrimaryName`.`source_name_id`, `SponsorPrimaryName`.`created`, `SponsorPrimaryName`.`modified`, `SponsorPrimaryName`.`name_id`, `SponsorPrimaryName`.`revision`, `SponsorPrimaryName`.`deleted`, `SponsorPrimaryName`.`actor_identifier` FROM `lsst_registry`.`cm_co_petitions` AS `CoPetition` LEFT JOIN `lsst_registry`.`cm_co_people` AS `ApproverCoPerson` ON (`ApproverCoPerson`.`id` = `CoPetition`.`approver_co_person_id` AND `ApproverCoPerson`.`co_person_id` IS NULL AND `ApproverCoPerson`.`deleted` IS NOT true) LEFT JOIN `lsst_registry`.`cm_co_enrollment_flows` AS `CoEnrollmentFlow` ON (`CoEnrollmentFlow`.`id` = `CoPetition`.`co_enrollment_flow_id`) LEFT JOIN `lsst_registry`.`cm_cous` AS `Cou` ON (`Cou`.`id` = `CoPetition`.`cou_id` AND `Cou`.`cou_id` IS NULL AND `Cou`.`deleted` IS NOT true) LEFT JOIN `lsst_registry`.`cm_co_people` AS `EnrolleeCoPerson` ON (`EnrolleeCoPerson`.`id` = `CoPetition`.`enrollee_co_person_id` AND `EnrolleeCoPerson`.`co_person_id` IS NULL AND `EnrolleeCoPerson`.`deleted` IS NOT true) LEFT JOIN `lsst_registry`.`cm_co_people` AS `PetitionerCoPerson` ON (`PetitionerCoPerson`.`id` = `CoPetition`.`petitioner_co_person_id` AND `PetitionerCoPerson`.`co_person_id` IS NULL AND `PetitionerCoPerson`.`deleted` IS NOT true) LEFT JOIN `lsst_registry`.`cm_co_people` AS `SponsorCoPerson` ON (`SponsorCoPerson`.`id` = `CoPetition`.`sponsor_co_person_id` AND `SponsorCoPerson`.`co_person_id` IS NULL AND `SponsorCoPerson`.`deleted` IS NOT true) LEFT JOIN `lsst_registry`.`cm_names` AS `ApproverPrimaryName` ON (`ApproverPrimaryName`.`co_person_id` = `ApproverCoPerson`.`id` AND `ApproverPrimaryName`.`primary_name` = true AND `ApproverPrimaryName`.`name_id` IS NULL AND `ApproverPrimaryName`.`deleted` IS NOT true) LEFT JOIN `lsst_registry`.`cm_names` AS `EnrolleePrimaryName` ON (`EnrolleePrimaryName`.`co_person_id` = `EnrolleeCoPerson`.`id` AND `EnrolleePrimaryName`.`primary_name` = true AND `EnrolleePrimaryName`.`name_id` IS NULL AND `EnrolleePrimaryName`.`deleted` IS NOT true) LEFT JOIN `lsst_registry`.`cm_names` AS `PetitionerPrimaryName` ON (`PetitionerPrimaryName`.`co_person_id` = `PetitionerCoPerson`.`id` AND `PetitionerPrimaryName`.`primary_name` = true AND `PetitionerPrimaryName`.`name_id` IS NULL AND `PetitionerPrimaryName`.`deleted` IS NOT true) LEFT JOIN `lsst_registry`.`cm_names` AS `SponsorPrimaryName` ON (`SponsorPrimaryName`.`co_person_id` = `SponsorCoPerson`.`id` AND `SponsorPrimaryName`.`primary_name` = true AND `SponsorPrimaryName`.`name_id` IS NULL AND `SponsorPrimaryName`.`deleted` IS NOT true)  WHERE `CoPetition`.`co_id` = 2 AND `CoPetition`.`co_petition_id` IS NULL AND `CoPetition`.`deleted` IS NOT true   ORDER BY `CoPetition`.`created` desc  LIMIT 25; 

      That SQL query takes many (> 30) seconds to return, causes a spike in database CPU, and results in a gateway timeout since the PHP code does not return a response for a long time.

      Attachments

        Issue Links

          Activity

            People

              shayna.atkinson@at.internet2.edu Shayna Atkinson
              scott.koranda@at.internet2.edu Scott Koranda
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: