Details
-
Improvement
-
Resolution: Fixed
-
Minor
-
2.4.0.patch
-
None
Description
By default, Oracle's jdbc driver makes very small fetches of data as ResultSets are fetched. This led to incredibly slow performance reading larger loader queries at Georgia Tech where it took hours to cursor through a 3M row result across our AWS link. (The problem of these small fetches are worst with network latency)
The solution to this is JDBC's Statement's setFetchSize(N) method (defaults to 10, at least with Oracle).
While this is a general problem for Grouper's different connections to databases, this is mostly a problem for Loading and, probably, Subject Sources and not the Grouper Database because Grouper has to have very little network latency to its main database.
Grouper should have a db.<dbname>.fetchSize property that it then uses for its database queries. There might be a hibernate property for cases when Grouper has too much latency to its database.