Create update_monitor_maria_db.sh

[rahmans1]

Briefly, what does this PR introduce?

Script to collect info from condor queue and populate a mysql database that will be updated at regular intervals

What kind of change does this PR introduce?

• Bug fix (issue #__)
• New feature (issue Script to update a database for job status monitoring using condor output #11)
• Documentation update
• Other: __

Please check if this PR fulfills the following:

• Tests for the changes have been added
• Documentation has been added / updated
• Changes have been communicated to collaborators

Does this PR introduce breaking changes? What changes might users need to make to their code?

No

Does this PR change default behavior?

No

[rahmans1]

@T-Britton Stuck on this error code. Seems like a permissions issue but cannot pinpoint it

ERROR 1142 (42000) at line 1: UPDATE command denied to user 'user'@'server' for table 'table'
./update_maria_db.sh: Error on line 105: mysql --host="$server" --user="$user" "$database" -e "$insert_query"

I wonder if it needs something like this To GRANT UPDATE privilege to user.

[T-Britton]

if a job finishes it may not be in condor_q. You instead need to find it in condor_history. Additionally, this looks like it updates all jobs. This will definitely not scale. during monitoring you should only query jobs which are in a non end state (e.g. Status 1 or 2 or sentinel values). Additionally, I wouldn't make monitoring do the inserts as there is a nice division of labor: during submission you do the inserts and the monitoring is a separate arm that just updates. the knock on effect is that your query for monitoring is just an update and not an insert unless it is a duplicate then update. I am actually not sure how the DB would recognize the duplicate key as there is no forced uniqueness on the other parts.

[rahmans1]

You are right about the scaling. Too many condor queries seem to slow down the entire thing. Gotta be a little smarter about how many times condor is queried within a short period of time.