Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Data access work #7

Open
wants to merge 9 commits into
base: master
Choose a base branch
from

Conversation

mescanne
Copy link

@mescanne mescanne commented Sep 2, 2019

Sample SQL script for analysing data access logs produced by GCS and BigQuery datasets.

I've adjusted README.md to add it in and provided instructions for setting up the sink, data access controls, as well as running the SQL.

WHEN service = 'bigquery' THEN
CASE
WHEN action IN ('bigquery.tables.delete',
'bigquery.datasets.delete',
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Align values with previous value. Same for the following WHEN statements.

-- Parts of the resource accessed
SPLIT(i.resource, '/') AS parts
FROM
`${PROJECT_ID}.data_access.cloudaudit_googleapis_com_data_access_*` d
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add AS before alias.

`${PROJECT_ID}.data_access.cloudaudit_googleapis_com_data_access_*` d
CROSS JOIN d.protopayload_auditlog.authorizationInfo i
WHERE
i.resource IS NOT NULL AND
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Move AND to the next line.

WHERE
  filter 1
  AND filter 2

SPLIT(i.resource, '/') AS parts
FROM
`${PROJECT_ID}.data_access.cloudaudit_googleapis_com_data_access_*` d
CROSS JOIN d.protopayload_auditlog.authorizationInfo i
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Align CROSS JOIN with FROM.

CASE
WHEN service = 'storage' THEN
CASE
-- See granular permissions here: https://cloud.google.com/storage/docs/access-control/iam-permissions
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please intend CASE statements. Use 2 spaces for all indentation through out the script.

-- See granular permissions here: https://cloud.google.com/bigquery/docs/access-control#bq-permissions
WHEN service = 'bigquery' THEN
CASE
WHEN action IN ('bigquery.tables.delete',
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please indent WHEN statements.

granted,
-- Project is of the resource or, if not there,
-- then for the method accessing it (eg for buckets)
CASE
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you need a ELSE to set a default value here? Otherwise, it will create NULLs.

actor,
-- Translate the action into an operation (READ/WRITE/ADMIN)
CASE
WHEN service = 'storage' THEN
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you think by checking the action value, you can determine the action type? So, maybe simplify the code a little with one level of CASE statement?

Same for service='bigquery' code below.

@@ -0,0 +1,8 @@
{
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add a description, what this file is for?

@freedomofnet
Copy link
Collaborator

Hi mescanne,

Please take a look of the comments when you have time.

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants