Unable to remote load a tegola config from S3

[matrottier]

Anybody tried a config file in S3 bucket + Tegola Lambda ?
i'm missing something with the permission on the bucket.
it's working only if my bucket is public.
i tried a "bucket policy" with the ARN of my lambda's role AND a permission policie for my lambda's role.
I would like it to be possible only with the role of the lambda.

It's more a question on the "so easy" AWS autorisation 🙄 , but a take a chance 🤞

[ARolek]

@matrottier I have set it up numerous times, though I just looked at my setup and I do have my S3 buckets publicly available as I'm using the architecture in this article: https://medium.com/@alexrolek/the-serverless-vector-map-stack-lives-22835d341d7d

Do you want every request to run through Lambda, or just tiles that have not yet been generated?

[matrottier]

i try to achieve something like that :

this is to allow editing of the file by other colleagues without all the permissions on lambda

[ARolek]

@matrottier I think that's a dangerous approach unless you're creating some sort of version prefix as the config changes. Without being mindful of the changes, your cache could contain outputs from numerous configs and you don't know which version they're derived from. A couple of questions for you:

• how often is your data changing?
• are you using s3 as a cache? If so, would you be open to not having an object cache or are the tiles too slow to produce on demand? You could potentially cache at the CDN layer instead of an object store and have a sensible TTL.

[matrottier]

for the config, i was thinking about clearing the cache when the config is updated. something like that

• data is updated every ~30min
• for now, no cache at all and the tiles will easy to produce.

[ARolek]

@matrottier ok in that case, I think could skip using S3 entirely, and just set a TTL at the CDN for 30 minutes. Let's assume you dropped S3 from your architecture, would that make the permission issue you're encountering going away, or is the config file in S3?

[matrottier]

config file in S3.
if my config file is full public, it's working with the "Amazon S3 virtual-hosted–style URL" (https://bucket-name.s3.region-code.amazonaws.com/key-name)

[ARolek]

Ok, so that's the underlying issue (I'm going to adjust the title of this issue). When you first opened this, I thought you were referring to using S3 as your cache, but what you're doing is trying to load a remote config from S3. Tegola only has http(s) loading capabilities for the remote config. For S3, you can have it operate as a static web server, and I believe that requires the public be publicly available.

Tegola could be enhanced to load configs from S3, if you want to take a pass at implementing it.