- Start Date: 2023-05-15
- RFC Type: feature
- RFC PR:
- RFC Status: draft
This RFC Proposes an improved CI experience for uploading source maps, debug symbols, and potentially other CI based operations by proposing a new way to get and manage access tokens specifically for this environment.
Today there are two ways to get access tokens for use with sentry-cli:
- per user access tokens
- internal organization integrations
Both are not great. The per user token is easy to get access to (which is why they are preferred in the docs still) but they run into the risk that a user departs an organization and an integration stops working. The organization integration flow is complex and requires elevated privileges. Either of those options have the additional complexity that there are a lot of extra settings to get right when configuring the tools. For instance the token itself does not know where it goes to, which requires organization slug and project slug to be set. All of this together means that the documentation does not put a user on the path of success but requires multiple separate steps to get everything in order.
We improved a lot of the inner workings of source maps and debug files at Sentry recently but these efforts are held back by the complexity of getting the token. The friction is still too high for many customers to make the necessary investment into getting source maps uploaded. From a documentation writing and onboarding experience, it's also not clear with the current system how the experience can be improved.
Additionally both Hybrid Cloud and Single Tenant would greatly benefit from automatically routing to the right URLs. Today the documentation is very quiet about how to get this system to work on a single tenant installation and customers are often required to work with CS to get source maps working.
The motivation is to add a new kind of token to Sentry which are fundamentally per-organization tokens, but with the ability to carry meta information that tools like sentry-cli can use to improve the user experience. These org level tokens can be created by anyone in the org, they can be given additional restrictions, and they can carry meta information such as routing data. For the purpose of this document they are called structural tokens.
The proposed token format is to leverage JWT as serialization format. The goals of the token align generally with both Macaroons and Biscuit but unfortunately the former standard has never seen much attention, and the latter is pretty new, not particularly proven and very complex. Either system however permits adding additional restrictions to the token which make them a very interesting choice for the use in our pipeline.
One of the benefits of having the tokens carry additional data is that the token alone has enough
information available to route to a Sentry installation. This means that sentry-cli or
any other tool just needs the token to even determine the host that the token should be
sent against. This benefit also applies to JWT or PASETO tokens which can be considered
for this as well. The RFC here thus proposes to encode this data into a regular JWT
token.
A serialized token is added a custom prefix sntrys_ (sentry structure) to make
it possible to detect it by security scrapers. Anyone handling such a token is
required to check for the sntrys_ prefix and disregard it before parsing it. This
can also be used by the client side to detect a structural token if the client is
interested in extracting data from the token.
We want to encode certain information into the tokens. The following attributes are defined:
sentry_site: references the target API URL that should be used. A token will always have a site in it and clients are not supposed to provide a fallback. For instance this would behttps://myorg.sentry.io/.sentry_org: a token is uniquely bound to an org, so the slug of that org is also always contained. Note that the slug is used rather than an org ID as the clients typically need these slugs to create API requests.
Potential fields:
sentry_projects: normally a token is valid for the entire org, but it could potentially be restricted. For operations such as source map uploads it might be interesting to issue tokens bound to a single project in which case the upload experience does not require providing the project slugs. However we currently do not want to start with this.
For JWT the facts could be encoded as custom claims:
{
"iss": "sentry.io",
"iat": 1684154626,
"sentry_site": "https://myorg.sentry.io/",
"sentry_org": "myorg"
}Encoded the token then is be sntrys_{encoded_jwt}.
Tokens are sent to the target sentry as Bearer token like normal. The server uses the
sntrys_ prefix to automatically detect a structural token. For existing tools that are
unaware of the structure behind structural tokens nothing changes.
Clients are strongly encouraged to parse out the containing structure of the token and to use this information to route requests. For the keys the following rules apply:
sentry_site: references the target API URL that should be used. A token will always have a site in it and clients are not supposed to provide an automatic fallback. If a site is not provided, one from the client config should be picked up (typicallysentry.io).org: a token is uniquely bound to an org, so the slug of that org is also always contained. Note that the slug is used rather than an org ID as the clients typically need these slugs to create API requests.
An example of this with a JWT token:
>>> import jwt
>>> tok = "sntrys_eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJzZW50cnkuaW8iLCJpYXQiOjE2ODQxNTQ2MjYsInNlbnRyeV9zaXRlIjoiaHR0cHM6Ly9teW9yZy5zZW50cnkuaW8vIiwic2VudHJ5X29yZyI6Im15b3JnIiwic2VudHJ5X3Byb2plY3RzIjpbIm15cHJvamVjdCJdfQ.ROnK3f72jGbH2CLkmswMIxXP1qZHDish9lN6kfCR0DU"
>>> jwt.decode(tok[7:], options={"verify_signature": False})
{
'iss': 'sentry.io',
'iat': 1684154626,
'sentry_site': 'https://myorg.sentry.io/',
'sentry_org': 'myorg',
'sentry_projects': ['myproject'] # not used currently
}The purpose of this change is to allow any organization member to issue tokens with little overhead. As users can already issue tokens with shocking levels of access to any of the orgs they are a member of there is a lot of room for improvement.
The proposed initial step is to only permit token issuance to support uploads and to permit all users in the org to issue such tokens. The tokens can be shown in the org's "Developer Settings" page under a new tab called "Tokens".
Such simple token issuance can then also take place in wizards and documentation pages.
Structural tokens change what needs to be communicated to users quite a bit. In particular
less information is necessary for tools that are compatible with structural tokens.
This for instance would change this complex webpack config from the docs which requires
matching org, project and manually creating a sentry token today:
const SentryWebpackPlugin = require("@sentry/webpack-plugin");
module.exports = {
devtool: "source-map",
plugins: [
new SentryWebpackPlugin({
url: "https://sentry.io/", // defaults to sentry.io
org: "demo-org",
project: "demo-project",
include: "./dist",
// Auth tokens can be obtained from https://sentry.io/settings/account/api/auth-tokens/
// and needs the `project:releases` and `org:read` scopes
authToken: process.env.SENTRY_AUTH_TOKEN,
}),
],
};With structural tokens this can be changed to a much more simplified version which also correctly handles Single Tenant:
const SentryWebpackPlugin = require("@sentry/webpack-plugin");
module.exports = {
devtool: "source-map",
plugins: [
new SentryWebpackPlugin({
authToken: "AUTO GENERATED TOKEN HERE",
project: "demo-project",
include: "./dist",
}),
],
};Some manual configuration remains as we still want ask users to provide the slug of the project explicitly to allow cross-org token issuance by default.
Additionally legacy tools will require more configuration. For tools not using sentry-cli but using the API directly, there might be a transitionary phase until the tool supports structural tokens. In that case the documentation would need to point out the correct way to configure this. The same applies to old installations of sentry-cli.
- The most important piece is the new token. As it behaves like any other token there is no immediate necessity for a tool to add support for structural tokens.
- Add a user interface to issue these new tokens on an org level.
- Add a user interface to issue these new tokens right from the documentation.
- Add support for structural tokens to sentry-cli to allow
organdprojectto be made optional. - Change documentation to no longer show
organdprojectfor tool config.
Addressing some questions that came up:
It would be possible to restrict tokens to a single project (or some projects). At a later point this might still be interesting when the tokens become more potent. For now these tokens can only be used to upload files which means that the damage that one org member can do against projects they are not a member of are limited. As such we are willing to accept the risk of issuing tokens across the entire org.
This also means that tools will still require the project slug to be provided for operations that are project bound. Today most of these operations are project bound, but we might want to investigate ways to bring most of these operations to the org level so that over time this information can be removed.
For instance for debug files there is no good reason why these files are not uploaded to org level to begin with. For source maps the situation is a bit more complex due to the optional nature of debug IDs. However in an increasing number of cases uploads should again be possible to the org level.
The benefit of a cross-org token is that this token can then later be used against other projects in the same pipeline without having to re-issue the token. For instance a CI job that first only uploads the frontend source maps might later want to do release creation for the backend as well. Having an overly restricted token would make this a more painful change.
Originally the idea came up to directly use DSNs for uploads. With debug IDs there is some potential to enable this as most of the system is write once and most indexing is now based on globally unique IDs. However this today does not work for a handful of reasons:
- Overwrites: DSNs are public and so someone who wants to disrupt a customer would be able to disrupt their processing by uploading invalid source maps or other broken files to a customer.
- DSNs do not have enough routing information: while a DSN encodes some information, it's only possible to go from a DSN to the ingestion system but not the API layer. A system could be added to relay to resolve the slugs and API URLs underpinning a DSN, but would reveal previously private information (the slugs) and requires a pre-flight to relay before making a request.
- DSN auth would really only work for source map uploads and debug file uploads, it could not be extended to other CI actions such as codecov report uploads or release creation due to the abuse potential caused by public DSNs.
- DSNs are limited to a single project and in some cases that might not be ideal. In particular for frontend + backend deployment scenarios being able to use one token to manage releases across projects might be desirable.
PASETO as an alternative to JWT can be an option. This should probably be decided based on what has most support. This proposal really only uses JWT for serialization of meta information, the actual validation of the JWT tokens only ever happens on the server side in which case the system can fully fall back to validating them based on what's stored in the database.
It's unclear if Biscuit is a great solution. There is a lot of complexity in it and tooling support is not great. However Biscuit is a potentially quite exiting idea because it would permit tools like sentry-cli to work with temporarily restricted tokens which reduces the chance of token leakage. The complexity of Biscuit however might be so prohibitive that it's not an appealing choice.