Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

226: support for array-shaped payloads #240

Merged
merged 3 commits into from
Mar 25, 2024
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion README-INPUT-OUTPUT.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ The state machine accepts the following input parameters:
* **lambdaARN** (required, string): unique identifier of the Lambda function you want to optimize
* **powerValues** (optional, string or list of integers): the list of power values to be tested; if not provided, the default values configured at deploy-time are used (by default: 128MB, 256MB, 512MB, 1024MB, 1536MB, and 3008MB); you can provide any power values between 128MB and 10,240MB (⚠️ [New AWS accounts have reduced concurrency and memory quotas, 3008MB max](https://docs.aws.amazon.com/lambda/latest/dg/gettingstarted-limits.html))
* **num** (required, integer): the # of invocations for each power configuration (minimum 5, recommended: between 10 and 100)
* **payload** (string, object, or list): the static payload that will be used for every invocation (object or string); when using a list, a weighted payload is expected in the shape of `[{"payload": {...}, "weight": X }, {"payload": {...}, "weight": Y }, {"payload": {...}, "weight": Z }]`, where the weights `X`, `Y`, and `Z` are treated as relative weights (not percentages); more details below in the [Weighted Payloads section](#user-content-weighted-payloads)
* **payload** (string, object, or list): the static payload that will be used for every invocation (object or string); when using a list the payload will be treated as a weighted payload if and only if it's in the shape of `[{"payload": {...}, "weight": X }, {"payload": {...}, "weight": Y }, {"payload": {...}, "weight": Z }]`, where the weights `X`, `Y`, and `Z` are treated as relative weights (not percentages); more details below in the [Weighted Payloads section](#user-content-weighted-payloads)
* **payloadS3** (string): a reference to Amazon S3 for large payloads (>256KB), formatted as `s3://bucket/key`; it requires read-only IAM permissions, see `payloadS3Bucket` and `payloadS3Key` below and find more details in the [S3 payloads section](#user-content-s3-payloads)
* **parallelInvocation** (false by default): if true, all the invocations will be executed in parallel (note: depending on the value of `num`, you may experience throttling when setting `parallelInvocation` to true)
* **strategy** (string): it can be `"cost"` or `"speed"` or `"balanced"` (the default value is `"cost"`); if you use `"cost"` the state machine will suggest the cheapest option (disregarding its performance), while if you use `"speed"` the state machine will suggest the fastest option (disregarding its cost). When using `"balanced"` the state machine will choose a compromise between `"cost"` and `"speed"` according to the parameter `"balancedWeight"`
Expand Down Expand Up @@ -65,6 +65,9 @@ You can use different alias names such as `dev`, `test`, `production`, etc. If y

### Weighted Payloads

> [!IMPORTANT]
> Your payload will only be treated as a weighted payload if it adheres to the JSON structure that follows. Otherwise, it's assumed to be an array-shaped payload.

Weighted payloads can be used in scenarios where the payload structure and the corresponding performance/speed could vary a lot in production and you'd like to include multiple payloads in the tuning process.

You may want to use weighted payloads also in case of functions with side effects that would be hard or impossible to test with the very same payload (for example, a function that deletes records from a database).
Expand Down
20 changes: 13 additions & 7 deletions lambda/utils.js
Original file line number Diff line number Diff line change
Expand Up @@ -383,13 +383,8 @@ module.exports._fetchS3Object = async(bucket, key) => {
* Generate a list of `num` payloads (repeated or weighted)
*/
module.exports.generatePayloads = (num, payloadInput) => {
if (Array.isArray(payloadInput)) {
// if array, generate a list of payloads based on weights

// fail if empty list or missing weight/payload
if (payloadInput.length === 0 || payloadInput.some(p => !p.weight || !p.payload)) {
throw new Error('Invalid weighted payload structure');
}
if (Array.isArray(payloadInput) && utils.isWeightedPayload(payloadInput)) {
alexcasalboni marked this conversation as resolved.
Show resolved Hide resolved
// if weighted array, generate a list of payloads based on weights

if (num < payloadInput.length) {
throw new Error(`You have ${payloadInput.length} payloads and only "num"=${num}. Please increase "num".`);
Expand Down Expand Up @@ -429,6 +424,17 @@ module.exports.generatePayloads = (num, payloadInput) => {
}
};

/**
* Check if payload is an array where each element contains the property "weight"
*/
module.exports.isWeightedPayload = (payload) => {
/**
* Return true only if the input is a non-empty array where the elements contain a weight property.
* e.g. [{ "payload": {...}, "weight": 5 }, ...]
*/
return Array.isArray(payload) && payload.every(p => p.weight && p.payload) && !!payload.length;
};

/**
* Convert payload to string, if it's not a string already
*/
Expand Down
73 changes: 66 additions & 7 deletions test/unit/test-utils.js
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ lambdaMock.on(DeleteFunctionCommand).resolves({});
lambdaMock.on(CreateAliasCommand).resolves({});
lambdaMock.on(DeleteAliasCommand).resolves({});
lambdaMock.on(InvokeCommand).resolves({});
lambdaMock.on(UpdateAliasCommand).resolves({})
lambdaMock.on(UpdateAliasCommand).resolves({});
const s3Mock = awsV3Mock.mockClient(S3Client);
s3Mock.reset();
s3Mock.on(GetObjectCommand).resolves({
Expand Down Expand Up @@ -611,12 +611,22 @@ describe('Lambda Utils', () => {
});
});

it('should explode if invalid weighted payloads', async () => {
expect(() => utils.generatePayloads(10, [])).to.throwError();
expect(() => utils.generatePayloads(10, [{}])).to.throwError();
expect(() => utils.generatePayloads(10, [1, 2, 3])).to.throwError();
expect(() => utils.generatePayloads(10, [{ weight: 1 }])).to.throwError();
expect(() => utils.generatePayloads(10, [{ payload: {} }])).to.throwError();
it('should return input array as output if not weighted', async() => {
let payloads = [
[],
[{}],
[1, 2, 3],
[{ weight: 1 }],
[{ payload: {}, weight: 1 }, { payload: {}}],
[{ payload: {} }],
];

payloads.forEach(payload => {
let output = utils.generatePayloads(10, payload);

expect(output.length).to.be(10);
expect(output.every(p => p === JSON.stringify(payload))).to.be(true);
});
});

it('should explode if num < count(payloads)', async () => {
Expand Down Expand Up @@ -826,6 +836,55 @@ describe('Lambda Utils', () => {

});

describe('isWeightedPayload', () => {
it('should return true for a correctly weighted payload', () => {
const validPayload = [
{ payload: { data: 'foo' }, weight: 5 },
{ payload: { data: 'bar' }, weight: 10 },
];
expect(utils.isWeightedPayload(validPayload)).to.be(true);
});

it('should return false for payload only containing weights (no "payload" property)', () => {
const validPayload = [
{ weight: 5 },
{ weight: 10 },
];
expect(utils.isWeightedPayload(validPayload)).to.be(false);
});

it('should return false for a payload that is not an array', () => {
const invalidPayload = { payload: { data: 'foo' }, weight: 5 };
expect(utils.isWeightedPayload(invalidPayload)).to.be(false);
});

it('should return false for an undefined payload', () => {
const invalidPayload = undefined;
expect(utils.isWeightedPayload(invalidPayload)).to.be(false);
});

it('should return false for an empty array payload', () => {
const invalidPayload = [];
expect(utils.isWeightedPayload(invalidPayload)).to.be(false);
});

it('should return false for an invalid payload array (elements missing weight property)', () => {
const invalidPayload = [
{ payload: { data: 'foo' } },
{ payload: { data: 'bar' }, weight: 10 },
];
expect(utils.isWeightedPayload(invalidPayload)).to.be(false);
});

it('should return false for an invalid payload (elements missing payload property)', () => {
const invalidPayload = [
{ weight: 5 },
{ payload: { data: 'bar' }, weight: 10 },
];
expect(utils.isWeightedPayload(invalidPayload)).to.be(false);
});
});

describe('fetchPayloadFromS3', () => {

it('should fetch the object from S3 if valid URI', async () => {
Expand Down
Loading