Doesn't work with next.js Edge runtime

[etodanik]

I'm writing an app on next.js which I plan to deploy to CloudFlare Pages + Workers. I know there's CloudFlare support, but when running the local development server of next.js, I get the following errors:

Learn More: https://nextjs.org/docs/messages/node-module-in-edge-runtime
    at <unknown> (webpack-internal:///(middleware)/./node_modules/next/dist/esm/server/web/globals.js:33)
    at Object.get (webpack-internal:///(middleware)/./node_modules/next/dist/esm/server/web/globals.js:33:19)
    at eval (webpack-internal:///(middleware)/./node_modules/pg/lib/crypto/utils-webcrypto.js:19:36)
    at (middleware)/./node_modules/pg/lib/crypto/utils-webcrypto.js (file:///home/danny/src/project/.next/server/pages/api/manageUser.js:3097:1)
    at __webpack_require__ (file:///home/danny/src/project/.next/server/edge-runtime-webpack.js:37:33)
    at fn (file:///home/danny/src/project/.next/server/edge-runtime-webpack.js:286:21)
    at eval (webpack-internal:///(middleware)/./node_modules/pg/lib/crypto/utils.js:7:22)
    at (middleware)/./node_modules/pg/lib/crypto/utils.js (file:///home/danny/src/project/.next/server/pages/api/manageUser.js:3108:1)
    at __webpack_require__ (file:///home/danny/src/project/.next/server/edge-runtime-webpack.js:37:33)
    at fn (file:///home/danny/src/project/.next/server/edge-runtime-webpack.js:286:21)
    at eval (webpack-internal:///(middleware)/./node_modules/pg/lib/crypto/sasl.js:3:16)

Would it make sense to generalize CloudFlare Workers detection into a broader Edge detection that also includes node edge runtime?

[brianc]

Yeah it would probably...how does one reproduce this issue locally? If running in node then pg should be using the node pieces.

[etodanik]

Set up a basic next.js app, create an API endpoint, set its' runtime to edge by specifying:
export const runtime = 'edge';

And then assume to use node-postgres from within that route.

CloudFlare Pages requires a next.js app to specify that all backend endpoints will be using the edge runtime.

There's a further complication that NodeEdge runtime isn't the same as CloudFlare workers. I suppose next.js' NodeEdge is emulating what Vercel's edge runtime would look like. But one important detail is that it won't have the ability to run cloudflare:sockets

[muhammedaksam]

I am trying to use Prisma, PrismaPg and Pg.

Error: The edge runtime does not support Node.js 'crypto' module.

Call Stack

Next.js
eval
node_modules\.pnpm\[email protected]\node_modules\pg\lib\crypto\utils-webcrypto.js (21:1)
(middleware)/./node_modules/.pnpm/[email protected]/node_modules/pg/lib/crypto/utils-webcrypto.js
file:///C:/Code/React/nextplate/.next/server/middleware.js (1292:1)

Next.js
eval
node_modules\.pnpm\[email protected]\node_modules\pg\lib\crypto\utils.js (8:3)
(middleware)/./node_modules/.pnpm/[email protected]/node_modules/pg/lib/crypto/utils.js
file:///C:/Code/React/nextplate/.next/server/middleware.js (1303:1)

Next.js
eval
node_modules\.pnpm\[email protected]\node_modules\pg\lib\crypto\sasl.js (2:16)

https://vercel.com/docs/functions/runtimes/edge-runtime#unsupported-apis

I know it's not really an issue related to Pg but I wanted to inform here anyway.

[mlntr]

I am using Next.js and Drizzle, and get the same: Error: The edge runtime does not support Node.js 'crypto' module. Learn More: https://nextjs.org/docs/messages/node-module-in-edge-runtime

[andyjy]

This patch fixes for me. Specifically: prevents the error Error: The edge runtime does not support Node.js 'crypto' module. when using pg under the Next.js Edge runtime locally.

pg+8.11.5.patch:

diff --git a/node_modules/pg/lib/crypto/utils-webcrypto.js b/node_modules/pg/lib/crypto/utils-webcrypto.js
index 0433f01..588066e 100644
--- a/node_modules/pg/lib/crypto/utils-webcrypto.js
+++ b/node_modules/pg/lib/crypto/utils-webcrypto.js
@@ -13,7 +13,10 @@ module.exports = {
  * The Web Crypto API - grabbed from the Node.js library or the global
  * @type Crypto
  */
-const webCrypto = nodeCrypto.webcrypto || globalThis.crypto
+
+// PATCH: Don't break under Next.js Edge runtime production build
+const webCrypto = ("webcrypto" in nodeCrypto ? nodeCrypto.webcrypto : undefined) ?? globalThis.crypto
+
 /**
  * The SubtleCrypto API for low level crypto operations.
  * @type SubtleCrypto

(I opted to use runtime-agnostic Feature Detection for webcrypto on the crypto import rather than try detect the Next.js Edge runtime specifically via typeof EdgeRuntime === "string", which I assume could be a valid alternative - haven't verified)

[janpio]

Quick note from @prisma that this affects some of our users and is being reported to us: prisma/prisma#24430

(This issue has a reproduction, but that includes Prisma, Next.js and next-auth - that is probably a bit more than what you are asking for @brianc. I am pretty sure just trying to use pg in a Next.js middleware should have the same result though.)

We would love to see this fixed and released, so that Prisma users wanting to use pg in Next.js projects with middleware can do so. Let us know if we can help in any way.

[brianc]

k i'll get this looked at today if I have time!

[brianc]

@janpio @andyjy @etodanik would one of y'all be able to provide me w/ more concrete steps to follow to set up a local reproduction? I did npx create-next-app@latest and did NOT use the API router (so the legacy or older form of nextjs). It created a hello api route:

So i tested it at http://localhost:3000/api/hello and it worked. great. then I went into the route file and added

export const runtime = 'edge'

and now its erroring when I hit the route:

So I'm doing something wrong, but not sure what.

This is the full contents of my hello.ts api route file:

// Next.js API route support: https://nextjs.org/docs/api-routes/introduction
import type { NextApiRequest, NextApiResponse } from "next";

type Data = {
  name: string;
};

export const runtime = 'edge'

export default function handler(
  req: NextApiRequest,
  res: NextApiResponse<Data>,
) {
  res.status(200).json({ name: "John Doe" });
}

[brianc]

update: I tried a different next-js app using the app router instead of the old pages/* approach

I just did same steps as above except said "yes" when the setup asked me to use an app router.

Setup looks like this:

my route.ts file:

import * as pg from 'pg'

export const runtime = 'edge'
const pool = new pg.Pool()

export async function GET() {
  const { rows } = await pool.query('SELECT 1 as one')

  return Response.json({ rows })
}

Immediately when I run 'yarn next dev' to start the project it errors out with being unable to import pg-native which is annoying....I've never messed w/ the edge runtime before - it's quite a bit different...so any thoughts on this?

Import trace for requested module:
./node_modules/pg/lib/connection-parameters.js
./node_modules/pg/lib/client.js
./node_modules/pg/lib/index.js
./src/app/api/route.ts
./node_modules/next/dist/build/webpack/loaders/next-edge-app-route-loader/index.js?absolutePagePath=%2FUsers%2Fbmc%2Fsrc%2Fpg-edge-test%2Fsrc%2Fapp%2Fapi%2Froute.ts&page=%2Fapi%2Froute&appDirLoader=bmV4dC1hcHAtbG9hZGVyP25hbWU9YXBwJTJGYXBpJTJGcm91dGUmcGFnZT0lMkZhcGklMkZyb3V0ZSZhcHBQYXRocz0mcGFnZVBhdGg9cHJpdmF0ZS1uZXh0LWFwcC1kaXIlMkZhcGklMkZyb3V0ZS50cyZhcHBEaXI9JTJGVXNlcnMlMkZibWMlMkZzcmMlMkZwZy1lZGdlLXRlc3QlMkZzcmMlMkZhcHAmcGFnZUV4dGVuc2lvbnM9dHN4JnBhZ2VFeHRlbnNpb25zPXRzJnBhZ2VFeHRlbnNpb25zPWpzeCZwYWdlRXh0ZW5zaW9ucz1qcyZyb290RGlyPSUyRlVzZXJzJTJGYm1jJTJGc3JjJTJGcGctZWRnZS10ZXN0JmlzRGV2PXRydWUmdHNjb25maWdQYXRoPXRzY29uZmlnLmpzb24mYmFzZVBhdGg9JmFzc2V0UHJlZml4PSZuZXh0Q29uZmlnT3V0cHV0PSZwcmVmZXJyZWRSZWdpb249Jm1pZGRsZXdhcmVDb25maWc9ZTMwJTNEIQ%3D%3D&nextConfigOutput=&preferredRegion=&middlewareConfig=e30%3D!
 ○ Compiling /api ...
 ⚠ ./node_modules/pg/lib/native/client.js
Module not found: Can't resolve 'pg-native' in '/Users/bmc/src/pg-edge-test/node_modules/pg/lib/native'

I know its trying to webpack the server...which is fun.

[andyjy]

@brianc here you go - reproduction for pages router:
https://github.com/andyjy/mre-pg-nextjs-edge

[brianc]

here you go - reproduction for pages router:

amazing, ty

[brianc]

@andyjy when i applied your patch to the crypto file I now get this:

⨯ Error: The edge runtime does not support Node.js 'net' module.
Learn More: https://nextjs.org/docs/messages/node-module-in-edge-runtime
    at Object.construct (webpack-internal:///(middleware)/./node_modules/next/dist/esm/server/web/globals.js:37:19)
    at getStream (webpack-internal:///(middleware)/./node_modules/pg/lib/stream.js:8:12)
    at new Connection (webpack-internal:///(middleware)/./node_modules/pg/lib/connection.js:19:36)
    at new Client (webpack-internal:///(middleware)/./node_modules/pg/lib/client.js:48:7)
    at handler (webpack-internal:///(middleware)/./src/pages/api/hello.ts:13:16)
    at eval (webpack-internal:///(middleware)/./node_modules/next/dist/esm/server/api-utils/index.js:32:16)
    at eval (webpack-internal:///(middleware)/./node_modules/next/dist/esm/server/lib/trace/tracer.js:115:36)
    at NoopContextManager.with (webpack-internal:///(middleware)/./node_modules/next/dist/compiled/@opentelemetry/api/index.js:2:7062)
    at ContextAPI.with (webpack-internal:///(middleware)/./node_modules/next/dist/compiled/@opentelemetry/api/index.js:2:518)
    at NoopTracer.startActiveSpan (webpack-internal:///(middleware)/./node_modules/next/dist/compiled/@opentelemetry/api/index.js:2:18108)
 ✓ Compiled in 107ms (263 modules)

[andyjy]

@andyjy when i applied your patch to the crypto file I now get this:

ah ok, turns out I am using import { Pool } from "pg" in my app, which the patch above solves for - whereas in the reproduction repo I happened to write import { Client } from "pg" which exposes the new net incompatibility.

Update: 🤦‍♂️ sorry, I realised that while the patch I shared above indeed prevents the Next.js local edge runtime compilation error, I'm not actually executing the pg Client under the Edge runtime locally. So the patch only fixes import, not usage.. (yes I know, facepalm..)

(I remembered I'm actually serving my API routes that use pg under the Node runtime, although my code that references pg is imported - albeit never executed - within some of my Edge runtime routes, which is why I produced the patch.)

Sorry for the incorrect/misleading post previously!

Thinking ahead to whether/how it's possible to get things actually working under the Next.js Edge runtime locally - it would probably(?) require breaking out of Next.js' local Edge Runtime "sandbox" to access node:net locally (i.e. via the Next.js Webpack config). So if this is the case, some tweaks to pg may be necessary but not sufficient alone.

If I get an actually-working prototype then I'll share back.

[janpio]

Tiny PR to @andyjy's repro to show the same problem with a Next.js middleware, the common case where Prisma users run into this: andyjy/mre-pg-nextjs-edge#1

Thinking ahead to whether/how it's possible to get things actually working under the Next.js Edge runtime locally - it would probably(?) require breaking out of Next.js' local Edge Runtime "sandbox" to access node:net locally (i.e. via the Next.js Webpack config). So if this is the case, some tweaks to pg may be necessary but not sufficient alone.

Semi informed comment from the sidelines: Afaik the Cloudflare Workers support for pg had similar challenges, that parts of the internal pg code had to be disabled or skipped to not use APIs that are not available in the CfW environment.

[brianc]

Sorry for the incorrect/misleading post previously!

absolutely no problem! I'll continue to noodle on this....might take a bit of time to do a similar retrofit that the CF stuff did. I'm also about to go out of town for a bit so...hang tight - i'll get there eventually!