Defer checking shader compile status

[bchirlsbiodigital]

I wrote my own (much simpler) function to compile shader programs, in which I don't check compile status of individual shaders until linking fails. It then goes back and gets the error status and log of each shader. I had a big batch of 55 shader programs to compile and link that went from 980ms down to 870ms, about 11% faster.

Reference: Don't check shader compile status unless linking fails

...unless you know of some reason there would be a shader error with no linking error?

Here's the code you can use as a reference, but it doesn't include any of your fancy options:

function createProgramInfo(gl: WebGLRenderingContext, shaderSources: string[]) {
	const shaders = shaderSources.map((source, i) => {
		const shaderType = shaderTypes[i];
		const shader = gl.createShader(gl[shaderType]);
		gl.shaderSource(shader, source);
		gl.compileShader(shader);
		return shader;
	});

	const program = gl.createProgram();
	shaders.forEach(shader => {
		gl.attachShader(program, shader);
	});

	gl.linkProgram(program);

	const linked = gl.getProgramParameter(program, gl.LINK_STATUS);
	if (!linked) {
		let compileError = false;
		shaders.forEach((shader, i) => {
			const compiled = gl.getShaderParameter(shader, gl.COMPILE_STATUS);
			if (!compiled) {
				compileError = true;

				const shaderError = gl.getShaderInfoLog(shader);
				const msg = addLineNumbersWithError(gl.getShaderSource(shader)) +
					'\nError compiling: ' + shaderTypes[i] + ':\n' + shaderError;
				console.error(msg);
			}
			gl.deleteShader(shader);
		});

		gl.deleteProgram(program);

		if (!compileError) {
			const linkError = gl.getProgramInfoLog(program);
			console.error(`Error in program linking: ${linkError}`);
		}

		return null;
	}

	return createProgramInfoFromProgram(gl, program);
}

[greggman]

Passing in a callback or using twgl.createProgramInfoAsync already does this. It also uses KHR_parallel_shader_compile if it exists.

I forgot if there was a reason I didn't make it do what you suggest for the non-async path. the code there is kind of a mess at the moment.

You'd get even more time back if you compiled all shaders and linked all programs and only after checked for results for all of them.

[bchirlsbiodigital]

Thanks, yeah I saw that. But right now I need it to be synchronous. I'm looking into batch compiling and then checking as you say; it's a little bit tricky but much less complicated than async if you need scenes to be complete.

[tuner]

I'm using the technique Greg suggested: compile and link everything, then check link status, etc. It provided significant speedup. Highly recommended!

[greggman]

I could add add createPrograms and createProgramInfos that like createTextures takes named info and creates multiple programs, doing it in the optimal way

Something like

const programInfos = twgl.createProgramInfos(gl, {
  flatShading: [ flatShadingVS, flatShadingFS ],
  phongShading: [ phongShadingVS, phongShadingFS ],
  particleCompute: {
    shaders: [ particleVS, particleFS ],
    transformFeedbackVaryings: [ 'velocity', 'position' ],
  },
});

[greggman]

Let me also point out that if you did this

const programInfos = await Promise.all([
  twgl.createPrgramInfoAsync(...),
  twgl.createPrgramInfoAsync(...),
  twgl.createPrgramInfoAsync(...),
  twgl.createPrgramInfoAsync(...),
]);

It would end up compiling and linking all of them before checking any of them.

[greggman]

So I put this in,

createProgram and createProgramInfo now compile both shaders, then link, then check the link status

added createPrograms and createProgramInfos. You pass them an object of names to ProgramSpecs and they return a object of names to WebGLProgram or names to ProgramInfo respectively. They will compile all shaders and link all programs before checking the result of any of them.

As before, all 4 take a callback if you want them to run async and there are async versions that return a promise so you can await on them.

[bchirlsbiodigital]

Thanks again @greggman and @tuner. I'll check out the new code.

Does anybody know if there's any performance benefit to first compiling all the shaders in a scene in one pass and then linking all the programs in a second pass? ...as opposed to compiling and linking each program in a single pass?

[bchirlsbiodigital]

Heads up: it looks like KHR_parallel_shader_compile is not any faster due to implementation issues in chromium and safari. Credit to @mvaligursky for the discovery.