Do autosprite calcs with CPU, not GLSL

Get rid of vertexSprite_vp.glsl/USE_VERTEX_SPRITE and return to doing
deformvertexes autosprite/autosprite2 vertex calculations on the CPU
(like it was until 2015). This fixes various bugs with autosprite[2]
BSP surfaces. Also it
fixes some particle systems which were broken for unknown reasons.

One problem with the vertex sprite GLSL is that it always applied depth
fade, regardless of whether it was requested. Depth fade means the alpha
will be reduced if there is something close behind the particle within a
certain depth (judging by the z buffer). With autosprite1, this depth
parameter was set equal to the size of the sprite. So it is like a
cube-shaped cloud (which matters when part of the cloud is inside a
wall). The shader-specified fade depth, if any, was ignored. With
autosprite2, the shader used a negative depth parameter, which just
breaks everything. By removing unwanted depth fade for
BSP surfaces, this fixes #997
(non-opaque autosprite2 BSP surfaces do not show up, as seen with the
various flames on KOsAD's metro map).

Also depth fade will no longer be automatically applied to all
particles. I believe we have just 3 Unvanquished shaders configured with
depth fade: acid tube acid, booster effect, and grenade smoke. Any other
particles could potentially look different due to removing depth fade.
So we may need some asset changes to adjust to this change.

Another issue is that USE_VERTEX_SPRITE was not available for lightmap
shaders. This meant that vertex positions were out of sync between
the generic and lightmap shaders, and the lightmap shader failed to render
anything. So this commit fixes #1246 (wrong lighting for autosprite[2]
BSP surfaces with lightmaps, as seen on map "defenxe") by calculating
the final vertex positions before uploading them, which ensures they are
the same for all GLSL shaders used.

With this commit, some particles that were previously not visible are
now rendered, for example:
- ones with the gfx/players/alien_base/green_acid shader
  which are emitted upon evolving, or damaging or destroying an alien
  buildable
- orange glowing "mark" (which is actually a particle) added at impact
  point of several weapons (rifle, shotgun...)
I believe the problem with green_acid must have been that Tess_AddSprite
oriented the triangles backward (CW instead of CCW or vice versa). And
unlike most particle shaders, the acid one fails to specify double-sided
culling, so it would disappear if oriented backward.
The problem with the orange mark (and some other impact particles) is that
it is positioned very close to the surface. The alpha more or less goes
to zero when using depth fade, if you are looking from a direction close
to the impact surface's normal. The impact mark particle was only
visible from tangential angles.

To implement CPU-side autosprite/autosprite2 transformations, I took the
code from an old (~2015) version of the file. But I ended up completely
writing the autosprite2 one.

When autosprites were first implemented in GLSL, there was also a
change in how the polygon is positioned: in the GLSL implementation it
faces towards the viewer, rather than Tremulous' behavior to face
opposite the view direction. I decided that the GLSL behavior is
superior for autosprite2 and reimplemented the CPU version that way (but
you can get the old behavior by setting the r_autosprite2Style cvar).
For autosprite the old behavior seems good enough so it once again faces
opposing the view direction.

This commit makes autosprite2 surfaces work with material system enabled
for the first time, by virtue of rendering them without using the
material system.

Author: slipher

src.cmake

@@ -159,7 +159,6 @@ set(GLSLSOURCELIST
     ${ENGINE_DIR}/renderer/glsl_source/vertexAnimation_vp.glsl
     ${ENGINE_DIR}/renderer/glsl_source/vertexSimple_vp.glsl
     ${ENGINE_DIR}/renderer/glsl_source/vertexSkinning_vp.glsl
-    ${ENGINE_DIR}/renderer/glsl_source/vertexSprite_vp.glsl
     ${ENGINE_DIR}/renderer/glsl_source/blurX_fp.glsl
     ${ENGINE_DIR}/renderer/glsl_source/blurX_vp.glsl
     ${ENGINE_DIR}/renderer/glsl_source/blurY_fp.glsl

src/engine/renderer/Material.cpp

@@ -247,12 +247,12 @@ void UpdateSurfaceDataGeneric3D( uint32_t* materials, Material& material, drawSu
 		gl_genericShaderMaterial->SetUniform_ColorMapBindless( BindAnimatedImage( 0, &pStage->bundle[TB_COLORMAP] ) );
 	}
 
-	bool needDepthMap = pStage->hasDepthFade || shader->autoSpriteMode;
+	bool needDepthMap = pStage->hasDepthFade;
 	if ( needDepthMap ) {
 		gl_genericShaderMaterial->SetUniform_DepthMapBindless( GL_BindToTMU( 1, tr.currentDepthImage ) );
 	}
 
-	bool hasDepthFade = pStage->hasDepthFade && !shader->autoSpriteMode;
+	bool hasDepthFade = pStage->hasDepthFade;
 	if ( hasDepthFade ) {
 		gl_genericShaderMaterial->SetUniform_DepthScale( pStage->depthFadeValue );
 	}
@@ -959,7 +959,7 @@ void MaterialSystem::GenerateWorldCommandBuffer() {
 		}
 
 		shader = shader->remappedShader ? shader->remappedShader : shader;
-		if ( shader->isSky || shader->isPortal ) {
+		if ( shader->isSky || shader->isPortal || shader->autoSpriteMode ) {
 			continue;
 		}
 
@@ -1067,13 +1067,12 @@ void BindShaderGeneric3D( Material* material ) {
 	gl_genericShaderMaterial->SetTCGenEnvironment( material->tcGenEnvironment );
 	gl_genericShaderMaterial->SetTCGenLightmap( material->tcGen_Lightmap );
 	gl_genericShaderMaterial->SetDepthFade( material->hasDepthFade );
-	gl_genericShaderMaterial->SetVertexSprite( material->vertexSprite );
 
 	// Bind shader program.
 	gl_genericShaderMaterial->BindProgram( material->deformIndex );
 
 	// Set shader uniforms.
-	if ( material->tcGenEnvironment || material->vertexSprite ) {
+	if ( material->tcGenEnvironment ) {
 		gl_genericShaderMaterial->SetUniform_ViewOrigin( backEnd.orientation.viewOrigin );
 		gl_genericShaderMaterial->SetUniform_ViewUp( backEnd.orientation.axis[2] );
 	}
@@ -1146,17 +1145,11 @@ void BindShaderScreen( Material* material ) {
 void BindShaderHeatHaze( Material* material ) {
 	// Select shader permutation.
 	gl_heatHazeShaderMaterial->SetVertexAnimation( material->vertexAnimation );
-	gl_heatHazeShaderMaterial->SetVertexSprite( material->vertexSprite );
 
 	// Bind shader program.
 	gl_heatHazeShaderMaterial->BindProgram( material->deformIndex );
 
 	// Set shader uniforms.
-	if ( material->vertexSprite ) {
-		gl_heatHazeShaderMaterial->SetUniform_ViewOrigin( backEnd.orientation.viewOrigin );
-		gl_heatHazeShaderMaterial->SetUniform_ViewUp( backEnd.orientation.axis[2] );
-	}
-
 	gl_heatHazeShaderMaterial->SetUniform_ModelMatrix( backEnd.orientation.transformMatrix );
 	gl_heatHazeShaderMaterial->SetUniform_ModelViewProjectionMatrix( glState.modelViewProjectionMatrix[glState.stackIndex] );
 }
@@ -1184,26 +1177,22 @@ void ProcessMaterialNOP( Material*, shaderStage_t*, drawSurf_t* ) {
 
 // ProcessMaterial*() are essentially same as BindShader*(), but only set the GL program id to the material,
 // without actually binding it
-void ProcessMaterialGeneric3D( Material* material, shaderStage_t* pStage, drawSurf_t* drawSurf ) {
-	shader_t* shader = drawSurf->shader;
-
+void ProcessMaterialGeneric3D( Material* material, shaderStage_t* pStage, drawSurf_t* ) {
 	material->shader = gl_genericShaderMaterial;
 
 	material->vertexAnimation = false;
 	material->tcGenEnvironment = pStage->tcGen_Environment;
 	material->tcGen_Lightmap = pStage->tcGen_Lightmap;
-	material->vertexSprite = shader->autoSpriteMode != 0;
 	material->deformIndex = pStage->deformIndex;
 
 	gl_genericShaderMaterial->SetVertexAnimation( false );
 
 	gl_genericShaderMaterial->SetTCGenEnvironment( pStage->tcGen_Environment );
 	gl_genericShaderMaterial->SetTCGenLightmap( pStage->tcGen_Lightmap );
 
-	bool hasDepthFade = pStage->hasDepthFade && !shader->autoSpriteMode;
+	bool hasDepthFade = pStage->hasDepthFade;
 	material->hasDepthFade = hasDepthFade;
 	gl_genericShaderMaterial->SetDepthFade( hasDepthFade );
-	gl_genericShaderMaterial->SetVertexSprite( shader->autoSpriteMode != 0 );
 
 	material->program = gl_genericShaderMaterial->GetProgram( pStage->deformIndex );
 }
@@ -1286,21 +1275,13 @@ void ProcessMaterialScreen( Material* material, shaderStage_t* pStage, drawSurf_
 	material->program = gl_screenShaderMaterial->GetProgram( pStage->deformIndex );
 }
 
-void ProcessMaterialHeatHaze( Material* material, shaderStage_t* pStage, drawSurf_t* drawSurf ) {
-	shader_t* shader = drawSurf->shader;
-
+void ProcessMaterialHeatHaze( Material* material, shaderStage_t* pStage, drawSurf_t* ) {
 	material->shader = gl_heatHazeShaderMaterial;
 
 	material->vertexAnimation = false;
 	material->deformIndex = pStage->deformIndex;
 
 	gl_heatHazeShaderMaterial->SetVertexAnimation( false );
-	if ( shader->autoSpriteMode ) {
-		gl_heatHazeShaderMaterial->SetVertexSprite( true );
-	} else {
-		gl_heatHazeShaderMaterial->SetVertexSprite( false );
-	}
-
 	material->program = gl_heatHazeShaderMaterial->GetProgram( pStage->deformIndex );
 }
 
@@ -1447,7 +1428,7 @@ void MaterialSystem::GenerateWorldMaterials() {
 		}
 
 		shader = shader->remappedShader ? shader->remappedShader : shader;
-		if ( shader->isSky || shader->isPortal ) {
+		if ( shader->isSky || shader->isPortal || shader->autoSpriteMode ) {
 			continue;
 		}
 
@@ -1807,6 +1788,7 @@ void MaterialSystem::Free() {
 	generatedWorldCommandBuffer = false;
 
 	dynamicDrawSurfs.clear();
+	autospriteSurfaces.clear();
 	portalSurfaces.clear();
 	portalSurfacesTmp.clear();
 	portalBounds.clear();
@@ -1948,6 +1930,17 @@ void MaterialSystem::AddPortalSurfaces() {
 	portalSurfacesSSBO.AreaIncr();
 }
 
+// autosprite[2] is not implemented in material system, draw them old-fashionedly
+void MaterialSystem::AddAutospriteSurfaces() {
+	tr.currentEntity = &tr.worldEntity;
+
+	for ( const drawSurf_t &drawSurf : autospriteSurfaces )
+	{
+		R_AddDrawSurf( drawSurf.surface, drawSurf.shader,
+		               drawSurf.lightmapNum(), drawSurf.fogNum(), drawSurf.bspSurface );
+	}
+}
+
 void MaterialSystem::RenderMaterials( const shaderSort_t fromSort, const shaderSort_t toSort, const uint32_t viewID ) {
 	if ( !r_drawworld->integer ) {
 		return;

src/engine/renderer/Material.h

@@ -98,7 +98,6 @@ struct Material {
 	bool tcGenEnvironment;
 	bool tcGen_Lightmap;
 	bool hasDepthFade;
-	bool vertexSprite;
 	bool alphaTest;
 
 	bool bspSurface;
@@ -217,6 +216,7 @@ class MaterialSystem {
 
 	std::vector<drawSurf_t*> portalSurfacesTmp;
 	std::vector<drawSurf_t> portalSurfaces;
+	std::vector<drawSurf_t> autospriteSurfaces;
 	std::vector<PortalSurface> portalBounds;
 	uint32_t totalPortals;
 	std::vector<shader_t*> skyShaders;
@@ -250,6 +250,7 @@ class MaterialSystem {
 						 const GLuint count, const GLuint firstIndex );
 
 	void AddPortalSurfaces();
+	void AddAutospriteSurfaces();
 	void RenderMaterials( const shaderSort_t fromSort, const shaderSort_t toSort, const uint32_t viewID );
 	void UpdateDynamicSurfaces();
 

src/engine/renderer/gl_shader.cpp

@@ -1597,7 +1597,7 @@ bool GLCompileMacro_USE_REFLECTIVE_SPECULAR::HasConflictingMacros( size_t permut
 {
 	for (const GLCompileMacro* macro : macros)
 	{
-		if ( ( permutation & macro->GetBit() ) != 0 && (macro->GetType() == USE_PHYSICAL_MAPPING || macro->GetType() == USE_VERTEX_SPRITE) )
+		if ( ( permutation & macro->GetBit() ) != 0 && macro->GetType() == USE_PHYSICAL_MAPPING )
 		{
 			//Log::Notice("conflicting macro! canceling '%s' vs. '%s'", GetName(), macro->GetName());
 			return true;
@@ -1612,7 +1612,7 @@ bool GLCompileMacro_USE_VERTEX_SKINNING::HasConflictingMacros( size_t permutatio
 	for (const GLCompileMacro* macro : macros)
 	{
 		//if(GLCompileMacro_USE_VERTEX_ANIMATION* m = dynamic_cast<GLCompileMacro_USE_VERTEX_ANIMATION*>(macro))
-		if ( ( permutation & macro->GetBit() ) != 0 && (macro->GetType() == USE_VERTEX_ANIMATION || macro->GetType() == USE_VERTEX_SPRITE) )
+		if ( ( permutation & macro->GetBit() ) != 0 && macro->GetType() == USE_VERTEX_ANIMATION )
 		{
 			//Log::Notice("conflicting macro! canceling '%s' vs. '%s'", GetName(), macro->GetName());
 			return true;
@@ -1631,7 +1631,7 @@ bool GLCompileMacro_USE_VERTEX_ANIMATION::HasConflictingMacros( size_t permutati
 {
 	for (const GLCompileMacro* macro : macros)
 	{
-		if ( ( permutation & macro->GetBit() ) != 0 && (macro->GetType() == USE_VERTEX_SKINNING || macro->GetType() == USE_VERTEX_SPRITE) )
+		if ( ( permutation & macro->GetBit() ) != 0 && macro->GetType() == USE_VERTEX_SKINNING )
 		{
 			//Log::Notice("conflicting macro! canceling '%s' vs. '%s'", GetName(), macro->GetName());
 			return true;
@@ -1648,20 +1648,6 @@ uint32_t GLCompileMacro_USE_VERTEX_ANIMATION::GetRequiredVertexAttributes() cons
 	return attribs;
 }
 
-bool GLCompileMacro_USE_VERTEX_SPRITE::HasConflictingMacros( size_t permutation, const std::vector< GLCompileMacro * > &macros ) const
-{
-	for (const GLCompileMacro* macro : macros)
-	{
-		if ( ( permutation & macro->GetBit() ) != 0 && (macro->GetType() == USE_VERTEX_SKINNING || macro->GetType() == USE_VERTEX_ANIMATION || macro->GetType() == USE_DEPTH_FADE))
-		{
-			//Log::Notice("conflicting macro! canceling '%s' vs. '%s'", GetName(), macro->GetName());
-			return true;
-		}
-	}
-
-	return false;
-}
-
 bool GLCompileMacro_USE_TCGEN_ENVIRONMENT::HasConflictingMacros( size_t permutation, const std::vector<GLCompileMacro*> &macros) const
 {
 	for (const GLCompileMacro* macro : macros)
@@ -1690,20 +1676,6 @@ bool GLCompileMacro_USE_TCGEN_LIGHTMAP::HasConflictingMacros(size_t permutation,
 	return false;
 }
 
-bool GLCompileMacro_USE_DEPTH_FADE::HasConflictingMacros(size_t permutation, const std::vector<GLCompileMacro*> &macros) const
-{
-	for (const GLCompileMacro* macro : macros)
-	{
-		if ((permutation & macro->GetBit()) != 0 && (macro->GetType() == USE_VERTEX_SPRITE))
-		{
-			//Log::Notice("conflicting macro! canceling '%s' vs. '%s'", GetName(), macro->GetName());
-			return true;
-		}
-	}
-
-	return false;
-}
-
 bool GLCompileMacro_USE_DELUXE_MAPPING::HasConflictingMacros(size_t permutation, const std::vector<GLCompileMacro*> &macros) const
 {
 	for (const GLCompileMacro* macro : macros)
@@ -1967,7 +1939,7 @@ void GLShader::DispatchComputeIndirect( const GLintptr indirectBuffer ) {
 	glDispatchComputeIndirect( indirectBuffer );
 }
 
-void GLShader::SetRequiredVertexPointers( bool vertexSprite )
+void GLShader::SetRequiredVertexPointers()
 {
 	uint32_t macroVertexAttribs = 0;
 
@@ -1980,13 +1952,6 @@ void GLShader::SetRequiredVertexPointers( bool vertexSprite )
 	}
 
 	uint32_t attribs = _vertexAttribsRequired | _vertexAttribs | macroVertexAttribs; // & ~_vertexAttribsUnsupported);
-
-	if ( vertexSprite )
-	{
-		attribs &= ~ATTR_QTANGENT;
-		attribs |= ATTR_ORIENTATION;
-	}
-
 	GL_VertexAttribsState( attribs );
 }
 
@@ -2045,7 +2010,6 @@ GLShader_generic::GLShader_generic( GLShaderManager *manager ) :
 	GLDeformStage( this ),
 	GLCompileMacro_USE_VERTEX_SKINNING( this ),
 	GLCompileMacro_USE_VERTEX_ANIMATION( this ),
-	GLCompileMacro_USE_VERTEX_SPRITE( this ),
 	GLCompileMacro_USE_TCGEN_ENVIRONMENT( this ),
 	GLCompileMacro_USE_TCGEN_LIGHTMAP( this ),
 	GLCompileMacro_USE_DEPTH_FADE( this )
@@ -2077,7 +2041,6 @@ GLShader_genericMaterial::GLShader_genericMaterial( GLShaderManager* manager ) :
 	GLDeformStage( this ),
 	// GLCompileMacro_USE_VERTEX_SKINNING( this ),
 	GLCompileMacro_USE_VERTEX_ANIMATION( this ),
-	GLCompileMacro_USE_VERTEX_SPRITE( this ),
 	GLCompileMacro_USE_TCGEN_ENVIRONMENT( this ),
 	GLCompileMacro_USE_TCGEN_LIGHTMAP( this ),
 	GLCompileMacro_USE_DEPTH_FADE( this ) {
@@ -2620,8 +2583,7 @@ GLShader_heatHaze::GLShader_heatHaze( GLShaderManager *manager ) :
 	u_VertexInterpolation( this ),
 	GLDeformStage( this ),
 	GLCompileMacro_USE_VERTEX_SKINNING( this ),
-	GLCompileMacro_USE_VERTEX_ANIMATION( this ),
-	GLCompileMacro_USE_VERTEX_SPRITE( this )
+	GLCompileMacro_USE_VERTEX_ANIMATION( this )
 {
 }
 
@@ -2652,8 +2614,8 @@ GLShader_heatHazeMaterial::GLShader_heatHazeMaterial( GLShaderManager* manager )
 	u_VertexInterpolation( this ),
 	GLDeformStage( this ),
 	// GLCompileMacro_USE_VERTEX_SKINNING( this ),
-	GLCompileMacro_USE_VERTEX_ANIMATION( this ),
-	GLCompileMacro_USE_VERTEX_SPRITE( this ) {
+	GLCompileMacro_USE_VERTEX_ANIMATION( this )
+{
 }
 
 void GLShader_heatHazeMaterial::SetShaderProgramUniforms( shaderProgram_t* shaderProgram ) {