Remanence
ROLE
Quality Director
SDET
DESCRIPTION
Remanence is a 3 Semester, First-Person combat game created with Unreal Engine 5.2 currently being polished for release by a cross-discipline team of 21.
I implemented Functional Testing for gameplay features
and wrote automated performance testing to track optimization progress
YEAR
2024
GENRE
Melee Combat
LANGUAGE
C++
C#
Python
Blueprints
Batch
TOOLS
Jenkins CI/CD
UE 5.2
Unreal Automation
Visual Studio
Batch
PLATFORM
Windows
Functional Testing
Gameplay Tests
Core gameplay functionality is trivial to test but time consuming if a human has a long list of cases to check off.
Automating these tests gives back to the team in two ways.
-
Less time testing, more time implementing
-
More frequent verification of functionality.
_gif.gif)
In-Editor Example pf Player Movement and Checkpoint tests
Performance Testing
Coming into this project I was aware of significant performance issues. This semester of the project is intended for polish. Adding more art and lighting to the level impacts performance further. Monitoring performance gives the whole team data to use while:
-
Set dressing
-
Optimizing poly-counts
-
Lighting
-
Multi-Enemy Encounters
-
Implementing VFX
I implemented per-room performance tests that capture performance metrics. This gives our data resolution and shows use where work needs to be done.
Performance testing is run by my Jenkins Server. Testing on one PC keeps data consistent. Along with capturing framerate, I run Unreal's Profiler to get more data points on what is taking extra time each frame.
Performance data output by per-room tests
Unreal Insights trace of performance testing
Automation
Automated Build Jobs
Nightly Build
While mostly archival, it is helpful in other ways:
-
Daily change log from source-control
-
Archival for playtesting
-
Testing of Shipping Build
-
Bug Tracking
-
Installer Building
Automated Testing
Test automation is the most impactful job that the server does on this project, here is what it does:
-
Runs every 2 hours if work has been committed
-
Functional Gameplay tests
-
Performance Testing
-
Profiling
Debug Build
The Debug job is for exactly it's name. It is setup with parameters to be customizable:
-
Source Control Revision
-
Build Mode, Development | Shipping
-
Tests Groups
-
Installer Compilation
This job is mainly used when a bug is only appearing in a packaged build or installed build. A Source Control Revision parameter lets us make a spread of builds to close in on a commit that caused a bug.
Build Artifacts
An important part of automation is the artifacts. Artifacts get archived in the build result to assist with debugging and data analysis.
Test Output
Performance data and output generated automatically by Unreal is translated to a format readable by Jenkins using a Python script
Unreal Insights Trace
The trace file is artifacted for analysis or comparing traces between revisions
Build Result
The packaged build itself is archived and also deployed to a team-accessible folder
Development Support
Lightmap Render Farm
Similar to my past Unreal Project Isshin, I setup Unreal's Swarm Render farm for distributed lighting builds. ~5x the speed of static lighting builds. The faster artists can see their work, the quicker they can iterate.
~50 cores from 5 PCs at work building light maps for our lighting artist
Jenkins API Emulator
For about a week during the project I was stuck off-site. This meant I had no access to the Jenkins Server. Since I was currently working on a script that accessed Jenkins' REST API, I needed a way to test my code.
That where Emulation comes in. I created a simple webserver in Python that serves the the two pieces of data I needed to access.
When I was able to return on-site all I needed to deploy the script was change the server URL the script referenced.