To get this Nanite stat visualization on the screen, type NaniteStats into the nearest Unreal viewport console.

This uses a cool piece of technology that Epic built, letting them raster imgui-esque graphics straight from a shader. The system is called ShaderPrint. This totally makes sense, ever since Epic shifted its internal renderer to a more modern GPU driven system, compute shaders and its brethren dominate rendering logic. Shaders aren’t relegated to just vertex transposition and pixel shading. Welcome to the future!.. Welcome to cough ~2015. Grumpy jokes aside, NaniteStats is a console command, and the last time I checked those types of viewport console inputs don’t auto complete (last time I checked being 5.3).

This console command accepts a single argument, that lowercase argument can be one of four thing:

• list - list available stat filters, will only list filters that are being used. For example, if you have VSM disabled then the VirtualShadowMaps won’t be listed. Listed through UE_LOG, not to viewport console.
• primary - specifies that we only care about Nanite stats for the primary view.
• * - specify a custom Nanite stat filter. Like VirtualShadowMaps.
• off

Scene is “Corridor” by Dylan Browne.

That’s it. P.S: when I started writing this article I wanted to make it into a bigger thing exploring ShaderPrint and its implementation, however that would have taken an order of magnitude time longer than just this initial piece. Hope this helps.

A long while ago I submitted a pull request to EpicGames/UnrealEngine’s main branch implementing a new reflected type called FTime.

The Why

Okay, cool… Why wouldn’t I just use a float instead?

What is FTime?

FTime is essentially your normal floating point number, but with a catch! It’s specifically intended to be used with timestamps and any game related time functionality. Because sometimes single precision floating point isn’t enough for your grand massively multiplayer duck game, you need your server to up for a longer period of time without time related calculations going haywire because you’ve hit the limit of your precision. However, sometimes it IS enough and the cost double precision doesn’t really make sense. FTime can switch between double floating point precision and single floating point precision with a simple compile time toggle, assuming you’re using a source engine build.

The Pull Request

The following is the description of the pull request I submitted here.

Objective

This pull request introduces a new UHT compatible type called FTime, with the aim of solving the game time precision on a case by case basis in a similar fashion to LWC’s FReal.

Motivation

Underlying precision motivations

Single precision floating point does not offer enough precision for a long running dedicated server. The default server tick rate of 30 ticks per second (NetServerMaxTickRate’s default, precision: 1.0/30.0) would start experiencing time related issues after 6.1 days.

Source With double precision floating point, the issues would go from appearing after 6.1 days to 8,925,512 years.

Case by case motivations

Nevertheless, outright converting every single usage of game time to a double wouldn’t be ideal. Not all games require double precision floating point time and the benefit from using single precision floating point could be more than marginal. Games that use short-lived worlds and replicate heavily could benefit from single precision. Or single-player mobile games, why increase the underlying memory footprint just because dedicated servers can’t run for more than 6 days? How is that relevant to the target platform?

Design notes

FTime is a simple double/float alias that lives inside of Math/Time.h, it’s included in CoreMinimal.h which covers both programs and games. A new UnrealHeaderTool property called FTimeProperty has been introduced for the corresponding FTime type alias ensuring blueprint/reflection compatibility.

UHT/Code-gen

FTimeProperty traits are generated based on what the current FTime type is, the traits are chosen by a compile time inheritance conditional, picking either the double property trait or the float one. (Sidebar: I opted to the use UE type traits in this case instead of std’s, not sure what official usage stance is anymore.) When it comes to actually generating the underlying property params, I opt to fake a double/float EPropertyGenFlags based on the PropertyDef’s cpp type.

      // Switch between EPropertyGenFlags Double and Float depending on FPropertyTypeTraitsTime parent, which in itself is driven by the udnerlying type of FTime.
      const TCHAR* PropertyGenFlag = PropertyDef.GetCPPType(nullptr, 0) == TEXT("double")
                                       ? TEXT("UECodeGen_Private::EPropertyGenFlags::Double")
                                       : TEXT("UECodeGen_Private::EPropertyGenFlags::Float");

From my testing, works well.

I originally wrote this on top 5.0.1 which doesn’t have the C# UHT version so that has not been implemented at the time of writing. Only old UHT is aware of FTimeProperty.

Engineering impact

Depending on what happens with this pull request, the ability to change the underlying FTime type would require a custom engine fork, which wouldn’t be an issue for most developers interested in this type of change.

Thanks for reading.

P.S: looks like the cherry picks I made from my engine branch have brought back some 5.0.1 code, will address in the morning. EDIT: needs a bit of additional with regards to 0.0f and 0.0 consistency, backwards compat in the DemoNetDriver serializer, a few invalid uses of FTime, and C# UHT.

A long while ago I posted a video demonstrating how useful something like WebAssembly can be for Unreal Engine scripting. If you’re interested: look forward to a future blog post that I will [[link]] back to here when I write a piece exploring said demo.

Patch for 5.0.1 (branch #5.0): https://github.com/MarkJGx/UnrealEngine/pull/1

The Preface:

I wanted to see if Unreal 5.0 could use PhysX without too much blood sweat and tears, and it could. This is more of a hacky proof of concept than an actually solid PhysX integration, but it works.

The How?

This isn’t a perfect re-integration of PhysX specifically when it comes to the inclusion of the headers in PhysXIncludes.h. I opted to go for simply and dirty solution of using the relative include paths instead. I am not a fan of this approach and would like to address when I have some more time. I did verify that the PhysX module does indeed specify a valid include directory and that’s where I left it. I also took the liberty of stripping old references to VS2015 in the physics modules (“PhysicsCore”) but I suspect that might have some issues.

Certain collision primitives on SM’s won’t be setup and won’t work. The underlying collision information is there it just isn’t loaded properly, recreating the collision will fix this. This is just something I haven’t gotten around to fixing due to lack of time.

The Preface

This is NOT a tutorial on how to add custom shading models in general, only a quick UE4 to UE5 transition helper.

The Why?

Unreal Engine 5 creates GBuffer encoding/decoding functions through code generation (ShaderCompiler.h/ShaderGenerationUtil.cpp) instead of hard coding (UE4) said functions. If you managed to port your custom shading model so far and get what you see above, then this blog is for you. Your custom shading model has be introduced into the new GBuffer code generation path.

The Resolution (Code)

Start out by adding your custom shading model to the a struct called FShaderMaterialPropertyDefines in ShaderMaterial.h (in the RenderCore module)

// struct FShaderMaterialPropertyDefines
  uint8 MATERIAL_SHADINGMODEL_SINGLELAYERWATER : 1;
  uint8 MATERIAL_SHADINGMODEL_THIN_TRANSLUCENT : 1;

  // Custom shading model
  uint8 MATERIAL_SHADINGMODEL_CUSTOM_SHADING_MODEL : 1;

What you’re doing is adding a flag to the material property defines struct, indicating whether that shading model is currently active or not.

Next up, navigate to FShaderCompileUtilities::ApplyFetchEnvironment in ShaderGenerationUtil.cpp and the new custom shading model flag to the function.

// FShaderCompileUtilities::ApplyFetchEnvironment(FShaderMaterialPropertyDefines& SrcDefines, FShaderCompilerEnvironment& OutEnvironment)
  FETCH_COMPILE_BOOL(MATERIAL_SHADINGMODEL_THIN_TRANSLUCENT);

    // Custom shading model
  FETCH_COMPILE_BOOL(MATERIAL_SHADINGMODEL_CUSTOM_SHADING_MODEL);

FETCH_COMPILE_BOOL() is a macro that checks if the shading model flag is present in the current compilation environment, if so the shading model flag will be set in the defines struct.

Make sure to match the flag with the the underlying custom shading model pre-processor defines you added before, the macro will turn that flag into a string literal and check the compilation environment against said string.

The underlying flag is added to the environment by GetMaterialEnvironment in MaterialHLSLEmitter.cpp. Fortunately, we don’t have to worry about that, as that’s already done by past you.

Last but not least, we have to mark our custom shading model as relevant to the GBuffer (and in doing so the GBuffer encoding/decoding codegen).

ShaderGenerationUtil.cpp

  // DetermineUsedMaterialSlots (global scope function)

  if (Mat.MATERIAL_SHADINGMODEL_CUSTOM_SHADING_MODEL)
  {
    SetStandardGBufferSlots(Slots, bWriteEmissive, bHasTangent, bHasVelocity, bHasStaticLighting, bIsStrataMaterial);
    // Uncomment if your shading model has custom data.
    // Slots[GBS_CustomData] = bUseCustomData; 
  }

That’s it, UE5 is aware of your custom shading model and will include it as a valid material slot in the GBuffer codegen. Thanks for reading.

The Result

How do you look up a value in a TMap/TSet without the underlying key type?

Let’s say you just have the hash key (uint32 that you can get from GetTypeHash) and nothing else. Do not fret!

The Resolution (Code)

I’ve crafted some helpers util’s to help you find a value inside of any TMap (easily translates to TSet). Here’s how you use TTypeHashOnlyKeyFunc:

// DO NOT USE the TTypeHashOnlyKeyFunc as a actual KeyFunc on your map/set, it is only worth using for anything lookup related.
TMap<FExampleKey, FExampleValue> KeyValueMap;


uint32 HashKey = GetTypeHash(Key);
TTypeHashOnlyKeyFunc<FExampleKey, FExampleValue>::FHashHelper Comparable(HashKey);
FExampleValue* Value = Comparable.FindByHash(KeyValueMap);

if(Value)
{
  // Woohoo
}

template <typename TKeyType, typename TValueType, bool bInAllowDuplicateKeys = false, typename TSetAllocator = FDefaultSetAllocator>
struct TTypeHashOnlyKeyFunc : public TDefaultMapKeyFuncs<TKeyType, TValueType, bInAllowDuplicateKeys>
{
  // Type that is the same size as the key type, the KeyFunc functions expect a key type, so let's just pretend to be one! Only tested with TMap's FindByHash.
  struct FHashHelper
  {
    static constexpr uint32 ByteRemainder = std::max(sizeof(uint32), sizeof(TKeyType)) - sizeof(uint32);

    FHashHelper(const uint32& Key) : Key(Key), RemainderBytes{}
    {
    };

    uint32 Key;
    uint8  RemainderBytes[ByteRemainder];

    TKeyType GetComparable() const
    {
      // We are the key! (not)
      return reinterpret_cast<const TKeyType&>(*this);
    }

    using FTypeHashOnlyMap = TMap<TKeyType, TValueType, TSetAllocator, TTypeHashOnlyKeyFunc<
                                    TKeyType, TValueType, bInAllowDuplicateKeys, TSetAllocator>>;


    TValueType* FindByHash(TMap<TKeyType, TValueType, TSetAllocator>& MapType) const
    {
      // let's pretend you have our KeyFunc handler instead!
      return reinterpret_cast<FTypeHashOnlyMap&>(MapType).FindByHash(Key, GetComparable());
    }
  };

  static_assert(sizeof(FHashHelper) == sizeof(TKeyType), "FHashKey must match the size of the key type!");


  FORCEINLINE static bool Matches(TKeyType A, TKeyType B)
  {
    const uint32 KeyA = reinterpret_cast<FHashHelper&>(A).Key;
    const uint32 KeyB = reinterpret_cast<FHashHelper&>(B).Key;

    return KeyA == KeyB;
  }


  FORCEINLINE static typename TEnableIf<sizeof(FHashHelper) == sizeof(TKeyType), TKeyType>::Type GetSetKey(
    typename TTypeHashOnlyKeyFunc::ElementInitType Element)
  {
    static_assert(sizeof(TKeyType) >= 4, "The key type has to be bigger than 4 bytes!");

    FHashHelper KeyHack(TTypeHashOnlyKeyFunc::GetKeyHash(Element.Key));
    return reinterpret_cast<TKeyType&>(KeyHack);
  }
};

The Explanation

template<typename KeyType, typename ValueType, typename SetAllocator /*= FDefaultSetAllocator*/, typename KeyFuncs /*= TDefaultMapHashableKeyFuncs<KeyType,ValueType,false>*/>
class TMap : public TSortableMapBase<KeyType, ValueType, SetAllocator, KeyFuncs>
//                                                                                              ^                                                                                  
// ------------------------------------------------------------------------->                   |
//                                                                                              right here!

Turns out Epic’s container types let you implement your own KeyFuncs handler(s)! By default TMap’s uses a TDefaultMapHashableKeyFuncs<KeyType,ValueType,false> KeyFunc, so what do the KeyFuncs handlers do? They act as middlemen for the underlying hash map and set containers, offering static compile time functions that containers use to check for:

• key equality using Matches(KeyLHS, KeyRHS)
• key type hash function GetKeyHash(KeyType),
• getting the key from a the key/value pair GetSetKey(Element/* A key value pair*/)

and so on. It is really nice being able to override the default container behavior through a template type argument. Which is especially useful if you want to have different hashing behavior per container without changing the underlying key type.

The Example

For example, if you have a regular old string and didn’t want to pay for a costly string hash function operation, you could implement a more performant hashing function directly into the container type without ever having to make a new string type. Pretty neat, huh?

The Underlying logic

With that out of the way, let me present you with a joke: A map container, a hash key and no one else walks into a bar…

Wait let me try that again: What if you only had a set/map, a hash key you got from wherever and nothing else. You know that the underlying hash key represents a key inside of that map, unfortunately there’s no easy way of looking it up.

You see by default TSet/TMap needs the whole type to look up the correct value, as the hash key simply points to the underlying hash bucket and from then on out it is a O(n) search, which compares each element in the hash bucket chain¹ through a equality operation, calling Matches in the KeyFuncs type. We can get around that limitation though, instead of just using the hash key for the hash bucket lookup, let’s use it for underlying equality check as well. Problem solved! Note: this method isn’t perfect, as you are the mercy of your hash function. If you can somehow manage a collision, the usual equality check will not save you. Although you potentially have 2^32 -1 possible hashes, so that’s alright.

Thanks for reading!

“Build UnrealHeaderTool in debug configuration (it needs to be just ‘Debug’, not ‘Debug Editor’ or anything else). You can just switch you active configuration to debug to do this and build UnrealHeaderTool. Do this after you build your target project and then go to Engine\Programs\UnrealHeaderTool\Saved\Logs\UnrealHeaderTool.log and copy the command line that’s in the log (note you only need the .uproject path and .manifest path) to UnrealHeaderTool’s command line in Visual Studio project properties (Configuration Properties -> Debugging -> Command Arguments). The set UnrealHeaderTool as your startup project and launch it in Visual Studio. You will now be able to debug UHT as well as the plugin (if it’s loaded).” - robert.manuszewski (slightly modified)

Source: https://forums.unrealengine.com/t/scripting-language-extensions-via-plugins/3542/181?u=markjg

Add a new global environment variable with the following name and value:

UE4_GITDEPS_ARGS=--cache=C:\Users\markjg\clones\GitDepsCache --cache-days=180 --cache-size-multiplier=5

The important line here is the --cache=YourLocalCacheDirectoryPathHere, that’s where your local cache will live. This directory can rapidly grow in size, at the time of writing my cache directory takes up around 26GB’s. You’re done!

How does this work? Every time you run the engine’s ./Setup executable, that’ll will invoke the GitDependency tool. Which checks for any invalid (outdated, non-existent) resources and opts to download them. Dependency fetches also happen after certain git operations, specifically on post checkout and post merge (Invoked by git hooks .git\hooks, as post-checkout, post-merge).

This is really painful and tedious on slow connections, you can opt to reduce the redundant downloads by caching the binaries by setting up the UE4_GITDEPS_ARGS environment variable, which as you can guess, passes additional arguments the GitDependency tool. All we care about now is caching, but there’s a few other arguments you can play around with.

PS C:\Users\markjg\clones\UnrealEngineFive> .\Engine\Binaries\DotNET\GitDependencies.exe ?
Invalid command line parameter: ?

Usage:
   GitDependencies [options]

Options:
   --all                         Sync all folders
   --include=<X>                 Include binaries in folders called <X>
   --exclude=<X>                 Exclude binaries in folders called <X>
   --prompt                      Prompt before overwriting modified files
   --force                       Always overwrite modified files
   --root=<PATH>                 Set the repository directory to be sync
   --threads=<N>                 Use N threads when downloading new files
   --dry-run                     Print a list of outdated files and exit
   --max-retries                 Override maximum number of retries per file
   --proxy=<user:password@url>   Sets the HTTP proxy address and credentials
   --cache=<PATH>                Specifies a custom path for the download cache
   --cache-size-multiplier=<N>   Cache size as multiplier of current download
   --cache-days=<N>              Number of days to keep entries in the cache
   --no-cache                    Disable caching of downloaded files

Detected settings:
   Excluded folders: none
   Proxy server: none
   Download cache: C:\Users\markjg\clones\GitDepsCache

Default arguments can be set through the UE4_GITDEPS_ARGS environment variable.

That’s it.